Requirements
Job Description:
We are seeking a skilled and motivated Data Engineer to join our team. The ideal candidate
will have expertise in designing, building, and maintaining robust data architectures across
multi-cloud environments. You will work closely with data scientists, business experts, and
IT teams to deliver scalable analytics solutions that drive business value
.
Key Responsibilities:
1. Cloud Architecture & Analytics
o Design and implement multi-cloud, mixed, and cloud-agnostic data and
analytics architectures.
o Build and manage scalable data pipelines on platforms like AWS and Azure
Databricks.
o Automate data preparation and integration tasks to streamline processes
and enhance productivity.
2. Data Integration & ETL Development
o Develop standards, data mappings, and ETL functions using various tools
and programming languages.
o Design and implement high-volume data ingestion and real-time streaming
pipelines.
o Work with IT teams to model and integrate application data structures into
data warehouses or data lakes.
3. Collaboration & Prototyping
o Collaborate with data scientists and business experts to develop, validate,
and operationalize analytics solutions.
o Partner with Data Science and Data Governance teams to optimize models
for data quality, security, and governance.
4. Testing & Monitoring
o Develop and maintain test cases, run test plans, and operationalize test
scenarios.
o Implement automated testing and continuous monitoring for data
architectures and pipelines.
INTERNAL
5. Infrastructure & Best Practices
o Renovate and modernize data management infrastructure to support
automation and track data consumption effectively.
o Establish and promote best practices for analytics, production data
architectures, and pipeline development.
Preferred Skills:
• Proven experience in implementing multi-cloud or cloud-agnostic data
architectures.
• Proficiency in data engineering tools and frameworks (e.g., Databricks, Apache
Spark, AWS Glue, Azure Data Factory).
• Strong programming skills in Python, SQL, or other relevant languages.
• Experience with ETL processes, data modeling, and real-time data streaming.
• Familiarity with automated testing and monitoring for data systems.
• Excellent collaboration skills with cross-functional teams, including data science
and IT teams.
Preferred Qualifications:
• Knowledge of data governance principles and frameworks.
• Hands-on experience with data lakes and data warehouses (e.g., Snowflake,
Redshift, Synapse).
• Certification in cloud platforms such as AWS, Azure, or GCP.
• Experience with DevOps practices for data engineering.
