Team Leadership:
· Lead and mentor a team of data engineers, providing technical guidance and support.
· Set clear objectives and priorities for the team and ensure the successful execution of projects.
· Foster a collaborative and innovative work environment. Engage and collaborate with stakeholders, business leaders and SME’s.
Data Pipeline Development:
· Build and optimize data pipelines for collecting, processing, and storing data from various sources.
· Implement data transformation and enrichment processes to support analytics and reporting.
Data Quality and Governance:
· Implement data quality checks and validation procedures to ensure data accuracy and consistency.
· Define and enforce data governance standards and best practices.
Integration and Collaboration:
· Collaborate with data analysts, data scientists, and other stakeholders to understand data requirements and provide data solutions.
· Integrate data from multiple sources, including databases, APIs, and external data providers.
Technology Stack:
· Stay up-to-date with the latest data engineering technologies and best practices.
· Select and manage the appropriate tools and technologies for data processing and storage, such as databases, data lakes, and ETL frameworks.
· Hands-on experience managing / working with Informatica, Snowflake, Java, Python, and cloud technologies is desired.
Performance Optimization:
· Monitor and optimize data pipelines for performance, reliability, and cost-effectiveness.
· Troubleshoot and resolve data-related issues as they arise.
Documentation and Knowledge Sharing:
· Maintain thorough documentation of data processes, architectures, and workflows.
· Promote knowledge sharing within the team and across the organization.
Qualifications:
· Bachelor's or Master's degree in computer science, data engineering, or a related field.
· Proven experience in data engineering with a focus on ETL, data modeling, and data pipeline development.
· Strong programming skills in languages such as Python, Java, or Scala.
· Expertise in working with data storage solutions like databases (SQL and NoSQL), data lakes, and cloud-based storage.
· Familiarity with big data technologies and cloud platforms (e.g., AWS, Azure, GCP).
· Experience with cloud-based ETL solutions (e.g., AWS Glue, Azure Data Factory).
· Leadership and team management experience.
· Strong problem-solving and communication skills.
· Understanding of data governance and data quality best practices.
Optional Qualifications:
· Relevant certifications in data engineering or cloud platforms.
· Experience with real-time data processing and streaming technologies.