Offer summary

Qualifications:

Proficient in Python and SQL for data manipulation and analysis., Experience with PySpark and familiarity with Databricks for data processing., Minimum of 3 years in data engineering or related fields with a proven track record., Strong communication skills and problem-solving abilities..

Key responsabilities:

Design, develop, and maintain scalable data pipelines for large data volumes.

Collaborate with data scientists and analysts to deliver high-quality datasets.

Implement data quality checks and optimize data storage processes.

Participate in Agile development processes and document data workflows.

Job description

Job Summary

We are seeking a skilled Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data processing and analytics, with expertise in Python and SQL. You will be responsible for designing, building, and maintaining scalable data pipelines and ensuring the integrity and availability of data for analysis. This role requires collaboration with data scientists, analysts, and other stakeholders to support data-driven decision-making across the organization.

Job Responsibilities

Design, develop, and maintain robust data pipelines to process large volumes of data from various sources.
Collaborate with data scientists and analysts to understand data requirements and deliver high-quality datasets.
Optimize data storage and retrieval processes to improve performance and efficiency.
Implement data quality checks and monitoring to ensure the accuracy and reliability of data.
Work with cloud-based technologies, particularly Databricks, to manage and analyze data.
Participate in Agile development processes, including sprint planning and retrospectives using Jira.
Document data processes, architectures, and workflows for future reference and compliance.
Stay updated with industry trends and best practices in data engineering and analytics.

Requirements

Proficient in Python as a core programming language for data manipulation and analysis.
Strong knowledge of SQL for querying databases and performing data transformations.
Experience with PySpark for processing large datasets in a distributed computing environment.
Familiarity with Databricks is a plus, enabling efficient data processing and collaboration.
Understanding of data warehousing concepts and ETL processes.
Experience working in Agile environments, particularly with Scrum methodologies (not as a Scrum Master).
Proficient in using Jira for project management and tracking progress.
Minimum of 3 years of experience in data engineering or related fields.
Proven track record of building and maintaining data pipelines in production environments.
Excellent communication skills to effectively collaborate with cross-functional teams.
Strong problem-solving abilities and attention to detail.