Offer summary

Qualifications:

Bachelor's degree in computer science or related field., 5+ years of experience in a similar role, leading data engineers., Extensive experience in agile project environments., Proficiency in Python or PySpark and Azure Data services..

Key responsabilities:

Develop and provide solutions on Big Data and web applications.

Migrate existing pipelines to Cloud using ADF and Databricks.

Develop new data extraction pipelines from multiple sources.

Engineer scalable analytics solutions on Azure for client projects.

Job description

Role description:

•Develop and provide solutions on Big Data;

• Develop high-traffic, flawless web applications using Python, Cloud platform, and PySpark;

• Code with performance, scalability, and usability in mind;

• Work on new tools in leading industry trends, with new and emerging technologies, prototypes and engineering process improvements;

• Migrate existing pipelines developed in ADF and python scripts from On-premise to Cloud using ADF and Databricks;

• Develop new pipelines using ADF for data extraction from multiple sources (MySQL databases;

• Oracle databases, CSV files) and Databricks (PySpark & SparkSQL) for curation, validation and applying required business transformation logic;

• Work on client projects to deliver Microsoft Azure based Data engineering & Analytics solutions;

• Engineer and implement scalable analytics solutions on Azure.

Qualifications:

• Bachelor's degree in computer science or related field

• 5+ years of experience in a similar role; leading fellow data engineers within the data engineering area;

• Extensive experience working in agile project environments;

• Working experience with Python or PySpark is required;

• Working experience and understanding of Azure Data-bricks, Azure Data Factory, Azure Data Lake, Azure SQL DW, and Azure SQL is required;

• The ability to apply such methods to solve business problems using one or more Azure Data and Analytics services in combination with building data pipelines, data streams, and system integration;

• Experience in driving new data engineering developments (e.g. applying new cutting-edge data engineering methods to improve the performance of data integration, using new tools to improve.

Required profile