Azure Data Engineer (ADF + Strong RDBMS and SQL)
Job description
Your missions
This is a remote position.
Azure Data Engineer
We are looking for a skilled and experienced Azure Data Engineer with expertise in ADF (Azure Data Factory), strong RDBMS and SQL skills, and familiarity with Informatica PWC and Oracle PL/SQL. As an Azure Data Engineer, you will be responsible for understanding and converting Informatica PWC and Oracle PL/SQL code into ADF and Azure SQL. Additionally, you will work on Azure and Databricks "Lakehouse" implementation, translating business requirements into technical solutions for data curation and consumption.
Responsibilities:
Data Curation and Transformation: Translate business requirements into technical solutions for data curation and transformation, specifically using Databricks Lakehouse from a modeling and ELT perspective.
Azure Data Engineering: Develop data pipelines and orchestration using Azure Data Factory and other Azure Modern Data Warehouse platform components.
SQL and Pyspark: Utilize SQL programming and Pyspark for data manipulation and processing tasks.
Azure Services: Work with Azure services such as ADLS (Azure Data Lake Storage), ADX (Azure Data Explorer), and Databricks.
Data Migration: Experience in SQL Server migration to the cloud and handling structured and unstructured datasets.
DevOps and CI/CD: Good knowledge of DevOps tools and processes, and experience in automating data pipelines through a CI/CD delivery methodology.
Databricks Expertise: In-depth hands-on implementation knowledge on Azure Databricks, Databricks Delta Lake, and managing Delta Tables.
Data Modeling: Understand and implement data modeling for the data lake.
Data Transfer and Oracle PL/SQL: Hands-on experience in using secure file transfer tools like Kiteworks, and Oracle PL/SQL knowledge will be a plus.
Effort Estimation: Provide effort estimation for new projects related to data engineering.
Complex Design and Agile Environment: Drive complex design and development efforts in an agile environment.
Qualifications:
Minimum of 5-7 years of experience working on Azure and Databricks "Lakehouse" implementation.
Strong experience in ADF (Azure Data Factory) and SQL as primary skills.
Familiarity with Informatica PWC and Oracle PL/SQL.
Experience in SQL programming and Pyspark.
2 or more years of experience as a Data Engineer, specifically in the Azure Modern Data Warehouse platform.
Good understanding of data modeling and data lake concepts.
Expertise in Databricks, ADLS, ADX, and other Azure services.
Prior experience in SQL Server migration to the cloud.
Knowledge of DevOps tools and processes.
Required profile
Experience
Go Premium: Access the World's Largest Selection of Remote Jobs!
- Largest Inventory: Dive into the world's largest remote job inventory. More than half of these opportunities can't be found on standard platforms.
- Personalized Matches: Our AI-driven algorithms ensure you find job listings perfectly matched to your skills and preferences.
- Application fast-lane: Discover positions where you rank in the TOP 5% of applicants, and get personally introduced to recruiters with Jobgether.
- Try out our Premium Benefits with a 7-Day FREE TRIAL.
No obligations. Cancel anytime.