Strong experience with configuration and administration of Azure data platforms
Requirements:
Provide code review and facilitate code management for engineering team
Enable data governance capabilities in data engineering lifecycle
Integrate with heterogeneous sources using linked services
Develop PySpark notebooks in Azure databricks cluster
Job description
Position: Azure Data Engineer Location: Chicago, IL Duration: 6-12 Months 100 % Remote
Over 10 years of IT experience
4+ recent years in Azure stack
4+ years of experience with DataOps & CI/CD practices to enable automated building, testing, and deployment of data products and applications
2+ years of experience in integrating with heterogenous sources using linked services.
2+ years of experience enabling data governance capabilities and configuration in data engineering lifecycle (data privacy, data security & compliance, and data quality)
2+ years integrating and nuances around maintaining GitHub and GitHub actions.
2+ recent years in ADF creating pipelines with several activities
2+ recent years of experience developing PySpark notebooks in Azure databricks cluster
1+ years of hardcore experience with SparkSQL
1+ years of relevant configuration experience in unity catalog integration and use
Strong experience with configuration and administration of Azure data platforms (Azure Data Factory, Azure Databricks, etc)
Provide code review and facilitate code management for engineering team to ensure best practices and performance standards
4 years in SDLC agile and using CI/ CD pipeline using Github actions.
1+ years in deployment and post deployment activities in production.
1+ year helping withstanding the infrastructure stack and linked services with various sources.
From: Arizona (USA), California (USA), District of Columbia (USA), Florida (USA), Georgia (USA), Indiana (USA), Massachusetts (USA), Michigan (USA), New York (USA), North Carolina (USA), Oregon (USA), Pennsylvania (USA), Texas (USA), Virginia (USA), Washington (USA) (Full Remote)
