Strong knowledge of building Azure data pipelines.
Hands-on experience building data pipelines using Databricks and scheduling Databricks jobs.
Experience in Azure networking and security principles.
Deep understanding of Bastion networking, including private link services and private endpoints setup.
Requirements:
Build and maintain Azure data pipelines using Databricks and schedule Databricks jobs.
Design and implement secure networking configurations and policies (Bastion, private link services, private endpoints, VNET peering) for remote resources.
Design and deliver infrastructure using Terraform and manage Azure DevOps agents.
Set up Azure Data Lake Storage Gen2 with RBAC and create DBFS mount points to enforce access control for end users.
Job description
DataBricks Developer/Consultant Location: Remote 100% Duration: 06 Month plus contract Rate: DOE
Skills:
Strong knowledge of building Azure data pipelines.
Hands on experience of building data pipelines using Databricks and scheduling databricks jobs.
Experience in Azure networking and security principles .
Deep understanding of Bastion networking process such as private link services and private endpoints setup.
Hands on experience designing and delivering solutions using Terraform and Azure DevOps agents.
Experience setting up Azure Data Lake Storage(ADLS GEN2) using Role Based Access mechanism.
Creating mount points for ADLS Gen2 storage in DBFS to implement RBAC for end users.
VNET Peering and Private link connectivity for remote subscription resources .
Experience working with structured and unstructured data including imaging & geospatial data.
Microsoft Azure Databricks and Azure Data Engineer certification will be an added advantage.
