Logo for Arting Digital

Mid-Senior Databricks Engineer

Role overview

Summary

This role focuses on developing and maintaining data pipelines and ETL workflows using Databricks and related technologies. You will collaborate with cross-functional teams to ensure data quality and optimize performance within the data engineering function.

Qualifications

  • 3–6 years of experience in Data Engineering.
  • Strong knowledge of SQL, Python, and PySpark.
  • Hands-on experience with Databricks and Apache Spark.
  • Experience with Delta Lake, Z-Ordering, and VACUUM.
  • Preferred: Knowledge of Apache Airflow or Azure Data Factory.

Responsibilities

  • Develop and maintain scalable data pipelines using Databricks, Python, and PySpark.
  • Build and automate ETL workflows using Delta Live Tables (DLT).
  • Optimize Spark jobs through partitioning, shuffling, and performance tuning.
  • Implement Delta Lake optimization techniques, including Z-Ordering and VACUUM.

Key facts

Hard skills

Key skills for this role

About the company

Arting Digital logo

Arting Digital

  • IT Services & Consulting
  • Scaleup
  • 51–200
Arting Digital is Digital Transformation organization working with client across marketing, analytics and digital transformation. We enable enterprises and marketers take advanced data-driven decisions and boost the effectiveness and success of their digital properties including websites, social media, online platforms, digital devices, the web or mobile-based applications, providing IT resources and more. Combining progressive analytics practices, technology, and domain expertise — our custom solutions help clients connect data and insights with business decisions. Join Our Talent Pool for Job Opportunities! Link: https://forms.gle/AdSTCwnbo2r3sQTx9

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

This is a remote position.

Position: Mid-Senior Databricks Engineer
Location: Remote
Experience: 3–6 Years
Employment Type: Full-Time

Required Skills & Qualifications

  • 3–6 years of experience in Data Engineering.

  • Strong knowledge of SQL, Python, and PySpark.

  • Hands-on experience with Databricks and Apache Spark.

  • Understanding of Spark optimization, partitioning, and shuffling.

  • Experience with Delta Lake, Z-Ordering, and VACUUM.

  • Experience building ETL pipelines using Delta Live Tables (DLT).

  • Knowledge of Apache Airflow or Azure Data Factory.

  • Understanding of data governance and data quality.

Key Responsibilities

  • Develop and maintain scalable data pipelines using Databricks, Python, and PySpark.

  • Build and automate ETL workflows using Delta Live Tables (DLT).

  • Optimize Spark jobs through partitioning, shuffling, and performance tuning.

  • Implement Delta Lake optimization techniques, including Z-Ordering and VACUUM.

  • Manage data orchestration using Airflow or Azure Data Factory.

  • Ensure data quality, governance, and pipeline performance.

  • Collaborate with cross-functional teams to deliver efficient data engineering solutions.




Salary: Open Budget

Related jobs

See all jobs

Other jobs at Arting Digital