Logo for Gigster

Senior Databricks Engineer/Architect

Role overview

Summary

This role focuses on designing and optimizing data pipelines and migration strategies to the Databricks platform. You will work with enterprise customers to modernize their data platforms and ensure data quality and performance, bridging the gap between technical teams and business stakeholders.

Qualifications

  • Experience as a Senior Data Engineer or Data Architect.
  • Specialization in designing, developing, and optimizing scalable data pipelines using PySpark and Databricks.
  • Skilled in architecting and executing data migrations to Databricks.
  • Adept at translating complex technical requirements into effective data solutions.
  • Ability to present technical strategies and project outcomes to business stakeholders and leadership.

Responsibilities

  • Design, develop, and maintain scalable, high-performance data pipelines.
  • Architect and lead data migration strategies to the Databricks platform.
  • Develop and optimize complex data transformations using PySpark.
  • Build and enhance ETL/ELT processes to support enterprise data initiatives.

Key facts

  • Remote from: United States
  • Full time
  • Senior (5-10 years)
  • English

Hard skills

Other skills

  • Collaboration
  • Communication
  • Presentations
  • Problem Solving

About the company

Gigster logo

Gigster

  • IT Services & Consulting
  • Startup
  • 11–50
Gigster builds top-tier software development teams. Our AI-powered platform ensures tailor-fit talent matching, accelerated delivery, cost efficiency, and guaranteed project outcomes. Perfect-Fit Talent Matching: Receive tailor-fit talent matching with Gigster’s AI-powered platform based on skillset, past performance, and personality type collected from 10+ years of project data. Guaranteed Outcome: Guaranteed pricing and outcomes for any fully-managed project based on 5,000+ deliverables. Flexibility: Get a fully-managed team with a project manager to orchestrate end-to-end project execution for big projects or elicit on-demand talent to add to your existing team. Elite Talent: Get access to our global talent pool of 50,000+ rigorously-vetted developers, designers, and project managers. Fast Start & Confident Delivery: Proven delivery workflow model for accelerated start and streamlined development for maximum efficiency in execution and project delivery. Founded in 2014, Gigster has completed over 5,000 projects with some of the largest companies in the world. Third party research firm, Constellation Research, completed a study that showed Gigster’s model results in 30% more efficiency in staffing, 60% lower delivery risk, and a 3.6x higher customer satisfaction score than other software development firms.

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Senior Databricks Engineer/Architect

Job Type: 1099 contract, long-term & ongoing

Location: Remote in US

About Virtasant

Virtasant is a global technology services company with a network of over 4,000 technology professionals across 130+ countries.

We help organizations modernize cloud platforms, improve engineering efficiency, and optimize cloud spend through architecture, automation, and FinOps best practices.

Our teams work directly with enterprise customers to identify inefficiencies, drive modernization efforts, and deliver measurable business outcomes.

What We're Looking For

Experienced Senior Data Engineer or Data Architect specializing in designing, developing, and optimizing scalable data pipelines using PySpark and Databricks.

Skilled in architecting and executing data migrations to Databricks, modernizing data platforms, and building robust ETL/ELT workflows that ensure data quality, performance, and reliability.

Adept at translating complex technical requirements into effective data solutions, collaborating with cross-functional teams, and presenting technical strategies, architecture designs, and project outcomes to business stakeholders and leadership.

What You'll Do

  • Design, develop, and maintain scalable, high-performance data pipelines.

  • Architect and lead data migration strategies to the Databricks platform.

  • Develop and optimize complex data transformations using PySpark.

  • Build and enhance ETL/ELT processes to support enterprise data initiatives.

  • Ensure data quality, integrity, scalability, and performance across data platforms.

  • Present technical solutions, migration roadmaps, and project progress to stakeholders.

  • Bridge the gap between technical teams and business stakeholders to align data solutions with organizational objectives.

Related jobs

See all jobs

Other jobs at Gigster