Logo for Sparq

Senior Data Engineer (Databricks)

Role overview

Qualifications

  • Recent hands-on experience building production batch solutions in Databricks
  • Strong development experience with Python and Apache Spark
  • Advanced SQL skills, including complex query development and performance tuning
  • Experience with Azure data services is preferred

Responsibilities

  • Build and migrate large-scale batch data pipelines from legacy data platforms into Databricks
  • Develop and optimize production-grade solutions using Python, Spark, and advanced SQL
  • Lead technical design conversations and guide implementation within an established architecture
  • Collaborate with developers, architects, and business stakeholders to communicate technical decisions

About the company

Sparq logo

Sparq

IT Services & IT Consulting

Sparq is the leading U.S.-based digital engineering consultancy for building, deploying and supporting digital applications, products and experiences. Sparq assembles trusted software engineering teams that collaboratively solve challenges with creativity and rigor. Our experts know how to work together—with each other and with clients—generating momentum, velocity and success.

Company details

Company typeSME
IndustryIT Services & IT Consulting
Company size501 - 1000

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

At Sparq, we help companies solve the right problems—not just build more technology.


We’re a modern product engineering partner blending strategy, craftsmanship, and speed to help organizations modernize confidently in the age of AI. From data ecosystems to digital products and AI acceleration, we turn complexity into clarity and ideas into impact.


If you’re driven to build what’s next, lead with empathy, and deliver excellence without ego, you’ll feel right at home at Sparq.



C2C is not available

Must be authorized to work in the U.S. without sponsorship

Why you will enjoy Mondays again:

  • Opportunity to work alongside a talented, diverse team in a collaborative and creative environment
  • Ongoing investment in your growth through training, mentorship, and industry certifications
  • Hands-on exposure to cutting-edge technologies across multiple industries
  • Recognition programs and employee awards within an inclusive culture that celebrates great work, including our AI Innovators Program
  • Remote or hybrid work options
  • Competitive salary + bonus opportunities
  • Robust benefits package, matching 401(k) plan, and substantial PTO
  • Tuition reimbursement

A Day in the Life:

  • Build and migrate large-scale batch data pipelines from legacy data platforms into Databricks.
  • Develop and optimize production-grade solutions using Python, Spark, and advanced SQL.
  • Support parallel legacy and cloud data environments throughout the migration and validation process.
  • Serve as a subject matter expert while remaining hands-on with day-to-day development and troubleshooting.
  • Actively use AI tools, including Claude through our Anthropic partnership, to streamline tasks, improve the quality of your work, and share best practices with teammates while continuously seeking new ways to integrate AI into your everyday workflows
  • Lead technical design conversations and guide implementation within an established architecture and project framework.
  • Collaborate with developers, architects, and business stakeholders to communicate technical decisions, risks, and progress.
  • Improve pipeline reliability and performance through testing, monitoring, tuning, and production support.
  • Use GitHub Copilot when helpful to support development productivity.
  • Participate in a rotating on-call schedule approximately every four to six weeks.

What it takes:

  • Recent hands-on experience building production batch solutions in Databricks, ideally within the past 12–18 months.
  • Strong development experience with Python and Apache Spark.
  • Advanced SQL skills, including complex query development and performance tuning.
  • Consultative approach and problem solving skills to successfully align digital solutions with long-term business goals of the client
  • Experience delivering large-scale migrations from legacy data platforms to cloud-based environments, preferably involving Teradata or a comparable platform.
  • Experience supporting legacy and cloud systems operating in parallel during a migration.
  • The ability to lead design and technical-direction discussions while remaining actively involved in coding and implementation.
  • Hands-on experience using AI tools to enhance daily work, or a strong desire to do so, including a willingness to experiment, learn, and champion AI adoption within your role
  • Strong communication skills across technical teams, architects, business partners, and leadership.
  • Availability to participate in an on-call rotation.
  • Experience with Azure data services—such as Data Factory, Synapse Analytics, Event Hubs, Delta Lake, Cosmos DB, and Azure DevOps—is preferred.
  • Exposure to Big Data technologies, NoSQL databases, Git, Jenkins, CI/CD practices, and Agile delivery methodologies is a plus.

Regardless of your specific role, we seek individuals who are excited to explore, adopt, and evangelize AI tools and methodologies. If you have experience in AI or a proven track record of rapidly learning and mentoring others on emerging tech, you’ll fit right in.


Equal Employment Opportunity Policy: Sparq is proud to offer equal employment opportunity without regard to age, color, disability, gender, gender identity, genetic information, marital status, military status, national origin, race, religion, sexual orientation, veteran status, or any other legally protected characteristic.


We are committed to providing equal employment opportunities and believe in an inclusive workplace. If you require reasonable accommodations to participate in the job application or interview process, please let us know by contacting recruiting@teamsparq.com



#LI-REMOTE

Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Data Engineer Related jobs

Other jobs at Sparq

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.