Match score not available

Data Engineer, Content Understanding

EXTRA HOLIDAYS - EXTRA PARENTAL LEAVE

Remote:

Hybrid

Contract:

Full time

Work from:

New York (US)

Offer summary

Qualifications:

Professional data engineering experience with distributed systems, Proficient in Scala, experienced with JVM-based frameworks.

Key responsabilities:

Build large-scale data pipelines on Google Cloud Platform
Optimize data quality and drive improvements
Collaborate with various technical stakeholders

Spotify Music XLarge https://www.lifeatspotify.com/

5001 - 10000 Employees

HQ: Stockholm

See more Spotify offers

Job description

Your missions

As a Software Engineer in our Content Understanding teams, you will help define and build ML deployed at scale in support of a broad range of use cases driving value in media and catalog understanding.

We are looking for engineers who are very enthusiastic about data to focus on building structured, high-quality data solutions. These solutions will be used to evolve our products bringing better experiences to our users and the global artist community alike. We are processing petabytes of data using tools such as BigQuery, Dataflow and Pub/Sub. When needed, we also develop our own data tooling such as Scio, a Scala API for Apache Beam, and Luigi, a Python framework for scheduling.

What You'll Do

Build large-scale batch and real-time data pipelines with data processing frameworks such as Scio, Spark on Google Cloud Platform

Leverage best practices in continuous integration and delivery

Help drive optimisation, testing and tooling to improve data quality

Collaborate with other Software Engineers, ML Engineers, Data Scientists and other stakeholders, taking on learning and leadership opportunities that will arise every single day

Create and maintain metrics datasets as well as dashboards that power data driven decisionsWork in an agile team to continuously experiment, iterate and deliver on new product objectives

Work on machine learning projects powering the experience that suits each user individually

Who You Are

You have professional data engineering experience and you know how to work with high volume, heterogeneous data, preferably with distributed systems such as Hadoop, BigTable, Cassandra, GCP, AWS or Azure

You know Scala language well, and are interested in spreading this knowledge in the team

You have experience with one or more higher-level JVM-based data processing frameworks such as Beam, Dataflow, Crunch, Scalding, Storm, Spark, Flink etc

You might have worked with Docker as well as Luigi, Airflow, or similar tools

You are passionate about crafting clean code and have experience in coding and building data pipelines

You care about agile software processes, data-driven development, reliability, and responsible experimentation

You understand the value of collaboration and partnership within teams

Where You Will Be

For this role you will be based in New York City, USA

The United States base range for this position is $122 716 - $175 308, plus equity. The benefits available for this position include health insurance, six month paid parental leave, 401(k) retirement plan, monthly meal allowance, 23 paid days off, 13 paid flexible holidays. These ranges may be modified in the future.