Match score not available

Unlock Premium Benefits Today!
🚀 Elevate your career with priority access.
Stand out to top recruiters with Jobgether Priority Access:
  • ✅ Top Match: We help you identify job offers where you're a perfect fit.
  • ✅ Exclusive Referral Service: Connect directly with recruiters as a Jobgether Certified Premium User.
  • ✅ Personalized Feedback: Receive feedback and expert guidance on applications, and have your profile manually reviewed.

Start Your Free Trial Now!

Start my free trial

Senior Data Engineer, DP Team (Remote, International/Non-U.S.)

Remote: 
Full Remote
Contract: 
Full time
Experience: 
Senior (5-10 years)
Work from: 
Israel

Offer summary

Qualifications:

5+ years of data engineering experience, Fluency in Python and SQL, Experience in Scala/Java is a plus, Strong understanding of RDBMS and query optimization, Knowledge of distributed production systems.

Key responsabilities:

  • Design and maintain scalable data processing systems
  • Optimize jobs for efficient resource usage
  • Monitor and ensure data quality across systems
  • Collaborate with analysts and build tools and datasets
  • Mentor junior team members
PulsePoint logo
PulsePoint Marketing & Advertising SME https://www.pulsepoint.com/
51 - 200 Employees
About PulsePoint
PulsePoint is an advanced data and technology company that is breaking new ground with programmatic solutions for healthcare organizations, to personalize information and improve outcomes. We are leading a revolution in radical health personalization to help organizations improve how they provide care, education and support to people when they need it most. Through programmatic automation, we interpret the hard to read signals of the customer journey and understand the connection points between relevance and engagement. This enables us to deliver a suite of tools that allow for the seamless activation of health options.
See more PulsePoint offers

Job description

Description

A bit about us:
PulsePoint is a leading healthcare ad technology company that uses real-world data in real-time to optimize campaign performance and revolutionize health decision-making. Leveraging proprietary datasets and methodology, PulsePoint targets healthcare professionals and patients with an unprecedented level of accuracy—delivering unparalleled results to the clients we serve. The company is now a part of Internet Brands, a KKR portfolio company and owner of WebMD Health Corp.
Sr. Data Engineer
PulsePoint Data Engineering team plays a key role in our technology company that’s experiencing exponential growth. Our data pipeline processes over 80 billion impressions a day (> 20 TB of data, 200 TB uncompressed). This data is used to generate reports, update budgets, and drive our optimization engines. We do all this while running against tight SLAs and provide stats and reports as close to real-time as possible.
The most exciting part about working at PulsePoint is the enormous potential for personal and professional growth. We are always seeking new and better tools to help us meet challenges such as adopting proven open-source technologies to make our data infrastructure more nimble, scalable and robust. Some of the cutting-edge technologies we have recently implemented are Kafka, Spark Streaming, Presto, Airflow, and Kubernetes.
What you'll be doing:
  • Design, build, and maintain reliable and scalable enterprise-level distributed transactional data processing systems for scaling the existing business and supporting new business initiatives
  • Optimize jobs to utilize Kafka, Hadoop, Presto, Spark, and Kubernetes resources in the most efficient way
  • Monitor and provide transparency into data quality across systems (accuracy, consistency, completeness, etc)
  • Increase accessibility and effectiveness of data (work with analysts, data scientists, and developers to build/deploy tools and datasets that fit their use cases)
  • Collaborate within a small team with diverse technology backgrounds
  • Provide mentorship and guidance to junior team members
Team Responsibilities:
  • Ingest, validate and process internal & third party data
  • Create, maintain and monitor data flows in Python, Spark, Hive, SQL and Presto for consistency, accuracy and lag time
  • Maintain and enhance framework for jobs(primarily aggregate jobs in Spark and Hive)
  • Create different consumers for data in Kafka using Spark Streaming for near time aggregation
  • Tools evaluation
  • Backups/Retention/High Availability/Capacity Planning
  • Review/Approval - DDL for database, Hive Framework jobs and Spark Streaming to make sure they meet our standards
Technologies We Use:
  • Python - primary repo language
  • Airflow/Luigi - for job scheduling
  • Docker - Packaged container image with all dependencies
  • Graphite - for monitoring data flows
  • Hive - SQL data warehouse layer for data in HDFS
  • Kafka - distributed commit log storage
  • Kubernetes - Distributed cluster resource manager
  • Presto/Trino - fast parallel data warehouse and data federation layer
  • Spark Streaming - Near time aggregation
  • SQL Server - Reliable OLTP RDBMS
  • Apache Iceberg
  • GCP - BigQuery for performance, Looker for dashboards
Requirements
  • 5+ years of data engineering experience
  • Fluency in Python and SQL
  • Experience in Scala/Java is a plus (Polyglot programmer preferred!)
  • Proficiency in Linux
  • Strong understanding of RDBMS and query optimization;
  • Passion for engineering and computer science around data
  • East Coast U.S. hours 9am-6pm EST preferred, but we can be flexible as long as you can work until 12pm/1pm EST; you can work fully remotely
  • Knowledge and exposure to distributed production systems i.e Hadoop
  • Knowledge and exposure to Cloud migration (AWS/GCP/Azure) is a plus
Selection Process:
1) Recruiter Screen (30 mins)
2) Hiring Manager Interview (45 mins)
3) Tech challenge take-home
4) SQL & Python Interview (60 minutes)
5) Team Interviews (60 mins + 3 x 45 mins) + SVP of Engineering (15 mins)
6) WebMD Sr. Director, DBA (30 mins)
WebMD and its affiliates is an Equal Opportunity/Affirmative Action employer and does not discriminate on the basis of race, ancestry, color, religion, sex, gender, age, marital status, sexual orientation, gender identity, national origin, medical condition, disability, veterans status, or any other basis protected by law.  

 

Required profile

Experience

Level of experience: Senior (5-10 years)
Industry :
Marketing & Advertising
Spoken language(s):
French
Check out the description to know which languages are mandatory.

Hard Skills

Graphite (Software)Scala (Programming Language)Java (Programming Language)Job Scheduling (Inventory Management)Data Quality AssessmentLinuxKubernetesgRPCCloud MigrationPython (Programming Language)Apache AirflowApache KafkaSparkrDocker (Software)Apache HadoopApache HiveRelational Database Management SystemsApache IcebergData EngineeringSQL (Programming Language)

Other Skills

  • Collaboration
  • Mentorship
Are you interested?

Data Engineer Related jobs

See more Data Engineer jobs
Sign in or sign up with Google or LinkedIn