Logo for PeerIslands

Data Pipeline Engineer

PeerIslands

Role overview

Qualifications

  • 7+ years of experience in Data Engineering / Data Pipeline development
  • Strong experience with Apache Kafka (Producers, Consumers, Replay, DLQ, Exactly-once/Idempotent processing)
  • Strong SQL and ETL/ELT fundamentals
  • Hands-on experience in Java and/or Python

Responsibilities

  • Design, build, and maintain scalable batch and real-time data ingestion pipelines
  • Develop and manage Bronze, Silver, and Gold (Medallion) data layers
  • Implement CDC, watermarking, checkpointing, and batch-to-stream data processing
  • Build robust data quality, validation, reconciliation, and monitoring frameworks

About the company

PeerIslands logo

PeerIslands

Company details

Links

Website LinkedIn See all jobs

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Location: India
Employment Type: Full-Time
Experience: Senior (7+ Years)

Roles & Responsibilities

  • Design, build, and maintain scalable batch and real-time data ingestion pipelines.
  • Develop and manage Bronze, Silver, and Gold (Medallion) data layers.
  • Implement CDC, watermarking, checkpointing, and batch-to-stream data processing.
  • Build robust data quality, validation, reconciliation, and monitoring frameworks.
  • Develop identity resolution, deduplication, and Golden Record (MDM) solutions.
  • Create and maintain source-to-canonical data mappings and crosswalks.
  • Ensure schema validation, versioning, and data contract enforcement.
  • Collaborate with cross-functional teams to onboard new data sources and optimize data pipelines.

Mandatory Skills

  • 7+ years of experience in Data Engineering / Data Pipeline development.
  • Strong experience with Apache Kafka (Producers, Consumers, Replay, DLQ, Exactly-once/Idempotent processing).
  • Strong SQL and ETL/ELT fundamentals.
  • Hands-on experience in Java and/or Python.
  • Experience with CDC, Batch & Streaming pipelines, and Medallion/Lakehouse architecture.
  • Experience implementing Data Quality, Validation, and Reconciliation frameworks.
  • Knowledge of Master Data Management (MDM), Identity Resolution, Deduplication, and Golden Record concepts.
  • Experience with source-to-target mapping, canonical data models, YAML/JSON configuration, and Git.

Good to Have

  • Experience with probabilistic record matching and record linkage.
  • Schema Registry (Avro/Protobuf).
  • Experience extracting data from legacy/Mainframe systems.
  • Financial reconciliation experience.
  • Healthcare or Benefits Administration domain experience.

Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Data Engineer Related jobs

Other jobs at PeerIslands

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.