Logo for SSC HR Solutions

Senior Data Engineer (Flink)

Role overview

Summary

This role focuses on building and maintaining real-time streaming pipelines at a high volume. You will work with technologies like Apache Flink and Kafka to ensure efficient data processing and recovery in production environments.

Qualifications

  • Strong hands-on experience with Apache Flink, including DataStream API and Flink SQL.
  • Experience with high-volume Kafka streaming and real-time data processing.
  • Strong knowledge of event time processing, checkpointing, savepoints, and state management.
  • Experience with data modelling for analytics and streaming into a lakehouse.
  • Experience monitoring pipeline health and handling job recovery and failures.

Responsibilities

  • Build and run real-time streaming pipelines.
  • Monitor pipeline health and handle recovery when jobs fail.
  • Process event time, checkpointing, savepoints, and state management in production.
  • Model data for analytics and streaming into a lakehouse.

Key facts

Hard skills

Key skills for this role

About the company

SSC HR Solutions logo

SSC HR Solutions

  • Staffing & Recruiting
  • 51–200
SSC HR Solutions is a recruitment and outsourcing company that supports businesses in their recruitment needs. Serving over 80 companies worldwide prominently featured on the Fortune 500. With more than 5000 successful hires, including 1000 outsourced employees; SSC proactively seeks, nurtures, and cultivates talent within the corporate realm. Our outsourcing services include but are not limited to: payroll, social insurance, labor law consultation, medical insurance, recruitment services and fully equipped office spaces. Contact us if you are pursuing prospective talent for your business or are looking to expand your job search. SSC HR Solutions looks forward to connecting you with your future. If you are interested or have any inquiries, please feel free to reach out to us. Contact: info@sscegypt.net Phone: +20 1025395408

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Builds and runs real time streaming pipelines, keeping latency low and jobs recoverable at high
volume.

  • Strong hands
    on Apache Flink across DataStream API and Flink SQL.
  • Event time processing, checkpointing, savepoints, and state management in
    production.
  • High volume Kafka streaming.
  • Data modelling for analytics and streaming into a lakehouse.
  • Monitors pipeline health and handles recovery when jobs fail.

Requirements

Requirements

pache Flink
(DataStream, Flink SQL), Kafka, Java, Scala, Python, SQL, Docker, Kubernetes

  • Strong hands-on experience with Apache Flink, including DataStream API and Flink SQL.
  • Experience with high-volume Kafka streaming and real-time data processing.
  • Strong knowledge of event time processing, checkpointing, savepoints, and state management.
  • Experience with data modelling for analytics and streaming into a lakehouse.
  • Experience monitoring pipeline health and handling job recovery and failures.
  • Apache Iceberg or other open table formats.
  • Airflow.
  • On premises or self managed cluster experience.

Related jobs

See all Data Engineer jobs

Other jobs at SSC HR Solutions