Logo for Cuculus GmbH

Senior Data Engineer – Integration Hub & Data Pipelines

Cuculus GmbH

Key Facts

Remote From: 
India
Full time
English

Hard Skills

Apache Airflow Apache NiFi Apache Kafka Apache Spark Data Engineering Linux Observability High Availability Design Continuous Delivery Batch Processing Application Programming Interface (API) Data Warehouse Architectures Data Consistency Relational Databases ETL (Extract Transform Load) Integration Platform As A Service (IPaaS) Data Lakes Data Security Data Integration Access Controls Data Validation Stream Processing SQL (Programming Language) Fault Tolerance Operational Data Store Data Governance Data Modeling Data Duplication Management Python (Programming Language) Data Quality Platform Design And Development CI/CD Well Logging Auditing Java (Programming Language) Data Collection Performance Systems Analysis Git (Version Control System) PostgreSQL Automated Information Systems Data Transformation Scalability Integration Platforms Compliance Management Scala (Programming Language) Project Coordination Metadata Management Microsoft Message Queuing Google Cloud Platform (GCP) Microsoft Azure Apache Parquet System Monitoring Kubernetes Observability AWS Cloud Services Docker (Software) Apache Iceberg Debugging Technical Solution Design Stakeholder Engagement

Other Skills

  • •
    Communication
  • •
    Collaboration
  • •
    Teamwork
  • •
    Mentorship
  • •
    Problem Solving

Roles & Responsibilities

  • 5+ years of hands-on experience as a Data Engineer or in a similar role.
  • Proven track record as an individual contributor on at least three end-to-end data engineering projects, from design to production.
  • Strong hands-on experience with Apache Airflow, Apache NiFi, Apache Spark, Apache Kafka, and PostgreSQL, with extensive data integration from APIs, files, and databases.
  • Strong SQL and data modeling skills; proficient in Python and/or Java/Scala; experience with Linux environments and Git; demonstrated problem-solving and performance-tuning abilities.

Requirements:

  • Design, build, and maintain robust ETL/ELT data pipelines for batch and streaming workloads, and implement data ingestion and transformation workflows using Airflow, NiFi, Spark, and Kafka.
  • Develop and operate a centralized data integration hub that supports multiple upstream and downstream systems, building reusable modular components with high availability, fault tolerance, and observability.
  • Design and manage data warehouses, data lakes, and operational data stores, applying appropriate data modeling, schema evolution, metadata, and versioning.
  • Implement data validation, monitoring, security controls, logging and alerting; establish CI/CD processes for data workflows and automate operational tasks.

Job description

Shape the utilities market of the future with us!
We are looking for an experienced Senior Data Engineer to design, build, and operate a scalable integration hub and data pipeline platform using modern open-source technologies. This role is hands-on and suited for an individual contributor who has delivered multiple end-to-end data engineering projects, from requirements through production deployment and operations.
You will play a critical role in enabling reliable data movement and integration across a diverse ecosystem of third-party systems, APIs, files, databases, and software platforms.
What is the role about?

Key Responsibilities


Data Pipeline Design & Development

  • Design, build, and maintain robust ETL/ELT data pipelines for batch and streaming workloads.
  • Implement data ingestion and transformation workflows using Apache Airflow, Apache NiFi, Apache Spark, and Kafka.
  • Integrate data from multiple sources including REST APIs, files, relational databases, message queues, and external SaaS platforms.
  • Optimize pipelines for performance, scalability, reliability, and cost efficiency.

Integration Hub & Platform Engineering

  • Develop and operate a centralized data integration hub that supports multiple upstream and downstream systems.
  • Build reusable, modular integration components and frameworks.
  • Ensure high availability, fault tolerance, and observability of data workflows.

Data Infrastructure & Storage

  • Design and manage data warehouses, data lakes, and operational data stores using PostgreSQL and related technologies.
  • Implement appropriate data modeling strategies for analytical and operational use cases.
  • Manage schema evolution, metadata, and versioning.
     
     

Data Quality, Security & Governance

  • Implement data validation, monitoring, and reconciliation mechanisms to ensure data accuracy and completeness.
  • Enforce data security best practices, access controls, and compliance with internal governance policies.
  • Establish logging, alerting, and auditability across pipelines.

Automation & Operations

  • Automate data workflows, deployments, and operational processes to support scale and reliability.
  • Monitor pipelines proactively and troubleshoot production issues.
  • Improve CI/CD practices for data engineering workflows.

Collaboration & Stakeholder Engagement

  • Work closely with data scientists, analysts, backend engineers, and business stakeholders to understand data requirements.
  • Translate business needs into technical data solutions.
  • Provide technical guidance and best practices across teams.
Required skills and qualifications
  • 5+ years of hands-on experience as a Data Engineer or in a similar role.
  • Proven experience as an individual contributor on at least three end-to-end data engineering projects, from design to production.
  • Strong hands-on experience with:
    • Apache Airflow / Dagster
    • Apache NiFi
    • Apache Spark
    • Apache Kafka
    • PostgreSQL 
  • Extensive experience integrating data from APIs, files, databases, and third-party systems.
  • Strong SQL skills and experience with data modeling.
  • Solid programming experience in Python and/or Java/Scala.
  • Experience with Linux environments and version control systems (Git).
  • Strong problem-solving, debugging, and performance-tuning skills.
Preferred skills and qualifications
  • Experience with cloud platforms (AWS, Azure, or GCP).
  • Experience with containerization and orchestration (Docker, Kubernetes).
  • Knowledge of data lake technologies and formats (Parquet, ORC, Iceberg, Delta Lake).
  • Familiarity with monitoring and observability tools for data pipelines.
  • Experience working in fast-paced or startup environments.
Apply for this job
About us
Cuculus is the key to providing utilities to all, while protecting the world’s precious resources. Jointly with our international partner network, we provide cutting-edge software and technology solutions to address utility challenges now and in the future. Cuculus will never tire of creating innovative technology and services that enable utilities and organisations to successfully transition to a new era of providing and managing electricity, water, and gas. The work we do is important for individuals, cities, and entire nations. Our work is serious, but we have fun with it, too. 

      
Ready to apply?
APPLY
·

Data Engineer Related jobs

IndiaData Engineer

Other jobs at Cuculus GmbH

We help you get seen. Not ignored.

We help you get seen faster — by the right people.

🚀

Auto-Apply

We apply for you — automatically and instantly.

Save time, skip forms, and stay on top of every opportunity. Because you can't get seen if you're not in the race.

✨

AI Match Feedback

Know your real match before you apply.

Get a detailed AI assessment of your profile against each job posting. Because getting seen starts with passing the filters.

Upgrade to Premium. Apply smarter and get noticed.

Upgrade to Premium

Join thousands of professionals who got noticed and hired faster.