Logo for Yadimen Consultancy Ltd

Data Engineer — PySpark + AWS Glue

Role overview

Qualifications

  • Strong hands-on experience with PySpark and AWS Glue
  • Proven track record in ETL pipeline development and optimisation
  • Experience orchestrating workflows with AWS Step Functions
  • Proficiency in serverless development using AWS Lambda (Python)

Responsibilities

  • Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue
  • Build and optimise data ingestion, transformation, and loading workflows
  • Orchestrate data workflows using AWS Step Functions
  • Ensure pipeline reliability, monitoring, and performance optimisation

Key facts

Hard skills

About the company

Yadimen Consultancy Ltd logo

Yadimen Consultancy Ltd

Computer Software / SaaS

Welcome to Yadimen, your trusted partner in transforming your business with todays most advanced technology. Our journey began in 2008 as Valwics in APAC Region with the expansion into Europe/EMEA regions through groundbreaking milestones, leading to the inception of Yadimen as an offshoot focusing on today's technology implementation services for the financial services industry. Here’s a snapshot of our evolution. Our story started as Valwics, with a specific focus on Technology transformations & Legacy transformations. This initiative aimed to deliver unmatched Technology solutions to local businesses, setting a robust foundation in technology consulting within APAC regions. With a strong team of 50 consultants, we have established a multinational presence, incorporating operations in India. We enriched our portfolio to include CRM consulting and digital experience management, showcasing proven expertise across multiple sectors, including finance, healthcare, telecommunications, and retail. In 2022, with 100+ expert consultants, Yadimen specialized in the financial services industry and product implementation services, leveraging our deep-rooted expertise from the industry. Operating across the US, UK, Nordic, and Western Europe, Yadimen Consulting offers tailored solutions that address the specific challenges and opportunities of the financial services sector. Yadimen Consulting is committed to building on the strong legacy of our history by providing innovative and strategic consulting services in technology, especially tailored for the financial services industry. Join us as we continue to shape the future of technology consulting and foster the growth and success of our clients globally.

Company details

Company typeSME
IndustryComputer Software / SaaS
Company size51 - 200

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Data Engineer — PySpark + AWS Glue

ETL & Data Pipelines | 2 Openings

Location: Remote (UK)

Employment Type: Contract / Permanent

Experience Level: 4–8 Years

Openings: 2

About the Role

We are looking for a skilled Data Engineer with solid hands-on experience in PySpark, AWS Glue, and ETL development to build and maintain scalable, production-grade data pipelines on AWS. You will be part of a delivery-focused team working on complex data engineering challenges, contributing to data lake architecture and end-to-end pipeline development.



Requirements

Key Responsibilities

•       Design, develop, and maintain scalable ETL pipelines using PySpark and AWS Glue

•       Build and optimise data ingestion, transformation, and loading workflows

•       Orchestrate data workflows using AWS Step Functions

•       Develop serverless functions using AWS Lambda (Python)

•       Work with data lake architectures on AWS to support analytical use cases

•       Ensure pipeline reliability, monitoring, and performance optimisation

Required Skills & Experience

•       Strong hands-on experience with PySpark and AWS Glue

•       Proven track record in ETL pipeline development and optimisation

•       Experience orchestrating workflows with AWS Step Functions

•       Proficiency in serverless development using AWS Lambda (Python)

•       Good SQL skills and a solid understanding of data processing principles

Nice to Have

•       Exposure to Java-based microservices

•       Understanding of REST APIs and backend service integrations

•       Experience working with AWS-based data lakes




Salary: Day Rate

Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Data Engineer Related jobs

Other jobs at Yadimen Consultancy Ltd

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.