Senior Data Engineer (Python / AWS / ML Pipelines) at Vigil

SUMMARY:

We are looking for a Senior Data Engineer to join one of our UK-based clients, working on large-scale data and machine learning pipelines in a production environment.

This role is focused on building and maintaining ML pipelines used for forecasting and data-driven decision-making in a high-scale platform. You will be working in a cloud-based environment, helping design, build, and deploy data and ML workflows into production.

We are looking for someone with strong Python experience, cloud experience (AWS), and experience building production data pipelines and ML infrastructure.

WHAT WILL YOU BE DOING:

• Building and maintaining data and machine learning pipelines in production;
• Developing ETL and data processing workflows using Python;
• Orchestrating workflows using tools such as Airflow and AWS Step Functions;
• Working with AWS Glue for data processing and pipeline execution;
• Supporting the deployment of machine learning models into production using SageMaker;
• Working closely with Data Scientists and ML Engineers to productionise ML models;
• Monitoring, troubleshooting, and improving pipeline reliability and performance;
• Contributing to architecture and technical decisions around data platform and ML infrastructure;
• Working in an agile environment with distributed teams.

WHAT WE ARE LOOKING FOR:

• Strong experience with Python;
• Experience building data pipelines and/or ML pipelines in production;
• Hands-on experience with Airflow;
• Experience with AWS Step Functions and AWS Glue;
• Experience deploying ML models using AWS SageMaker;
• Strong experience with AWS cloud;
• Experience working with production systems at scale;
• Experience working in cross-functional teams with Data Scientists;
• Strong communication skills and ability to work in a distributed team environment.
• Strong English communication skills, both written and verbal

AWESOME BUT NOT REQUIRED:

• Experience with GCP;
• Experience with monitoring and observability tools.

WHAT’S IN IT FOR YOU?

• Be part of our collegial environment where responsibility and authority are shared equally amongst colleagues and help create our company culture
• A culture in which we don’t criticise failure but ensure we learn from our mistakes
• An Agile environment where your ideas are welcome
• The possibility to grow and experience different projects
• Ongoing Training & Mentoring
• The possibility of travel

- ATTENTION! THIS POSITION IS FOR PORTUGAL OR BRAZIL BASED ONLY