Lead Data Software Engineer

We are looking for a Lead Data Software Engineer to join our company. This position brings together hands-on technical work and leadership responsibility, offering the chance to shape data architecture while driving delivery on critical projects. You will work closely with stakeholders and engineering teams to make sure solutions stay reliable and scalable as demands evolve. If you enjoy pairing hands-on technical involvement with accountability for delivery success, this role could be a strong fit for you. Responsibilities Develop and preserve the codebase supporting ETL and ELT pipelines, together with large batch, micro-batch, and streaming systems Set up the infrastructure required to pull, transform, and load data efficiently from a variety of sources using ADF, Spark, Kafka, or comparable technologies Identify, plan, and roll out internal process improvements, such as automating manual tasks, improving data delivery, and modifying infrastructure to increase scalability Architect and release data services solutions using technologies such as Spring Boot, ReactJS, NoSQL, and other related UI and API tools Maintain governance across delivery management and production processes, aligned with the chosen delivery model Act as the key contact for delivery-related matters with senior leadership and stakeholders, covering escalations, upsells, and ramp-downs Provide technical direction throughout the delivery process, keeping the architecture solid and forward-thinking while ensuring implementation meets quality expectations Write user stories and their corresponding acceptance criteria as part of an agile/scrum process Coordinate work across multiple disciplines and stakeholders to keep delivery progressing Ensure project delivery aligns with customer processes and methodologies, applying strong knowledge of different delivery models with an agile-focused approach Establish a continuous delivery risk management framework that enables proactive decision-making throughout the delivery lifecycle Monitor and enhance delivery productivity over time Serve as a technical resource for Data Engineers, helping ensure their work meets quality and timeliness standards Oversee production support and deployment responsibilities Requirements A minimum of 5 years of experience in a similar position At least one year of experience leading and managing teams Solid command of core Python for data engineering purposes Proficient in SQL for querying and processing data Hands-on experience with Apache Spark for large-scale data processing Familiarity with Anthropic Claude Code Experience using Databricks for data engineering and analytics tasks Excellent English proficiency (B2 level or higher) Nice to have Experience with Apache Kafka for managing streaming data Working knowledge of Java Hands-on experience with PySpark Familiarity with Scala Experience with Spark Streaming