Match score not available

Lead Data Software Engineer (Databricks)

Remote:

Full Remote

Experience:

Senior (5-10 years)

Work from:

Greece

Offer summary

Qualifications:

Experience as a Lead Data Engineer, Solid understanding of data architectures, Expertise in Python and cloud-native technologies, Hands-on experience with Databricks.

Key responsabilities:

Lead design and development of data architectures
Collaborate with machine learning teams for integration

EPAM Systems XLarge http://www.epam.com

10001 Employees

See more EPAM Systems offers

Job description

We are seeking a Lead Data Software Engineer to join our team and lead the development of our innovative supply-chain data analytic platform.

This platform provides an integrated view of suppliers, products, material categories, shipments, and compliance activities while leveraging artificial intelligence to deliver insights and recommendations tailored to the specific needs of our clients.

If you are interested in this opportunity – apply!

Responsibilities

Lead the design and development of data architectures and ETL pipelines using Databricks and external orchestrators such as Airflow
Collaborate with machine learning teams to integrate AI-driven insights into the platform
Implement system-level improvements and components for data engineering
Optimize data-intensive applications for performance and scalability
Ensure adherence to software engineering best practices, including containerization, unit testing, linting, and code style checks
Maintain and enhance data workflow, Delta Lake, and Delta Live Tables within Databricks
Provide technical guidance and mentorship to other data engineers in the team
Communicate effectively with clients and stakeholders to understand their needs and provide tailored solutions
Handle ambiguity and work independently to drive projects to completion

Requirements

Experience as a Lead Data Engineer in a project employing Databricks platform
Solid understanding of data architectures and data modeling skills
Experience designing and building ETL pipelines with Databricks using external orchestrators (e.g., Airflow)
Hands-on experience with Databricks (Delta Lake, workflows, Delta Live Tables, deployment, and versioning)
Expertise in Python and cloud-native technologies
Proficiency in Spark/PySpark
Engineering experience with either AWS or Azure
Experience with big data and performance optimization of data-intensive applications
Strong proactivity, with client-facing experience
Ability to deal with ambiguity and drive projects independently
Desire to work in a transparent and quickly moving startup environment
Fluent English communication skills at a C1 level

Nice to have

Experience setting up or maintaining CI/CD pipelines on Azure DevOps
Understanding of Data Observability and Data Quality Monitoring
Experience integrating data quality checks in data pipelines

We offer

For you:

Paid annual vacation
Paid sick leave days
Private health insurance
Stable income
Meal and home office compensation

For your comfortable work:

Remote and hybrid work opportunities
Corporate laptop
Possibility to work on your own device
Free licensed software
Relocation opportunities
Free wellbeing activities

For your growth:

Possibility to create a Personal Development Plan from the first day in the company
Free trainings for technical and soft skills
Free access to LinkedIn Learning platform
Free access to internal and external e-Libraries
Certification opportunities
Language courses
Internal technical and non-technical communities
Possibility to contribute in internal, opensource products

The remote option applies only to the Candidates who will be working from any location in Greece.

EPAM is a leading global provider of digital platform engineering and development services. We are committed to positively impacting our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to learn and grow continuously. You will join a dedicated, creative, and diverse community that will help you discover your fullest potential.

EPAM is committed to providing our global team of 52,800+ EPAMers with inspiring careers. EPAMers lead with passion and honesty and think creatively. Our people are the source of our success, and we value collaboration, try always to understand our customers' business, and strive for the highest standards of excellence.