Match score not available

Data Engineer Senior

Remote:

Full Remote

Experience:

Senior (5-10 years)

Work from:

Portugal

Offer summary

Qualifications:

Strong experience with Delta Lake architecture, Proficiency in SQL, Spark/PySpark, and Python, Expertise in Azure data services, Knowledge of ML and MLFlow.

Key responsabilities:

Onboard new data sources and use cases
Design and implement scalable data pipelines

Körber Machinery XLarge https://www.koerber.com/

10001 Employees

See all jobs

Job description

Key Responsibilities

Onboard New Data Sources: Collaborate with business areas to integrate new data sources and use cases. Design and implement scalable data pipelines using Azure Data Factory and Event Hubs
Implement Security Best Practices: Ensure compliance with enterprise security requirements and implement best practices.
Technical Support: Provide second-level technical support to users.
Collaborate with Users: Work with platform users to create scalable, production-grade data processing environments.
Monitor Platform Health: Oversee platform health, costs, and performance to ensure optimal operation.
Support Automation: Develop and maintain automation tools for various platform components.
Data Lake Architecture: Build and manage data lake architecture using Delta Lake and Databricks.
Data Governance: Implement data governance and security controls to protect data integrity and privacy.
Reusable Components: Create reusable components and templates for data ingestion and processing.

Required Skills

Data Modeling: Strong experience with data modeling and Delta Lake architecture.
Machine Learning: Knowledge of ML and MLFlow.
Programming Skills: Proficiency in SQL, Spark/PySpark and Python.
Data Platform Experience: Familiarity with enterprise data platforms, especially Databricks.
Event Data Processing: Experience with event data processing.
Security Knowledge: Understanding of security best practices and compliance requirements.
Azure Data Services: Expertise in Azure data services like Data Factory and Event Hubs.

What We Offer

High-impact role in a small, fast-moving team
Shape the future of industrial data platforms
Work with cutting-edge cloud technologies
Flexible remote work policy
Competitive compensation
International work environment

Recruitment Team

For questions please contact us via KDcampusporto.recruitment@koerber.com

Please apply exclusively via the “Apply now” button. Recruiter: Vasco Bessa

We value diversity and therefore welcome all applicants - regardless of gender, nationality, ethnic or social origin, religion/belief, disability, age, sexual orientation and identity, and any other protected status.