Offer summary
Qualifications:
Over 5 years of experience in data engineering, Strong programming skills in Python, Advanced knowledge of PySpark, Experience with AWS EMR Serverless, Deep understanding of Apache Airflow.
Key responsabilities:
- Design and develop data processing pipelines using Python and PySpark
- Implement and manage scalable clusters with AWS EMR Serverless
- Automate complex data workflows using Apache Airflow
- Monitor performance and resolve data pipeline issues
- Document data processes ensuring access to resources