Offer summary

Qualifications:

Minimum of 2 years of hands-on experience with Databricks and AWS., Strong proficiency in Python programming and experience with Apache Spark., Solid understanding of AWS technologies such as EC2, S3, and SQS., Proficiency in SQL and experience with DevOps processes..

Key responsabilities:

Design, develop, and maintain scalable data pipelines using Databricks.

Write and deploy Python code for ETL processes and production environments.

Collaborate with data scientists to meet their data requirements.

Monitor and troubleshoot data pipelines and system performance.

Job description

We're seeking a talented and motivated Databricks Engineer to join our growing data engineering team. In this role, you will be responsible for designing, developing, and deploying data pipelines and solutions using Databricks, Python, and AWS services. You will play a key role in building and maintaining our data infrastructure to support our business needs.

Responsibilities:

Design, develop, and maintain scalable and reliable data pipelines using Databricks.
Write and deploy Python code for data processing and ETL (Extract, Transform, Load) processes.
Package and deploy Python code effectively for production environments.
Utilize AWS foundation technologies such as EC2, S3, and SQS to build robust data solutions.
Implement and adhere to DevOps processes and utilize relevant tools for continuous integration and continuous deployment (CI/CD).
Write and execute SQL queries for data analysis and manipulation.
Collaborate with data scientists and analysts to understand their data requirements and provide necessary data infrastructure.
Monitor and troubleshoot data pipelines and system performance.
Stay up-to-date with the latest technologies and best practices in data engineering and cloud computing.

Requirements