Offer summary

Qualifications:

Proficiency in Python, PySpark, and AWS services such as EMR, Lambda, DynamoDB, Athena, and OpenSearch., Experience in API development and optimization., Familiarity with data analysis, Sagemaker, and Elasticache is advantageous., Experience working in Agile teams using tools like JIRA..

Key responsibilities:

Update and improve PySpark scripts on EMR on EKS.

Develop and optimize AWS Lambda functions in Python.

Create new data processing scripts using PySpark.

Collaborate on data analysis and dashboard development for data quality and model insights.

Job description

Data Engineer

We are seeking an experienced Data Engineer to join the Digital Memory (DIMSUM) pod within OTA. The ideal candidate is proficient in Python, PySpark, and AWS tooling, specifically EMR, Lambda functions, DynamoDB, Athena, OpenSearch, and API development. Familiarity with Sagemaker and Elasticache is a plus.

Responsibilities:

Update and enhance existing PySpark scripts running on EMR on EKS.
Develop and optimize Lambda functions (Python).
Create new PySpark scripts for data processing.
Support the development of business insights through data analysis.
Collaborate on Tableau dashboards for data quality monitoring.
Contribute to iterative development for Customer Frustration Score and Predictive Intent models.
Engage in Agile processes, working closely with a 4-member team, using JIRA for project management.

Qualifications:

Expertise in Python, PySpark, AWS EMR, Lambda, DynamoDB, Athena, OpenSearch.
Experience with API development and optimization.
Familiarity with data-driven solutions, Sagemaker, and Elasticache.
Experience working in an Agile environment with team communication tools like JIRA.
Exposure to ML models and algorithms is a plus, though not mandatory.

This is an exciting opportunity to work on innovative digital memory initiatives in a collaborative and agile team environment!

Required profile