AWS Engineer (Redshift/Glue)

On behalf of our client, Affinity is seeking an experienced AWS Engineer to participate in a major data transformation initiative. This project involves migrating legacy Informatica jobs and Teradata data to AWS. The AWS Engineer will be responsible for building and managing AWS services to support ETL processes, ensuring a smooth transition of data and workflows to AWS Redshift. You will collaborate closely with the ETL Developer and AWS Solution Architect to design, implement, and optimize scalable cloud-based data solutions.

• Design and implement AWS infrastructure, including Redshift, S3, Glue, and Lambda, to support the migration of Informatica workflows and data.
• Migrate and transform legacy Informatica ETL workflows to AWS-native solutions, ensuring performance, scalability, and cost-effectiveness.
• Work closely with the ETL Developer to develop seamless data pipelines for data ingestion and transformation using AWS services.
• Monitor and troubleshoot AWS infrastructure, optimizing for performance, cost, and scalability to support ETL processes.
• Build and maintain data pipelines for the migration of data from Teradata to AWS, ensuring data consistency and integrity throughout the process.
• Automate infrastructure deployment using AWS CloudFormation or Terraform, ensuring efficient and consistent deployments.
• Collaborate with the DevOps Engineer to integrate CI/CD processes for continuous infrastructure deployment and updates.
• Assist with unit testing and validation of migrated workflows to ensure that data meets the original requirements and business needs.
• Provide technical leadership on AWS best practices and optimize resource utilization within the cloud environment.
• Document infrastructure, processes, and configurations to ensure knowledge transfer and future support.
  
  

• 5+ years of experience as a cloud engineer, with extensive hands-on experience in AWS.
• Expertise in AWS services, including Redshift, Glue, S3, Lambda, and experience with infrastructure-as-code tools like CloudFormation or Terraform.
• Experience with ETL processes, particularly in migrating on-premise systems such as Informatica to AWS.
• Strong proficiency in SQL and experience with data warehousing technologies like Redshift and Teradata.
• Experience developing and managing data pipelines and orchestrating workflows in AWS.
• Performance tuning experience for optimizing ETL jobs and cloud infrastructure.
• Familiarity with automation tools and infrastructure-as-code principles for AWS deployment.
• Problem-solving mindset with the ability to troubleshoot and optimize cloud environments.
• Experience working in Agile environments and utilizing collaboration tools such as Jira and Confluence.
  
  

• Familiarity with AWS Glue and orchestration frameworks for ETL processes.
• Experience with CI/CD pipelines using Jenkins, AWS CodePipeline, or equivalent tools.
• Understanding of data reconciliation processes between Teradata and AWS.
• Prior experience in public sector or enterprise environments is a plus.
• Strong interpersonal and communication skills with a focus on collaboration and teamwork.

Job Number: 11099