Role overview

Qualifications

Strong experience in AWS Data Engineering - Serverless Architecture
Hands-on expertise with AWS Glue, Lambda, S3, Athena, Step Functions
Strong Python and SQL skills
Experience integrating Qualtrics API or structured survey data

Responsibilities

Design and maintain ETL pipelines using AWS Glue (PySpark)
Develop AWS Lambda functions (Python / Node.js) for serverless data processing
Manage AWS S3 for Qualtrics input/output storage and optimized data access
Monitor pipelines using CloudWatch logs and metrics

Key facts

Remote from: California (USA)
Fixed term
Data Engineer
English

Hard skills

Amazon Web Services

About the company

Veracity Software Inc

The definition of Veracity is accuracy and habitual truthfulness. This truly defines our brand in simplistic terms. At Veracity, we believe in doing everything in the ‘Right Way’, which means you have to challenge your own innovations and solutions all the time. Our strength lies in our employees, and with the right blend of business and technical expertise, we set the bar high for Veracity. From inception, Veracity focused on building cloud based SaaS solutions while using the latest and most advanced technology stack. We have delivered cloud based customized solutions and services to various clients across both North America and India. We have a global headquarter in Woodbridge, NJ while our offshore office is in Pune, India. We believe in bringing Global talent together, which helps us in delivering cost effective, yet highly scalable and efficient solutions. With the vision of trying to be one of the trusted IT partners as well as one of the most respected solution providers with an onsite/offshore model, Veracity is equipped with the right set of highly skilled individuals who have expertise in delivering the most effective business services and solutions for our clients. Veracity’s mission is to become the Trusted IT Partner for various businesses across globe.

Company details

Company size51 - 200

Links

Website LinkedIn See all jobs

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

AWS Data Engineer – Qualtrics Integration
Role Type: Contract
Location: Torrance, CA / Remote

Domain: Enterprise Survey Platform / Qualtrics
Primary Stack: AWS | Python | SQL | Qualtrics API

The AWS Data Engineer – Qualtrics Integration is responsible for designing, building, and maintaining scalable, automated data pipelines that support Qualtrics survey ingestion, transformation, and downstream reporting.
This role focuses on serverless AWS data engineering, integrating Qualtrics APIs with AWS services to process structured survey data, dealer hierarchies, and reporting files. The engineer will ensure data accuracy, automation, monitoring, and performance across end-to-end workflows.

Key Responsibilities
AWS Cloud & Data Engineering

Design and maintain ETL pipelines using AWS Glue (PySpark)
Develop AWS Lambda functions (Python / Node.js) for serverless data processing
Manage AWS S3 for Qualtrics input/output storage and optimized data access
Orchestrate workflows using AWS Step Functions and MWAA (Airflow)
Query structured datasets using AWS Athena
Monitor pipelines using CloudWatch logs and metrics

Data Processing & ETL

Transform and aggregate Qualtrics datasets (CSV, JSON, XML)
Merge multiple source files (e.g., dealer master + employee files) into unified hierarchies
Automate ingestion, transformation, validation, and report generation
Implement reusable, scalable ETL frameworks

API & Integrations

Integrate with Qualtrics APIs to extract raw survey data and response files
Implement event-driven processing (S3 triggers → Lambda)
Connect AWS pipelines with CRM, ERP, BI tools, or downstream platforms

Data Quality & Reporting

Perform data validation and quality checks prior to Qualtrics ingestion
Generate formatted output files aligned to business-defined templates
Support ad-hoc analysis using SQL and Athena

Programming & Automation

Develop robust Python scripts for Glue, Lambda, and automation tasks
Write optimized SQL queries for structured data access
Use Bash/Shell scripting for file movement and preparation

DevOps & Security (Nice to Have)

Configure IAM roles and permissions securely
Implement Infrastructure as Code (Terraform / CloudFormation)
Support CI/CD pipelines for data workflows

Required Skills & Experience

Strong experience in AWS Data Engineering & Serverless Architecture
Hands-on expertise with AWS Glue, Lambda, S3, Athena, Step Functions
Experience with MWAA (Airflow) for orchestration
Strong Python and SQL skills
Experience integrating Qualtrics API or structured survey data
Ability to troubleshoot pipeline failures and performance issues

Preferred / Nice-to-Have Skills

Experience with Terraform or CloudFormation
CI/CD for data pipelines
Experience supporting BI tools (Power BI, Tableau, etc.)
Knowledge of data governance and security best practices

Ideal Candidate Profile

AWS-focused Data Engineer with strong automation mindset
Comfortable working with survey / VoC / structured data
Strong debugging, monitoring, and optimization skills
Able to work independently in enterprise environments

Recruiter Submission Template –

Full Name:
Degree Major with University & Completion Year:
Total Years of Data Engineering Experience:
Total Years of AWS Data Engineering Experience:
Total Years of Experience with Serverless Architecture:
Experience with AWS Glue (PySpark ETL)? (Yes/No – please elaborate):
Experience with AWS Lambda (Python/Node.js)? (Yes/No – please elaborate):
Experience with AWS S3 data storage & optimization?
Experience with AWS Athena? (Yes/No):
Experience implementing data validation & quality checks?
Experience working with Qualtrics API? (Yes/No – please specify):
Experience with event-driven processing (S3 → Lambda)?
Experience integrating AWS with CRM / ERP / BI tools?
Python Experience (years & libraries used):
SQL Experience (years & databases/tools):
Experience with IAM & role-based access?
Brief description of a recent AWS data pipeline you built (tools + outcome):
Motivation / Reason for Interest in Qualtrics Data Engineering Role:
Contact Number:
Email ID:
LinkedIn Profile URL:
Full Address (Street, City, State, Zip):
Notice Period (in weeks):
Current Work Authorization Status:
Expected Hourly Rate:
W2 / C2C (If C2C, corporation name):
Are you comfortable working primarily Remote with Limited Onsite Flexibility?