Knowledge of data privacy and compliance standards
Excellent written and verbal communication skills
Requirements:
Design and implement scalable ETL pipelines
Develop and optimize T-SQL stored procedures
Architect secure data workflows in Azure
Ensure data quality and compliance
Collaborate with software engineering teams
Monitor and troubleshoot pipeline failures
Lead data governance initiatives
Mentor junior engineers
Job description
This is a remote position.
SUMMARY
The Senior Data Engineer is responsible for designing, building, and maintaining robust data pipelines using Python, SQL Server (T-SQL, Stored Procedures), and integrating various APIs within a modern Azure infrastructure. The role supports critical data operations in regulated environments.
JOB RESPONSIBILITIES
Design and implement scalable ETL pipelines with Python and SQL Server for data ingestion, transformation, and integration from diverse sources, including RESTful APIs.
Develop and optimize T-SQL stored procedures to fulfill business logic and reporting needs.
Architect secure, efficient data workflows leveraging Azure Data Factory, Azure SQL, and Azure Functions.
Ensure data quality, lineage, and compliance with regulations such as HIPAA and SOC2.
Collaborate with software engineering teams to align data structures and APIs with application requirements, including those based on .NET and C++ environments.
Work cross-functionally with data analysts, security engineers, and DevOps to support comprehensive data needs.
Monitor and troubleshoot pipeline failures and data discrepancies.
Lead data governance initiatives, best practices, and performance tuning efforts.
Mentor junior engineers and participate in peer code reviews.
QUALIFICATIONS
5+ years of experience in Data Engineering or a similar role.
Strong proficiency in SQL Server, including stored procedures and performance tuning.
Advanced skills in Python for ETL development and automation.
Hands-on experience with Azure data services such as Data Factory, Blob Storage, and Azure SQL.
Experience integrating data from RESTful APIs and third-party sources.
Knowledge of data privacy and compliance standards, especially HIPAA and SOC2, relevant to financial and healthcare sectors.
Strong problem-solving abilities and capacity to work independently in a fast-paced, regulated environment.
Excellent written and verbal communication skills.
Experience with additional Azure services like Azure Synapse, Azure DevOps, and Azure Functions.
Understanding of data modeling and warehousing concepts.
Exposure to cybersecurity data, SIEM tools, or SOC operations is advantageous.
Background in MSP/MSSP environments or consulting.
Familiarity with Power BI or other data visualization tools.
Knowledge of .NET Framework and C#-based APIs, particularly in data consumption contexts.
JOB REQUIREMENTS
Should be willing to accept a long-term work-from-home arrangement.
Should be amenable to a permanent night shift schedule.
