Logo for Datavail

Senior Data Engineer

Role overview

Qualifications

  • 8+ years in data engineering/data warehousing
  • 3+ years delivering Azure solutions and production Microsoft Fabric experience
  • Advanced T-SQL, Spark SQL, Python/PySpark
  • Strong dimensional modeling and working knowledge of Data Vault 2.0

Responsibilities

  • Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure
  • Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns
  • Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers
  • Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover

Key facts

Hard skills

Other skills

  • Governance
  • Communication
  • Mentorship

About the company

Datavail logo

Datavail

  • IT Services & Consulting
  • 1,001–5,000
Datavail is a proven technology consulting, implementation, and managed services partner headquartered in Boulder, Colorado. Our world-class tech-enabled delivery platform enables us to provide an extensive solutions portfolio, including database management, analytics, application development, cloud infrastructure, and enterprise applications consulting — on-premises and in the cloud. With tens of thousands of hours of experience, our goal is to enable our clients to build modern, data-driven organizations that scale. Datavail maintains strong partnerships with leading technology companies to help our clients leverage the best-fit platform to optimize outcomes. In addition, we have poured over $25 million in R&D to bring world-class systems and application monitoring tools to the market. We seek to leverage the greatest that technology and talent have to offer to simplify and operationalize the IT systems organizations use to manage their most valuable data. We select the best across the globe so we can implement complex solutions for top brands as well as small-to-midsize organizations. Our passionate team of 1200+ certified, cloud-focused consultants deliver industry-focused solutions to 500+ customers with an average lifespan of 7+ years. These committed tech professionals help our clients jump-start their projects, improve time to value & reduce the risk for the core industries that we serve. Our Technology Alliances: Datavail has leading partnerships with AWS, Microsoft, Oracle, Google, Maria DB, Mongo DB, MySQL and can provide 24×7 managed services for all leading database and applications environments including: · AWS · Azure · Oracle · MongoDB · SQL Server · Db2 · PostgreSQL · MySQL · Cassandra · Hadoop · MariaDB · Oracle E-Business Suite (EBS)/ Oracle PeopleSoft/ Oracle Hyperion technologies · Oracle Enterprise Resource Planning (ERP) and Enterprise Performance Management (EPM) technologies · Oracle Analytics & Business Intelligence tech.

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

  • 8+ years in data engineering/data warehousing; 3+ years delivering Azure solutions and production
  • Microsoft Fabric experience
  • Advanced T-SQL, Spark SQL, Python/PySpark; working knowledge of KQL and DAX
  • Deep experience with Delta Lake, Spark tuning, medallion/lakehouse patterns, CDC and incremental loading
  • Strong dimensional modeling and working knowledge of Data Vault 2.0
  • Practical CI/CD, Git, automated testing, infrastructure as code, DataOps, governance and security experience
  • Professional-services/client-facing delivery experience
  • Authorization to work in the US and ability to travel up to 15%
  • Preferred: DP-700, DP-600, DP-203; streaming/event-driven, RAG/GenAI, Fabric IQ/data agents, and industry experience
  • Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure
  • Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns
  • Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers
  • Build Fabric solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Spark notebooks, Real-Time Intelligence and Power BI
  • Implement dimensional and Data Vault models, data-quality testing, reconciliation, observability, logging, alerting and SLA monitoring
  • Establish Git-based CI/CD and automated deployment using Azure DevOps or GitHub, Fabric deployment pipelines, REST APIs, Bicep and Terraform
  • Apply Purview catalog/lineage, least-privilege access, sensitivity controls, encryption and secure connectivity
  • Support migrations from SQL Server, SSIS, SSAS, legacy warehouses and Synapse to Fabric/Azure, including conversion, reconciliation and cutover
  • Prepare governed data for AI/ML and RAG workloads, including ingestion, chunking, embeddings, indexing and telemetry
  • Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover

Related jobs

See all Data Engineer jobs

Other jobs at Datavail