Data Engineer - AI

This is a remote position.

Job Summary

We are seeking a skilled Data Engineer to support enterprise AI and Generative AI initiatives by building scalable, secure, and AI-ready data platforms. The ideal candidate will have strong expertise in Azure Databricks, Snowflake, Python, and SQL, along with experience integrating modern Azure AI services into enterprise data ecosystems.

In this role, you will partner with Data Engineers, AI Engineers, Data Scientists, and business stakeholders to develop high-quality data pipelines, enable Retrieval-Augmented Generation (RAG) solutions, and prepare enterprise data for AI-powered applications using Microsoft's AI ecosystem.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Databricks, Snowflake, and Azure Data Factory.
  
• Build ELT/ETL pipelines to ingest data from ERP, CRM, manufacturing, supply chain, and other enterprise applications.
  
• Develop AI-ready datasets that support machine learning and Generative AI use cases.
  
• Integrate structured, semi-structured, and unstructured data into centralized data platforms.
  
• Support Retrieval-Augmented Generation (RAG) solutions by preparing document repositories, metadata, embeddings, and search indexes.
  
• Collaborate with AI Engineers to expose enterprise data securely to LLM-powered applications.
  
• Optimize data models, transformations, and query performance for analytics and AI workloads.
  
• Implement data quality, governance, lineage, monitoring, and security best practices.
  
• Build reusable data transformation frameworks using Python, SQL, Spark, and dbt (if applicable).
  
• Develop REST API integrations to support AI services and enterprise applications.
  
• Participate in architecture discussions, code reviews, CI/CD implementations, and Agile ceremonies.
  
• Support production deployments, monitoring, troubleshooting, and performance tuning.
  

Required Qualifications

• 6+ years of experience in Data Engineering.
  
• Strong hands-on experience with Azure Databricks and Snowflake.
  
• Proficiency in Python, SQL, and PySpark.
  
• Experience building scalable ELT/ETL pipelines using Azure Data Factory or similar orchestration tools.
  
• Strong understanding of data warehousing, dimensional modeling, and data lake architectures.
  
• Experience working with REST APIs and integrating cloud-based services.
  
• Familiarity with Git, Azure DevOps, and CI/CD pipelines.
  
• Experience working in Agile/Scrum environments.
  

AI & Azure AI Experience (Required)

• Experience supporting enterprise AI or Generative AI initiatives.
  
• Working knowledge of Azure Machine Learning (Azure ML).
  
• Experience with Microsoft AI Foundry (Azure AI Foundry) for AI solution development and orchestration.
  
• Experience integrating Azure OpenAI Service into enterprise applications.
  
• Knowledge of Azure AI Search for enterprise search and Retrieval-Augmented Generation (RAG) solutions.
  
• Exposure to Microsoft Copilot Studio for developing AI-powered copilots and conversational experiences.
  
• Understanding of vector search, embeddings, prompt engineering, and LLM integration.
  
• Experience preparing enterprise data for AI model training, inference, and knowledge retrieval.