Hands-on experience with Databricks (clusters, notebooks, Delta Lake)
Strong Python and SQL skills
Experience with AI parsing/extraction and text-processing workflows
Background in data migration and troubleshooting
Requirements:
Migrate existing data pipelines and workflows to Databricks
Analyze the customer's current solution and implement improvements
Build and optimize Python- and SQL-based pipelines
Develop and enhance AI-driven parsing and extraction components
Job description
Overview
We are looking for a Databricks Engineer with strong Python, SQL, and AI parsing/extraction skills to help migrate an underperforming customer solution to Databricks.
Key Responsibilities
Migrate existing data pipelines and workflows to Databricks.
Analyze the customer's current solution and implement improvements.
Build and optimize Python- and SQL-based pipelines.
Develop and enhance AI-driven parsing and extraction components.
Ensure performance, data quality, and reliability.
Collaborate with technical and business teams to deliver a stable solution.
Required Skills
Hands-on experience with Databricks (clusters, notebooks, Delta Lake).
Strong Python (required) and SQL skills.
Experience with AI parsing/extraction and text-processing workflows.
Background in data migration and troubleshooting.
Preferred
Experience with Azure Databricks, MLflow, or Lakehouse architecture.
Knowledge of document intelligence tools and DevOps workflows.
