Lead ML/AI Solution Engineer with Databricks

We are seeking a seasoned Lead ML/AI Solution Engineer to architect, develop, and launch a personalization platform that delivers recommendation and ranking models via a scalable API layer. The system comprises a Python FastAPI service exposed through AWS API Gateway and Lambda, integrating with Databricks Model Serving endpoints. The engineer will partner closely with Data Scientists to operationalize personalization models, guarantee production-grade reliability, and confirm performance through testing and monitoring. Responsibilities Partner with Data Scientists to adapt, package, and deploy personalization models (recommendation and ranking models) to Databricks Model Serving using MLflow Develop and maintain a FastAPI service that invokes one or more model serving endpoints Apply production-grade engineering practices, including caching, connection pooling, request timeouts, and fail-open fallback mechanisms Set up secure authentication between services and Databricks using Service Principals Establish and maintain the AWS integration layer, including API Gateway, AWS Lambda, and networking/egress connectivity Define and govern infrastructure using Terraform and build a simple CI/CD pipeline Enable observability and monitoring, including latency, error, and fallback metrics Run load and performance testing to validate system behavior under expected traffic volumes Specify REST API contracts and implement contract testing to enable early integration by consuming teams Carry out offline evaluation of personalization models and support online validation through A/B testing or shadow testing where feasible Present technical results and business impact to both technical and non-technical stakeholders Requirements 5+ years of experience in ML/AI engineering, building and serving recommendation or ranking models, including offline model evaluation At least 1 year of relevant leadership experience Hands-on knowledge of Collaborative Filtering, Two-Tower Architectures, and Learning-to-Rank approaches Strong proficiency in Python with production experience building services using FastAPI Understanding of asynchronous processing, connection pooling, and caching strategies Expertise in timeout handling, fail-open mechanisms, and resilient service design Background in registering, versioning, and serving ML models with MLflow and Databricks Model Serving Familiarity with Service Principal authentication, endpoint scaling, and version management Competency in monitoring latency, errors, and fallback rates, including creating alerts and dashboards Skills in load-testing tools such as k6 or Locust, with the capability to validate p99 latency targets Capability to explain technical concepts and outcomes in business terms, working with engineering, data science, and business stakeholders English proficiency at an Upper-Intermediate level (B2) or higher Nice to have Proficiency in Infrastructure-as-Code with Terraform and CI/CD pipeline implementation adequate to deploy and hand over a reproducible POC environment Familiarity with AWS API Gateway and Lambda integration, including awareness of cold starts and performance optimization Knowledge of VPC networking and egress connectivity to Databricks Expertise in OpenAPI-first REST API design and contract testing with Pact or schema-based testing approaches Capability to architect services for multi-region deployment and future scalability Understanding of GenAI for personalization, including embeddings, vector search, LLM-based re-ranking, and personalized explanations powered by GenAI