This is a remote position.
We are looking for a hands-on Senior AI Engineer with 4-6 years of engineering experience and deep, current expertise in building Generative AI systems that run in production. This is a core AI engineering role, not a data analysis or reporting role: you will spend your time on LLM application architecture, retrieval design, agentic systems, fine-tuning, evaluation, inference optimization and model serving. We expect a strong grasp of how transformer models actually behave, including tokenization, context windows, attention costs, embeddings and decoding, and the ability to turn that understanding into systems that are accurate, fast and affordable at scale. You will own AI components end to end, from data and retrieval pipelines through model integration, evaluation harnesses, deployment and production monitoring, and will work directly with product managers, solution architects and client stakeholders on enterprise AI delivery. As a senior engineer you will also set the technical direction on the AI components you own, review the work of others, and raise the bar for what ships. The role suits an engineer who reads papers and model cards, prototypes quickly, and holds a high standard for production quality.
Requirements
Responsibilities
- Design and build production LLM applications: RAG pipelines, agentic and tool-calling systems, and multi-step reasoning workflows.
- Engineer retrieval layers on vector databases, covering chunking strategy, embedding model selection, metadata filtering, hybrid search and re-ranking.
- Design context and prompting strategies: system prompt architecture, structured output, function/tool schemas, context compression and memory management.
- Build evaluation harnesses and guardrails to measure accuracy, groundedness, safety, regression and hallucination rates, and drive improvements from the results.
- Optimize inference cost and latency through caching, batching, model routing, quantization, streaming and speculative decoding.
- Deploy and serve models using vLLM, TGI, Triton or managed inference, including self-hosted open-weight deployments for data-sensitive clients.
- Select and integrate foundation models (OpenAI, Anthropic, Google, Llama, Mistral, Qwen) against accuracy, latency, cost and data-residency requirements.
- Expose AI capabilities as secure, well-documented, streaming-capable APIs and integrate them into enterprise applications.
- Operate AI services in production with containers and CI/CD, including tracing, token and cost telemetry, prompt and model versioning, and rollback.
- Lead code and design reviews on AI components, mentor engineers, and contribute reusable frameworks and standards to the AI practice.
- Prototype against new model releases, agent frameworks and inference techniques, and bring what proves out into client delivery.
Essential Skills
Job
- 4-6 years of engineering experience, with at least 2-3 years building and shipping production LLM or Generative AI systems.
- Expert Python, covering asynchronous programming, typing, testing, streaming and API development with FastAPI.
- Deep hands-on LLM application development: prompt and context engineering, structured output, tool calling, function schemas and failure handling.
- Working understanding of transformer internals: tokenization, attention and context-length costs, embeddings, temperature and sampling, and decoding behaviour.
- Proven experience designing and shipping RAG systems, including chunking strategies, embedding selection, hybrid retrieval, re-ranking and retrieval evaluation.
- Hands-on experience with agent and orchestration frameworks such as LangChain, LangGraph, LlamaIndex, CrewAI or the OpenAI/Anthropic agent SDKs.
- Strong working knowledge of vector databases (e.g., Pinecone, Weaviate, Qdrant, FAISS, pgvector) and index and similarity trade-offs.
- Practical experience with fine-tuning and model adaptation (LoRA/QLoRA, PEFT, instruction tuning), including when not to fine-tune.
- Hands-on PyTorch, Hugging Face Transformers and the surrounding ecosystem (datasets, accelerate, PEFT).
- Experience with LLM evaluation and observability tooling (RAGAS, LangSmith, DeepEval, Langfuse) and building custom eval sets.
- Experience reducing inference cost and latency in production, with a clear account of what you measured and what you changed.
- Hands-on experience with at least one major cloud platform (AWS, Azure or GCP) and its AI/ML services, plus Docker and CI/CD.
- Strong database and API fundamentals across SQL (PostgreSQL) and NoSQL (MongoDB, Redis), with Git and disciplined code review practices.
Personal
- Strong engineering judgement, with the ability to reason about accuracy, cost and latency trade-offs rather than defaulting to the largest model.
- Genuine depth of interest in the field: reads papers, model cards and evaluations, and tests claims instead of taking them at face value.
- Clear communication skills, including the ability to explain AI trade-offs and limitations to non-technical stakeholders.
- Strong ownership and accountability, with the ability to drive work independently end to end.
- Able to mentor engineers and lift the technical standard of the team.
- Comfortable working in fast-paced, ambiguous and rapidly evolving problem spaces.
Preferred Skills
Job
- Experience with multi-agent orchestration, planning loops and long-running autonomous workflows.
- Experience with multimodal models covering vision, speech or document understanding, including OCR-heavy document pipelines.
- Experience with distributed or accelerated training and GPU resource management.
- Knowledge of graph-based retrieval (GraphRAG), knowledge graphs and hybrid symbolic approaches.
- Experience with model serving infrastructure, autoscaling GPU workloads and cost governance.
- Familiarity with responsible AI practices: bias evaluation, red-teaming, PII handling and governance frameworks (EU AI Act, NIST AI RMF).
- Exposure to microservices architecture, event-driven systems and distributed systems fundamentals.
- Open-source contributions, published work, or a portfolio of AI systems built outside of client work.
- Personal
- Builder's mindset, comfortable prototyping quickly and discarding what does not hold up.
- Consulting orientation, balancing technical ideals against client timelines and budgets.
Other Relevant Information
- Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field.
- Relevant certifications in AI/ML or cloud platforms (AWS/Azure/GCP) are a plus.
- A portfolio of production Generative AI systems, open-source contributions or published work is highly desirable.
Benefits
- This role offers the flexibility of working remotely in India.
LeewayHertz is an equal opportunity employer and does not discriminate based on race, colour, religion, sex, age, disability, national origin, sexual orientation, gender identity, or any other protected status. We encourage a diverse range of applicants.
