Senior MLOps / LLMOps Engineer

Your Typical Day With Us

As a Senior MLOps/LLMOps Engineer, you will be at the forefront of deploying, optimizing, and monitoring LLMs in production environments. Your role will involve building and maintaining scalable pipelines, ensuring low-latency inference, and implementing best practices in monitoring and observability. You will also work with state-of-the-art tools like Hugging Face and MLFlow to fine-tune models and integrate them into robust AI solutions.

Key Responsibilities

• Model Deployment & Management
• Develop and maintain scalable pipelines for deploying LLMs, focusing on efficient, low-latency inference.
• Utilize tools like Hugging Face and MLFlow for seamless model integration and version control.
• Automate deployment processes, including model validation and continuous integration.
• Monitoring & Observability:
• Implement comprehensive monitoring frameworks to track performance and reliability of models in production.
• Use advanced observability tools to proactively detect and address performance issues.
• Deploy alerting systems to ensure rapid response to anomalies in model behavior.
• Infrastructure Optimization:
• Architect and optimize cloud and on-premise infrastructure to support large-scale LLM operations.
• Collaborate with cloud providers like AWS, Azure, and GCP to optimize costs and performance.
• Work with backend engineers to ensure smooth integration of AI models into conversational platforms.
• Collaboration & Documentation:
• Partner with AI engineers and data scientists to align on project objectives and deployment strategies.
• Document MLOps processes, best practices, and tools to maintain operational excellence.
• Provide training and support to team members on MLOps methodologies and tools.

What You’ll Need

• Experience
• 5+ years of experience in MLOps, DevOps, or related fields, with a focus on deploying and managing LLMs or other large-scale machine learning models.
• Proven experience with tools like Hugging Face, MLFlow, and containerization technologies (Docker, Kubernetes).
• Strong experience with cloud platforms (AWS, Azure, GCP) and infrastructure as code (Terraform).
• Hands-on experience in reducing inference latency and optimizing AI infrastructure.
• Technical Skills:
• Proficiency in Python, with experience in ML libraries such as TensorFlow, PyTorch, and related frameworks.
• Expertise in CI/CD pipelines, version control (Git), and orchestration tools.
• Familiarity with Generative AI, prompt engineering, and deploying models at scale.
• Soft Skills:
• Excellent problem-solving skills with the ability to tackle complex challenges independently.
• Strong communication skills, with the ability to translate technical concepts for non-technical stakeholders.
• A proactive mindset with a focus on continuous learning and staying updated with industry trends.

Don’t just take our word for it  

Zeals has received the LinkedIn Top Startups award twice in a row. We believe this is a testament to our growth and our ability to attract great talents–who could potentially be your colleague! 

Our Tech Stack

• Backend Languages: Go, Python, Elixir
• Frontend Languages: HTML, CSS, JavaScript (TypeScript, React.js, Recoil, Zod, Tanstack, etc)
• Mobile Languages: None
• Infrastructure: Google Cloud Platform, GCP Pub/Sub, Kubernetes, MongoDB, MySQL, Postgres, BigQuery, Elasticsearch, Qdrant
• Configuration management tool: Terraform
• CI/CD: Github Actions, ArgoCD, CircleCI
• Monitoring: Grafana, GCP Cloud Monitoring, GCP Logging, GCP Cloud Trace, Opsgenie
• Data: BigQuery, Parquet, Spark, Scala, Python, dbt
• Knowledge Tools: Confluence
• Other: GitHub / Slack / Jira
• Process: Scrum

Ready to roll up our sleeves

We obsess over bringing hospitality to everyone–be it our customers or you! You’re an individual with unique needs. So, While we can’t attract you with ping-pong tables or all-you-can-drink espresso bars, we promise you a once-in-a-lifetime experience and co-workers that you can brag about to your friends, family, or anyone you meet the next time you go out for a party! . As an employer we are ready to roll-up our sleeves and change the status quo if it means to make your experience better!

• Salary Range : competitive (performance review every 6 months)
• Performance review: twice a year
• Holidays: 10 days paid holidays during the first year, weekends off, national holidays, summer and winter break. 
• Visa support: We sponsor visas for the right candidates. You can expect full visa support from our professional HR team. 
• Flexible working: Highly flexible, remote-first international organization 
• For Japanese residence: Work from anywhere, interim work from overseas, full flex time
• Housing allowance (within 1.5KM away from office)
• Club activity allowance
• Shuffle Lunch allowance (Cross department lunch)
• Zeals Bar (bi-monthly free flow beer party)