LLMOps Engineer-Remote

Location: Remote

Duration: 11-12 months

Rate: DOE

Key Responsibilities:

• Provide operational support for Large Language Models (LLM), including incident management, ongoing model management and maintenance, and user support.
• Monitor and maintain GenAI solutions, including performance monitoring, user interaction monitoring, and efficacy monitoring.
• Implement and uphold LLMOPS guardrails, ensuring adherence to best practices and implementation standards.
• Conduct bias audits and other guardrail checks for implemented GenAI solutions.
• Support code management, including proactive code optimization beyond basic break-fixes.

Core Skills Required:

• Strong experience in developing, deploying, and managing large language models, specifically on Microsoft Azure.
• Proven expertise in Azure cloud platforms and services, including Azure Machine Learning, Azure Kubernetes Service, and Azure DevOps.
• Solid understanding of LLMOPS guardrails implementation practices.
• Proficiency in Python for scripting and automation.
• Excellent communication skills with the ability to explain complex technical concepts to non-technical stakeholders.

Job Duties:

• LLM Operations Support:

  • Manage incidents related to LLM, ensuring timely resolution and minimal impact on operations.
  • Continuously manage and maintain LLM models to ensure optimal performance and reliability.
  • Provide user support, including troubleshooting and resolving user inquiries related to LLM.

• GenAI Solutions Monitoring and Maintenance:

  • Monitor performance metrics of GenAI solutions and take proactive measures to optimize performance.
  • Monitor user interactions with GenAI solutions to identify potential improvements or issues.
  • Monitor the efficacy of GenAI solutions, ensuring they meet predefined performance benchmarks.

• Guardrails Implementation and Monitoring:

  • Implement and enforce guardrails for LLMOPS to maintain operational integrity and compliance.
  • Conduct bias audits and other guardrail checks to ensure ethical and responsible use of AI solutions.

• Code Management Support:

  • Collaborate with development teams to optimize code beyond basic break-fixes, focusing on proactive improvements and efficiencies.

Additional Requirements:

• Experience with Terraform for infrastructure deployment and management is essential.
• Familiarity with CI/CD pipelines and version control systems.
• Ability to work independently and as part of a remote team, demonstrating strong initiative and problem-solving skills.