Logo for Workiy Inc.

QA Lead – Agentic AI Platform

Roles & Responsibilities

  • QA Leadership
  • Python Test Automation
  • AI/LLM Testing
  • Conversational AI Testing

Requirements:

  • Define quality standards
  • Develop automated evaluation frameworks
  • Lead test strategy
  • Ensure release readiness for AI-powered solutions

Job description

This is a remote position.

Job Description:


We are seeking an experienced QA Lead to drive end-to-end quality assurance for an advanced Agentic AI platform. This role will be responsible for defining quality standards, developing automated evaluation frameworks, leading test strategy, and ensuring release readiness for AI-powered conversational solutions. The ideal candidate will have strong expertise in Python-based test automation, AI/LLM testing, conversational AI validation, and Retrieval-Augmented Generation (RAG) systems. Responsibilities include building evaluation harnesses, creating test plans, executing regression testing, validating data quality, conducting AI safety and prompt injection testing, managing defect triage, and leading a team of QA and Data Testers. The QA Lead will collaborate closely with engineering, product, architecture, and data teams to ensure reliable, scalable, and high-quality AI solutions are delivered while maintaining robust quality gates and release standards.


Requirements

  • QA Leadership
  • Python Test Automation
  • AI/LLM Testing
  • Conversational AI Testing
  • RAG (Retrieval-Augmented Generation) Testing
  • LLM Evaluation Frameworks
  • Golden Dataset & Regression Testing
  • Intent & Context Validation
  • Multi-Turn Dialogue Testing
  • Data Quality Testing
  • Prompt Injection & AI Safety Testing
  • Release Readiness & Quality Gates
  • Defect Management & Test Strategy
  • CI/CD Integrated Testing
  • Team Leadership & Mentoring


Related jobs

Other jobs at Workiy Inc.

We help you get seen. Not ignored.

We help you get seen faster — by the right people.

🚀

Auto-Apply

We apply for you — automatically and instantly.

Save time, skip forms, and stay on top of every opportunity. Because you can't get seen if you're not in the race.

AI Match Feedback

Know your real match before you apply.

Get a detailed AI assessment of your profile against each job posting. Because getting seen starts with passing the filters.

Upgrade to Premium. Apply smarter and get noticed.

Upgrade to Premium

Join thousands of professionals who got noticed and hired faster.