Offer summary

Qualifications:

1-2+ years experience in conversation-AI QA, CX analytics, or contact-center QA., Strong understanding of GenAI concepts like LLMs and prompt engineering., Excellent attention to detail to identify nuances in call data., Ability to translate call logs into clear, concise insights..

Key responsibilities:

Review and annotate customer calls to assess quality.

Identify and report bugs, mis-routes, and UX issues.

Create actionable tickets with detailed call data and suggested improvements.

Collaborate with cross-functional teams to implement fixes and improvements.

Job description

We have just launched Hearth AI Receptionist—an LLM-powered call-handling agent that answers the phone 24/7, books jobs, and syncs every detail back to our CRM. To keep raising the bot’s containment rate, accuracy, and customer delight, we need a detail-obsessed AI Quality Assurance Specialist who will:

Audit real customer calls and transcripts every day.
Surface bugs, mis-routes, hallucinations, and UX friction the moment they appear.
Translate raw call data into crisp product insights and prioritized fixes for our engineers and PMs.
Own the feedback loop that makes our conversational AI smarter week after week.

What You’ll Do

Review & annotate calls – Score 150–200 calls/week against a rubric (intent detection, booking accuracy, brand tone, compliance).
Write clear tickets – Reproduce issues with call IDs, transcripts, and suggested prompts or flow tweaks
Curate training data – Flag high-quality examples and edge cases for fine-tuning our LLM and speech models.
Partner cross-functionally – Present weekly findings to Product, Data Science, and Customer Success; join sprint retros to ensure fixes land.

Who You Are

1-2+ years in conversation-AI QA, CX analytics, or contact-center QA.
Comfortable with GenAI concepts (LLMs, prompt engineering, speech-to-text).
Impeccable attention to detail—hear the nuance in a contractor’s accent and spot a mis-tagged intent.
Storyteller who can turn messy call logs into a one-page executive summary.
Bonus: experience in home-services, SaaS, or voice UX design.

Required profile