Logo for Hello Heart

Clinical AI Quality Specialist

Role overview

Summary

The Clinical AI Quality Specialist role focuses on evaluating and monitoring the clinical behavior of Nia, an AI health assistant for cardiovascular health. You will work with the Clinical AI Quality & Governance Team to ensure that Nia meets clinical standards and effectively handles high-risk situations.

Qualifications

  • PharmD (or equivalent clinical doctorate) with an active US license.
  • 3+ years of combined clinical practice and clinical AI evaluation.
  • Experience designing or running evaluations for LLM-based systems.
  • Command of triage and escalation, crisis and self-harm handling, and medication safety.
  • Excellent written and verbal communication.

Responsibilities

  • Run Nia’s offline evaluations for clinically relevant changes and write test cases.
  • Review production conversations using defined sampling methods and prepare monthly results.
  • Test Nia's detection and handling of high-risk situations, including medical emergencies.
  • Describe failures clearly for engineering to act on and keep clinical specifications current.

Key facts

  • Remote from: United States
  • Full time
  • Mid-level (2-5 years)
  • AI Specialist
  • $130K – $143K / year
  • English

Hard skills

Key skills for this role

Other skills

  • Communication
  • Problem Solving
  • Teamwork
  • Adaptability

About the company

Hello Heart logo

Hello Heart

  • Digital Health
  • Scaleup
  • 201–500
Hello Heart is on a mission to change the way people care for their hearts. The company provides the first app and connected heart monitor to help people track and manage their heart health. With Hello Heart, users take steps to control their risk of heart attacks and stroke – the leading cause of death in the United States. Peer-reviewed studies have shown that high-risk users of Hello Heart have seen meaningful drops in blood pressure, cholesterol and even weight. Recognized as the digital leader in preventive heart health, Hello Heart is trusted by more than 130 leading Fortune 500 and government employers, national health plans, and labor organizations. Founded in 2013, Hello Heart has raised more than $138 million from top venture firms and is a best-in-class solution on the American Heart Association’s Innovators’ Network and CVS Health Point Solutions Management platform. Visit www.helloheart.com for more information.

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

About Hello Heart:

Hello Heart is on a mission to make heart attacks a thing of the past.

We’re an AI company focused exclusively on heart health, building a platform that predicts and prevents cardiac events before they happen—identifying risk up to 10 days in advance versus 10 years in traditional clinical models. 

This is already working at scale. Hello Heart has been shown to reduce inpatient hospital days by 47% and deliver ~$1,800 in annual savings per member. Hello Heart is the cardiac prevention partner to over 80% of large U.S. health plans and serves hundreds of public and private employers.

We’re defining how the #1 cause of death—heart disease—is managed in the AI era. Join us.

About the roleHello Heart’s AI team builds Nia, an AI health assistant for people managing their cardiovascular health. As a Clinical AI Quality Specialist, you will evaluate and monitor Nia’s clinical behavior before and after release, working alongside the Clinical AI Quality & Governance Team Lead. You will run evaluations, review production conversations, and turn findings into documented decisions with product and engineering. This is a hands-on role for a clinician who is comfortable working with AI tools.

Responsibilities

  • Run Nia’s offline evaluations for clinically relevant changes. Write test cases, including tricky and ambiguous ones, and score outputs against the expected clinical behavior. Build and maintain evaluation datasets and contribute to scoring methods 
  • Review production conversations using defined sampling methods. Take part in the weekly review and help prepare monthly results that state what was and wasn’t covered.
  • Test whether Nia detects and correctly handles high-risk situations, including medical emergencies, self-harm, and medication safety. Measure how often it escalates too much or too little.
  • Describe each failure clearly enough for engineering to act on, and flag your best read of the cause (for example, a wrong answer, missing patient context, or an escalation that fired too late). Make sure each unsafe or incorrect finding has an owner, a ticket, and a decision.
  • Keep clinical specifications, review records, and evidence current and auditable. Write clear, precise clinical assessments that engineering and product can act on.

Qualifications: 

  • PharmD (or equivalent clinical doctorate) with an active US license. 3+ years of combined clinical practice and clinical AI evaluation.
  • Questions assumptions and investigates before drawing conclusions. Reads current guidelines, drug labels, and primary sources, and cites them in written assessments. Uses AI tools to work faster, and verifies AI output against sources before relying on it.
  • Experience designing or running evaluations for LLM-based systems, reviewing production conversations, and using monitoring tools (e.g., Langfuse).
  • Command of triage and escalation, crisis and self-harm handling, and medication safety, grounded in current guidelines.
  • Comfortable working with AI tools and evaluation workflows.
  • Excellent written and verbal communication. Explains clinical risk clearly to engineering and product audiences, writes precise clinical assessments and documentation, and accepts feedback without defensiveness.
  • Genuine passion for how AI can improve preventive care and chronic disease management. Curious, open to learning, and comfortable with ambiguity and a fast-moving field.

Preferred Qualifications: 

  • Clinical experience in cardiovascular care or hypertension management.
  • Experience with structured rating of clinical transcripts, including agreement between raters.
  • Experience reviewing crisis and self-harm behavior.
  • Experience designing or running evaluations for LLM-based systems, or using monitoring tools such as Langfuse.

The United States base salary range for this full-time position is $130,000 to $143,000. Salary ranges are determined by role and level. Individual compensation is based on job-related skills, experience, and relevant education or training. Compensation details listed in United States job postings reflect annual salary and bonus only and do not include equity or benefits.

Hello Heart has a positive, diverse, and supportive culture - we look for people who are collaborative, creative, and courageous. Oh, and if you want to see some recent evidence of the fun things we do at Hello Heart, check out our Instagram page.  

Related jobs

See all AI Specialist jobs

Other jobs at Hello Heart