AI Analyst II

• Designs, builds, and administers moderately complex AI solutions and agents using Copilot, Copilot Studio, Power Automate, and other approved tools.
• Provides ongoing oversight of AI solutions at the business level.
• Evaluates and optimizes AI solution behavior and identifies opportunities for configuration improvements.
• Monitors metrics and business outcomes.
• Performs testing, validation, and quality review of AI solution outputs to ensure accuracy, usability, and alignment with expected business outcomes.
• Maintains documentation related to AI solution configuration, workflows, prompts, user guidance, and performance observations.
• Collaborates with technical, governance, compliance, and business partners to support responsible use, adoption, and ongoing enhancement of AI solutions.
• Perform any other job related duties as requested.

• Bachelor's degree in a related field required
• Equivalent years of relevant work experience may be accepted in lieu of required education
• Three (3) years of experience in healthcare operations, business analysis, automation, digital transformation, technology solutions, or a related field required
• Experience to include a background in AI-enabled tools, digital engagement, automation, or conversational technologies required
• Experience setting up or improving service journeys, knowledge resources, workflows, or AI-enabled experiences in a cross-functional environment preferred
• Experience in healthcare, health insurance, or another regulated industry is preferred

• Understanding of generative AI tools, AI agents, large language model behavior, prompt patterns, agent instructions, model limitations, knowledge retrieval, and evaluation methods
• Aptitude for designing AI-enabled healthcare service experiences, customer or employee journeys, conversational flows, and operational workflows
• Ability to translate complex healthcare operation’s needs into clear AI experience designs, prompts, agent instructions, business requirements, acceptance criteria, and quality standards
• Ability to support testing, validation, and quality review of AI solution outputs to ensure accuracy, usability, and alignment with expected business outcomes
• Analytical, critical thinking, problem-solving, facilitation, communication, and documentation skills
• Knowledge of responsible AI principles, privacy considerations, accessibility, security, risk controls, and human-in-the-loop review practices
• Ability to establish and maintain effective working relationships with cross-functional stakeholders
• Ability to manage multiple assignments, prioritize work, meet deadlines, and adapt in a fast-paced environment
• Proficiency with Microsoft Office tools, including Word, Excel, PowerPoint, Outlook, Teams, and related collaboration tools

• None

• General office environment; may be required to sit or stand for extended periods of time
• Travel is not typically required

Compensation Range:

CareSource takes into consideration a combination of a candidate’s education, training, and experience as well as the position’s scope and complexity, the discretion and latitude required for the role, and other external and internal data when establishing a salary level. In addition to base compensation, you may qualify for a bonus tied to company and individual performance. We are highly invested in every employee’s total well-being and offer a substantial and comprehensive total rewards package.

Compensation Type (hourly/salary):

Organization Level Competencies

• Fostering a Collaborative Workplace Culture

• Cultivate Partnerships

• Develop Self and Others

• Drive Execution

• Influence Others

• Pursue Personal Excellence

• Understand the Business

This job description is not all inclusive. CareSource reserves the right to amend this job description at any time. CareSource is an Equal Opportunity Employer. We are dedicated to fostering an environment of belonging that welcomes and supports individuals of all backgrounds.