Evaluation Specialist Sr

As an Evaluation Specialist Sr supporting the VHA AI Education Hub (TOPR 0032), you will design and implement comprehensive evaluation systems that measure the adoption, impact, and effectiveness of AI education resources across the Veterans Health Administration. Working at the intersection of federal healthcare and emerging technology, you will help the VA understand how AI literacy programs are reaching and transforming the workforce that serves our nation's Veterans. This is a high-visibility role that directly informs how one of the largest integrated healthcare systems in the world invests in and expands its AI education strategy.

The ideal candidate is a rigorous, data-driven evaluator who is equally comfortable translating complex findings into clear executive narratives and rolling up their sleeves to design a survey instrument or facilitate a structured stakeholder interview. You thrive in a collaborative federal environment, can manage competing priorities with minimal oversight, and bring a genuine passion for evidence-based decision-making.

Design and implement end-to-end evaluation systems to assess program impact, resource adoption, and learning outcomes associated with the VHA AI Education Hub's initiatives.

Apply a mixed-methods approach — including content analysis, quantitative tracking, surveys, focus groups, and structured interviews — to generate robust, actionable insights on program performance.

Oversee the generation, formatting, and communication of performance reports, ensuring findings are accurately translated into strategic recommendations and integrated into current program strategies.

Meet regularly with Government stakeholders to discuss evaluation progress, present findings, propose new assessment initiatives, and help establish organizational priorities for the AI Education Hub.

Develop data-driven sustainment and expansion recommendations that guide the long-term growth of AI education resources, drawing on evaluation evidence and emerging federal healthcare workforce needs.

Conduct secondary research and environmental scans to benchmark the VHA AI Education Hub against comparable federal, healthcare, and AI literacy programs, informing continuous improvement efforts.

Minimum 5 years of experience designing and implementing program evaluation systems, with demonstrated expertise in both quantitative and qualitative assessment methodologies.

Bachelor's degree (BA/BS) in Evaluation, Public Health, Public Policy, Social Sciences, Education, or a related field required.

Prior experience supporting federal agency clients — preferably within VA, VHA, DoD, or another federal health or workforce development context — with the ability to navigate government stakeholder environments effectively.

Ability to obtain and maintain VA network access and meet all VHA personnel security requirements, including applicable background investigation requirements for contractor personnel.

Must be legally authorized to work in the United States without sponsorship, now or in the future, in compliance with federal contractor employment eligibility requirements.

Demonstrated proficiency with evaluation tools and methods including survey design, focus group facilitation, structured interviews, and secondary/desk research, with experience translating findings into executive-level reports and strategic recommendations.

1. Experience evaluating technology adoption, digital literacy, or AI/emerging technology training programs within a federal or large healthcare organization. 2. Familiarity with VHA or broader VA organizational structures, workforce development priorities, or health informatics initiatives. 3. Proficiency with data visualization or reporting tools (e.g., Power BI, Tableau, or equivalent) to communicate evaluation findings to non-technical government audiences. 4. Master's degree in Evaluation, Public Health, Organizational Development, or a related discipline. 5. Knowledge of federal performance management frameworks such as OMB guidance, GPRA/GPRAMA, or HHS/VA evaluation policies that govern program assessment and reporting.