Psychometrician

Description

The Psychometrician is a leadership role within the competency team. This individual will play a vital role in leading efforts to develop, evaluate and analyze data related to human knowledge, skills, and abilities (KSA) in one of HealthStream’s flagship software solutions. This job is responsible for developing the statistical support metrics for item development as well as conducting validation studies of assessment performance. This individual will synthesize learner results to draw inferences about assessment difficulty, performance, learning level, etc. and will coordinate question generation, revision, and deprecation. Additionally, this person will work with subject matter experts (SME) to develop test questions that align with clinical competencies. This individual must have a solid background in research methods and statistical analysis, as well as excellent communication skills. This position will work closely with content development and executive leadership, subject matter experts, product managers, data scientists, and other internal stakeholders in assessing and improving the effectiveness of assessment tools.

ESSENTIAL DUTIES OR RESPONSIBILITIES

• Oversees the planning, implementation, validation, and management of all psychometric and technical requirements of a healthcare competency testing program.
• Oversees the data entry, administration, etc. of a database of clinical competencies and their relationships to clinical content.
• Coordinates content experts and internal stakeholders to design and improve assessments.
• Works with content development leadership to perform job analysis and test blueprint design.
• Analyzes test data with item response theory (IRT).
• Recommends appropriate analysis approaches and identifies advantages and disadvantages of several options.
• Designs scaling and equating approaches and writes associated technical specifications.
• Develops psychometric targets for test construction and creates test construction specifications.
• Constructs tests, and evaluates tests built by others according to psychometric targets.
• Assures the veracity of data.
• Assists in writing technical and research reports.
• Consult with internal stakeholders on assessment results and implementation strategies.
• Research and stay up to date with developments in the field of psychometrics.
• Communicate (orally and in writing) technical concepts in non-technical ways with internal and external stakeholders.
• Synthesize information and produce coherent summaries and recommendations.
• May provide item writing workshops.

Qualifications

1. Education, Experience and Knowledge Required:

• Master's degree or Ph.D. in educational statistics, measurement, psychometrics, or Ed. D in educational measurement or related field required.
• Four (4) or more years in educational assessment with increasing job responsibilities.
• Expert knowledge of statistical analysis and research methods
• Strong written and oral communication skills
• Experience working with healthcare-related statistics and evaluation strongly preferred.

1. Skills Required:

• Strong analytical and critical thinking skills
• Proficient in SAS, SPSS, R, Python, or other statistical software
• Use of WINSTEPS, IRTPRO, or other IRT-based software in a research or operational environment

1. Abilities Required:

• Ability to score educational assessments accurately.
• Ability to interpret and communicate results effectively to stakeholders.
• Ability to develop and validate tests and assessments.