AI Safety Data Scientist (Contractor)

The Personalization mission makes deciding what to play next easier and more enjoyable for every Client listener, driving loved features like Blend and Client Weekly. This role joins the team to help measure and monitor safety risks across Spotify's conversational and agentic AI products. The AI Safety Data Scientist will investigate emerging risks and establish robust measurement and monitoring systems that directly inform model, policy, and product improvements.

TOP 3 REQUIRED SKILLS

• Python (Hands-on development for pipelines and data analysis)
• SQL (Database querying and data extraction)
• AI Safety (Delivered safety evaluations or mitigations for real AI/ML products)

ADDITIONAL DESIRED SKILLS

• LLM-as-a-judge workflows / calibrating LLM judges
• Multi-turn or tool-using agent evaluations
• Multilingual safety evaluations
• Human-in-the-loop evaluation design

IDEAL CANDIDATE PROFILE

The ideal candidate is an experienced Data Scientist who brings proven hands-on experience delivering AI safety evaluations or mitigations for live production machine learning models. They are comfortable structuring ambiguous problems, defining metrics from scratch, and navigating a complex cross-functional stakeholder environment across engineering, research, policy, and Trust & Safety. Excellent written communication skills and a healthy dose of curiosity and skepticism are key to success in this role.

• Develop risk monitoring systems for conversational, recommender, and tool-using AI systems.
• Design safety measurements for single- and multi-turn features, including production monitoring metrics, risk-specific decision criteria, and model performance metrics.
• Build reusable Python pipelines, LLM-as-a-judge workflows, and reporting dashboards.
• Work directly with Product, Engineering, and Trust & Safety to translate insights into safety mitigations and improvements.
• Communicate technical results clearly to both technical and non-technical stakeholders.
• Ensure complete monitoring tool coverage for safety visibility as new AI features roll out over the next 6–12 months.