Labeler / Annotator – AI Response Evaluation (Castilian Spanish)

About the Role

We’re looking for a Castilian Spanish Labeler / Annotator to evaluate responses generated by AI systems. In this role, you’ll perform side-by-side evaluations of outputs from different AI models across a variety of real-world scenarios. Your analysis will help improve how AI systems understand and communicate in Castilian Spanish.

This is not a translation role. It is an evaluation and analysis role that requires strong judgment, cultural awareness, and careful attention to detail.

What You'll Do

• Perform side-by-side comparisons of AI-generated responses.
• Evaluate outputs for accuracy, relevance, clarity, instruction-following, and overall quality.
• Assess AI responses across general-purpose questions and answers, web search results, file-based and image-based responses, content-generation tasks, and single-turn and multi-turn conversations.
• Identify nuances in language, tone, meaning, and cultural context specific to Castilian Spanish.
• Apply detailed, scenario-specific annotation guidelines accurately and consistently.
• Document evaluation decisions clearly and provide rationale when required.
• Maintain consistent quality and judgment across a high volume of evaluations.
• Participate in training, calibration, and ongoing quality-review activities.

What You'll Bring

• Native-level or professional fluency in Castilian Spanish.
• Deep familiarity with the linguistic conventions, tone, idioms, and cultural context of Spanish as used in Spain.
• Strong English reading comprehension, including the ability to understand detailed guidelines written in English.
• Prior experience with side-by-side labeling, annotation, or comparative content evaluation.
• Excellent analytical thinking and attention to detail.
• Ability to identify subtle differences in quality, meaning, tone, and instruction-following.
• Ability to learn and consistently apply structured evaluation frameworks.
• Strong written communication and the ability to explain evaluation decisions clearly.
• Ability to work independently while maintaining alignment with team quality standards.

Preferred Qualifications

• Background in linguistics, translation, localization, or a related field.
• Experience with AI response evaluation or model-quality assessment.
• Experience with data labeling or annotation.
• Experience evaluating search relevance, content quality, or user-facing digital experiences.

Training and Qualification

• All new hires will complete a structured onboarding and qualification program that includes:
• Training sessions and guided practice exercises.
• Calibration against established quality benchmarks.
• A qualification review before moving into production work.
• Ongoing feedback and calibration to support consistency and quality.

Compensation

We offer competitive compensation aligned with local market conditions and experience.

The estimated compensation range is CLP $13,500 to CLP $15,350 per hour. The estimated full-time monthly equivalent is CLP $2,158,000 to CLP $2,455,000.

Actual compensation will be determined based on experience, skills, and internal equity.

Location and Employment Structure

Chile

This role will be hired through an Employer of Record partner to support compliance with local employment, payroll, and benefits requirements. Eligible employees will receive benefits in accordance with local requirements and the terms of their employment.