About the Role
We’re seeking an experienced Senior Code Reviewer with deep Python expertise to review evaluations of AI-generated code responses. You’ll ensure annotation teams maintain high standards for instruction-following, correctness, and functionality in AI training datasets.
Key Responsibilities
● Review and audit annotator evaluations of AI-generated Python code
● Validate code functionality, security, and best practices across multiple programming
Languages
● Execute proof-of-work methodology using Docker-based environments
● Provide constructive feedback to maintain annotation quality standards
● Work within established guidelines for evaluation consistency
Required Qualifications
● Minimum 4+ years in Python development, QA, or code review
● Strong Python expertise, including syntax, debugging, and testing
● Docker proficiency for environment isolation and testing
● Excellent written communication and documentation skills
● Experience with structured QA or annotation workflows
Preferred Qualifications
● Experience in AI training, LLM evaluation, or model alignment
● Familiarity with RLHF pipelines and annotation platforms
● Advanced Docker knowledge for complex environment replication
Why Join Us?
● Work at the intersection of AI and software development
● Remote-first culture with flexible arrangements
● Competitive hourly compensation
● Early-stage pipeline project with long-term potential
Application Process
Two-Stage Assessment:
- Technical Task (60 minutes) – Coding assessment with recorded responses
- Technical Interview (25 minutes) – Only for candidates who pass the task
Requirements:
● Individual candidate engagement model with managed oversight
● Early submission with competitive rates preferred
● Multiple coding languages required (list provided separately)
● Available for immediate start upon project finalization
Ready to shape the future of AI? Apply now and become our Senior Code Reviewer!
