Computational Linguist with Gen AI experience

Sigma AI is a global training data collection, preparation and annotation services company. With 30+ years of experience in the data annotation space, we support companies with the right mix of people, processes and technology to train smarter AI that serves humans better.

About the Job 

We’re looking for a versatile Computational Linguist to join our R&D team focused on evaluating and supporting Generative and Agentic AI systems. This role combines linguistic expertise, data analysis, and hands-on experimentation with large language models. You’ll help design annotation workflows, create and refine guidelines and internal documentation, prototype task-specific evaluation metrics, configure annotation tools, and analyze annotator, model and system performance using real-world data, contributing to papers and articles as needed. The ideal candidate should demonstrate technical leadership in driving complex projects from concept to delivery. 

This is a hybrid linguistics + data science role: ideal for someone who can move between qualitative language analysis and quantitative evaluation. You’ll work cross-functionally with researchers and annotators to design innovative, rigorous, and scalable evaluation processes for LLM-powered workflows.  

Required Qualifications 

• Master’s degree (or equivalent experience) in Computational Linguistics, NLP, Linguistics, or a related field 

• 2+ years of experience in NLP or AI projects (industry or research) 

• At least one year of experience with Gen AI and/or Agentic AI  

• Experience using and fine-tuning transformer-based language models (e.g., BERT, GPT) 

• Proficiency in Python programming 

• Proficient with NLP and data science libraries: pandas, numpy, scikit-learn, NLTK 

• Experience with generative AI SDKs and frameworks (e.g., OpenAI, Google, Anthropic, LangChain) 

• Comfortable with Linux environments and Bash scripting 

• Experience working with public datasets (e.g. Hugging Face, Kaggle) 

• Familiarity with LLM behavior, prompt-based evaluation, and generative model outputs 

• Comfortable with structured data formats (JSONL, CSV), Jupyter notebooks, and pandas-based analysis 

• Experience using Git for version control and collaborative development 

• Understanding of model evaluation methodologies, including human-AI comparison and red teaming 

• Strong written communication skills for documenting experiments and results 

• Experience working in cross-functional or research-oriented teams 

• Fluent in English 

Preferred Qualifications 

• Strong understanding of current trends and techniques in generative AI 

• Experience with annotation tools (e.g., Label Studio, Prodigy) and quality metrics for human data 

• Experience designing annotation tasks and workflows (e.g., Label Studio or similar tools) 

• Experience creating and curating bespoke datasets 

• Familiarity with evaluation challenges in creative or subjective NLP tasks 

• Understanding of linguistic typology, multilingual NLP, or sociolinguistic variation 

• Experience working in WSL environments 

• Experience collaborating with annotation teams and working with QA processes