Logo for Allata

Test Harness Engineer

Role overview

Summary

This role focuses on designing and building testing systems that ensure the quality of a technology platform at Allata. You will collaborate with engineering and product teams to implement AI-powered test harnesses and maintain automated testing processes, contributing to the overall quality assurance efforts.

Qualifications

  • Proven background in software engineering or test automation, including designing QA automation frameworks.
  • Hands-on experience building AI agents or agent-based workflows.
  • Experience with at least one agent orchestration framework.
  • Strong JavaScript/TypeScript skills, with working knowledge of Python or Java.
  • Fluent English.

Responsibilities

  • Design and extend AI-powered test harnesses where agents plan, execute, validate, and analyze test scenarios.
  • Integrate LLMs with tools, APIs, and automation frameworks, implementing evaluation and guardrails.
  • Build and maintain automated UI, API, and integration test suites that run reliably in CI/CD pipelines.
  • Turn Jira acceptance criteria and Figma designs into test plans, and run exploratory, regression, and functional testing.

Key facts

Hard skills

Other skills

  • Analytical Skills
  • Communication

About the company

Allata logo

Allata

  • IT Services & Consulting
  • 201–500
Allata (pronounced a-ley-ta) is a strategy, architecture and enterprise-level application development company focused on helping clients enhance or scale business opportunities, create efficiencies and automate processes through custom technologies.We are building a different kind of firm – focused on doing exciting, transformational work for great clients and bringing caring and dedicated people to make our clients goals a reality. Our vision is to build an energized group of talented professionals that can stand strong on their own but work better as a networked team. We enable business agility at the intersection of people, process, and technology. We provide solutions and expert services to assist businesses to become more nimble, transformative, and disruptive in their respective industries. We define vision, strategy, and value creation models for shaping strategic product designs, managing, and transforming enterprise delivery.Just as strongly as we care about our clients, we feel that it is important to give back to the community and non-profits that we are passionate about. Every month, Allata donates 2% of our net income to a charitable cause our team believes in.We live by our mantra: Family comes first, clients are king, we take great care of our people.

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Allata is a global consulting and technology services firm founded in 2016, with 350+ employees across the US, India, and Argentina. We partner with some of the world's largest enterprises to move AI from idea to production — building intelligent agents, data foundations, and governance frameworks that scale.

Our teams work at the intersection of strategy, engineering, data, and design, helping clients modernize their technology, unlock data value, and create meaningful digital experiences. At Allata, you'll join an agile, cross-functional team that works closely alongside clients — making a real impact and building lasting partnerships.


We're looking for a Test Harness Engineer to join a client product team and design and build the testing systems that protect the quality of a growing technology platform. This role goes beyond writing test scripts: you will build AI-powered test harnesses where agents help plan, execute, validate, and analyze testing activities.

You'll combine strong software engineering with automation, exploratory testing, and a product-oriented mindset. You'll collaborate closely with engineering and product teams in a fast-paced environment where quality is a shared responsibility and continuous improvement is highly valued. You will make architecture and tooling decisions and take ownership of quality from design to release.


Responsibilities:
  • Design and extend AI-powered test harnesses where agents plan, execute, validate, and analyze test scenarios.
  • Integrate LLMs (OpenAI, Claude, Gemini, or similar) with tools, APIs, and automation frameworks, with evaluation and guardrails to prevent false positives.
  • Build and maintain automated UI, API, and integration test suites that run reliably in CI/CD pipelines, keeping them fast and maintainable.
  • Turn Jira acceptance criteria and Figma designs into test plans, and run exploratory, regression, and functional testing across QA and UAT.
  • Identify and communicate defects with clear reproduction steps and evidence, and drive triage with developers and product.
  • Use AI tools to accelerate test design and failure analysis, critically validating their output.
  • Mentor engineers on harness design and automation practices.

  • Requirements:
  • Proven background in software engineering or test automation, including designing QA automation frameworks that teams rely on in production.
  • Hands-on experience building AI agents or agent-based workflows, including tool/function calling, context management, and prompt engineering.
  • Experience with at least one agent orchestration framework (LangGraph, LangChain, CrewAI, or similar).
  • Ability to implement evaluation, validation, and guardrail mechanisms for AI-driven test execution.
  • Strong JavaScript/TypeScript skills, with working knowledge of Python or Java.
  • Hands-on experience with UI and API test automation tools (e.g., Cypress, Playwright, REST Assured, Postman/Newman, Karate).
  • Experience testing REST APIs and asynchronous, event-driven systems (message queues, pub/sub, or similar).
  • Experience working with SQL or NoSQL databases for test data setup and validation.
  • Experience integrating automated tests into CI/CD pipelines.
  • Fluent English.

  • Nice to have:
  • Experience with MCP (Model Context Protocol) or other tool-integration standards for AI agents.
  • Experience with performance testing tools such as k6 or Gatling.
  • Experience with accessibility testing.
  • Experience testing feature-flagged releases and gradual rollouts.
  • Experience working with complex SaaS or consumer-facing products.

  • Soft Skills:
  • Senior-level autonomy: you make sound decisions with little oversight and drive solutions end to end.
  • Strong analytical skills with a risk-based approach, and clear communication with developers and product.
  • Pragmatic about what to automate versus what to test exploratorily.
  • Dedication to continuous learning and development, for yourself and for others.
  • Related jobs

    See all Software Test Engineer jobs

    Other jobs at Allata