QA Engineer

Founded in 2012, H2O.ai is on a mission to democratize AI. As the world’s leading agentic AI company, H2O.ai converges Generative and Predictive AI to help enterprises and public sector agencies develop purpose-built GenAI applications on their private data. With a focus on Sovereign AI—secure, compliant, and infrastructure-flexible deployments—H2O.ai delivers solutions that align with the highest standards of data privacy and control.

Our open-source technology is trusted by over 20,000 organizations worldwide, including more than half of the Fortune 500. H2O.ai powers AI transformation for companies like AT&T, Commonwealth Bank of Australia, Chipotle, Workday, Progressive Insurance, and NIH.

H2O.ai partners include NVIDIA, Dell Technologies, Deloitte, Ernst & Young (EY), Snowflake, AWS, Google Cloud Platform (GCP), VAST Data and MinIO. H2O.ai’s AI for Good program supports nonprofit groups, foundations, and communities in advancing education, healthcare, and environmental conservation. With a vibrant community of 2 million data scientists worldwide, H2O.ai aims to co-create valuable AI applications for all users.

H2O.ai has raised $256 million from investors, including Commonwealth Bank, NVIDIA, Goldman Sachs, Wells Fargo, Capital One, Nexus Ventures and New York Life.

For more information, visit www.h2o.ai.

About This Opportunity

You will be a core QA engineer responsible for the end-to-end quality and reliability of h2o.ai's product portfolio.

This is a remote position, with a preference for candidates based in Canada within the PST timezone.

• Design, build, and maintain Python-first automation frameworks (pytest + Playwright + async) for UI, API, and end-to-end workflows of h2o.ai's product portfolio.
• Perform deep exploratory and manual testing of the h2o.ai's product portfolio web UI (chat interfaces, document upload/processing, agent builder, maker suite, evaluation hub, enterprise admin console).
• Use h2o.ai's product portfolio itself (and other GenAI tools) in creative ways to:
  • Generate realistic test documents, datasets, and edge-case prompts.
  • Auto-generate or refine test cases via prompt engineering.
  • Rapidly summarize and debug massive, complex log files (e.g., Kubernetes pods, etc.).
  • Explain cryptic LLM traceback chains or hallucination root causes in seconds instead of hours.
• Root-cause of difficult, intermittent failures in distributed RAG/LLM systems by combining traditional log analysis with GenAI-assisted debugging.
• Create and execute chaos experiments targeting LLM routing, vector database latency, GPU OOM, retrieval failures, and token-limit edge cases.
• Build and manage ephemeral h2o.ai's product portfolio clusters on Kubernetes for testing (Helm, custom operators).
• Own UI regression suites (Playwright) and accessibility testing.
• Write reproducible, high-quality bug reports that developers love and regularly verify fixes across the full stack.
• Collaborate closely with the h2o.ai's product portfolio feature teams in an Agile environment and influence testability from the design phase.

• 2-4 years of QA experience with a strong mix of automation and hands-on manual/exploratory testing.
• Decent Python skills and experience building maintainable test frameworks from scratch.
• Real-world experience testing modern React/TypeScript web applications and writing bulletproof Playwright or Selenium tests.
• Hands-on Kubernetes experience in a testing or test-environment context (kubectl, Helm, writing manifests, debugging pods).
• Proven ability to use generative AI tools daily to accelerate debugging, test-data creation, and log analysis (you’ve already used h2o.ai's product portfolio, ChatGPT, Claude, or similar in your current QA workflow).
• Comfort reading and triaging complex logs from LLM frameworks, vector DBs, and tracing systems.
• Solid grasp of CI/CD (GitHub Actions preferred) and infrastructure-as-code concepts.

• Prior experience testing RAG systems, agentic workflows, or enterprise chat/assistant platforms.
• Experience with visual diffing of generated outputs (documents, charts, markdown).
• Chaos engineering on Kubernetes (Chaos Mesh, Litmus) or GPU workloads.
• Familiarity with Chaos engineering principles.
• Basic understanding of containerization (Docker/Kubernetes concepts like pods and kubectl) in a testing context.

• Market leader in total rewards
• Remote-friendly culture
• Flexible working environment
• Be part of a world-class team
• Career growth
• Base salary range: $80,000 - $95,000 CAD