Role overview

Qualifications

2+ years of QA experience, including ownership of test strategy for a product or major feature area
Strong analytical and problem-solving skills, with attention to both functional correctness and AI output quality
Excellent written communication, with the ability to write clear, concise, actionable bug reports backed by evidence
Experience with bug tracking tools and test management workflows

Responsibilities

Define and own the test strategy, process, and QA playbook for the product
Evaluate AI output quality across dimensions like accuracy, citation reliability, and appropriately scoped reasoning
Build automated guardrails integrated into CI/CD that cover critical flows
Partner with legal domain experts to understand how different customer types use the product, and translate that into test coverage

Key facts

Remote from: California (USA)
Full time
Mid-level (2-5 years)
QA Engineer
English

Hard skills

AI Testing User Interface Testing Bug Tracking And Management CI/CD Legal Technology Computer Programming

Other skills

Quality Assurance
Analytical Skills
Problem Solving
Communication

About the company

Filevine

Computer Software / SaaS

Filevine is changing the way legal work gets done for law practitioners and their clients. As the leading legal work platform, Filevine is dedicated to empowering all organizations with tools to simplify and elevate complex, high-stakes legal work. Powering everything from document management and client communication to contract lifecycle management and business analytics, over 25,000 legal professionals use Filevine daily to deliver excellence in every contract, deadline, and result. Filevine is recognized on the Deloitte Fast 500, has been named one of the Utah Business Fast 50 and is among the top 50 fastest-growing privately-owned software companies according to the 2021, 2022, and 2023 Inc. 5000 list.Filevine believes in a brighter future where the intersection of legal work and business is made more seamless, transparent, and effortless for all legal professionals and everyone they interact with through the power of legal technology.

Company details

Company typeScaleup

IndustryComputer Software / SaaS

Company size201 - 500

Links

Website LinkedIn See all jobs

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

Filevine is a Legal AI company delivering Legal Operating Intelligence for the future of legal work. Grounded in a singular system of truth, Filevine brings together data, documents, workflows, and teams into one unified platform—where modern legal work happens with clarity and consistency.

Powered by LOIS, the Legal Operating Intelligence System, Filevine connects context across every matter to transform legal operations from reactive to proactive. LOIS reads, understands, and reasons across your data to surface insight, automate complexity, and give professionals the clarity and confidence to see more, know more, and do more. Fueled by a team of exceptional collaborators and innovators, Filevine’s rapid growth has earned AI awards and recognition from Deloitte and Inc. as one of the most innovative and fastest-growing technology companies in the country.

Role Summary:

This role owns quality for LOIS for Word, Filevine's AI-powered legal drafting product. You'll help build and own the QA function: the test strategy, the process, and the automation. You'll work at the intersection of legal AI and product quality, evaluating everything from AI output accuracy to UI behavior across the product. A big part of this role is moving quality upstream, so it's built into how features are planned and shipped from the start.

The ideal candidate has 2+ years of QA experience, brings a sharp eye for both functional bugs and AI quality issues, is comfortable building from scratch in ambiguity, and gets genuinely uncomfortable shipping something they haven't tried to break.

What You'll Do:

Define and own the test strategy, process, and QA playbook for the product

Move QA upstream by partnering with product and engineering early, so test plans are ready before features ship

Evaluate AI output quality across dimensions like accuracy, citation reliability, and appropriately scoped reasoning

Design and maintain evaluation sets that account for non-deterministic output and catch regressions over time

Test how AI-generated edits apply to documents, including accuracy and formatting fidelity across document structures

Build automated guardrails integrated into CI/CD that cover critical flows

Write structured, reproducible bug reports

Partner with legal domain experts to understand how different customer types use the product, and translate that into test coverage

Approach QA with a product and design mindset. If something works but could be better, we want to hear it

Compare behavior across product surfaces and document discrepancies

What You'll Need:

2+ years of QA experience, including ownership of test strategy for a product or major feature area

Track record of building QA process and automation from scratch in a fast-moving environment

Comfort operating in high ambiguity, where features ship quickly and you figure out how to test them fast

Strong analytical and problem-solving skills, with attention to both functional correctness and AI output quality

Ability to reason about non-deterministic systems and define what "correct" means when there's no single right answer

Excellent written communication, with the ability to write clear, concise, actionable bug reports backed by evidence

Experience with bug tracking tools and test management workflows

Nice To Have:

Experience testing AI or LLM-based products, especially for output accuracy and eval design

CI/CD experience setting up pipelines and release gates for automated tests

A design or UX sensibility, with an eye for where product workflows could be smoother

Prior experience in legal technology or comfort reading legal contracts

Programming experience beyond test automation

Compensation Information: $105,000 - $150,000

The salary range represents the low and high end of the salary range for this position. The total compensation package for this position will be determined by each individual’s location, qualifications, education, work experience, skills and performance.

An individual’s final offer will be weighted by their primary work location and expertise.

Cool Company Benefits:

- A dynamic, rapidly growing company, focused on helping organizations thrive

- Medical, Dental, & Vision Insurance (for full-time employees)

- Competitive & Fair Pay

- Maternity & paternity leave (for full-time employees)

- Short & long-term disability

- Opportunity to learn from a dedicated leadership team

- Top-of-the-line company swag

Privacy Policy Notice

Filevine will handle your personal information according to what’s outlined in our Privacy Policy.

Communication about this opportunity, or any open role at Filevine, will only come from representatives with email addresses using "filevine.com". Other addresses reaching out are not affiliated with Filevine and should not be responded to.

#LI-DNI