Logo for DevRev

Software Engineer - AI Performance

DevRev

Role overview

Qualifications

  • 3–10+ years of experience as a Software Engineer in backend, systems, or infrastructure roles
  • Strong foundation in Python or similar backend languages — Go, Java, C#, or Node.js
  • Hands-on experience in building APIs and cloud-native architectures (AWS, GCP, or Azure; Kubernetes a plus)
  • Understanding of metrics, profiling, or observability tools (Grafana, Prometheus, ELK, or similar)

Responsibilities

  • Participate in a structured training track to develop expertise in AI benchmarking, profiling, and performance tuning
  • Build and scale benchmarking infrastructure for evaluating AI systems in enterprise settings
  • Design agent evaluation pipelines that measure reasoning, accuracy, alignment, and user outcomes
  • Collaborate with AI and Infra teams to instrument model-serving systems (vLLM, Triton, Ray, or Kubernetes-based deployments)

Key facts

Hard skills

Other skills

  • Curiosity
  • Analytical Skills

About the company

DevRev logo

DevRev

Computer Software / SaaS

At DevRev, we're building the future of work with Computer – your AI teammate. Unlike traditional tools, Computer unifies all your data sources, tools, and workflows into a single AI-ready platform, giving employees real-time insights, proactive suggestions, and powerful agentic actions. It extends your existing software with AI-native apps and agents that work alongside your teams and customers – updating workflows, coordinating across teams, and eliminating repetitive work. We call this Team Intelligence: human-AI collaboration that breaks down silos, brings people back together, and frees you to solve bigger problems. Backed by Khosla Ventures and Mayfield with $150M+ raised, DevRev is trusted by global companies across industries.

Company details

Company typeScaleup
IndustryComputer Software / SaaS
Company size501 - 1000

Links

Website LinkedIn See all jobs

Your match analysis

See how your profile stacks up against this role.

We compared the job requirements to your profile to show where you're strong and where you fall short.

Job description

About DevRev

At DevRev, we're building the future of work with Computer – your AI teammate. Unlike traditional tools, Computer unifies all your data sources, tools, and workflows into a single AI-ready platform, giving employees real-time insights, proactive suggestions, and powerful agentic actions. It extends your existing software with AI-native apps and agents that work alongside your teams and customers – updating workflows, coordinating across teams, and eliminating repetitive work. We call this Team Intelligence: human-AI collaboration that breaks down silos, brings people back together, and frees you to solve bigger problems. Backed by Khosla Ventures and Mayfield with $150M+ raised, DevRev is trusted by global companies across industries.

About the role

DevRev is ushering in the SaaS 2.0 era — where enterprise software is not just connected, but intelligent. Our mission is to reintroduce the Computer to the enterprise: not as a form filler or a workflow router, but as an agent that thinks, reasons, and acts on behalf of users.
We’re looking for seasoned software engineers who have built and scaled applications, APIs, and cloud architectures — and are ready to level up into AI performance and benchmarking.
You don’t need prior AI experience — just strong engineering fundamentals, curiosity, and the drive to learn. DevRev will provide structured training and mentorship to help you bridge into this next frontier of engineering.
 
What you'll do
  • Participate in a structured training track to develop expertise in AI benchmarking, profiling, and performance tuning.
  • Build and scale benchmarking infrastructure for evaluating AI systems in enterprise settings.
  • Design agent evaluation pipelines that measure reasoning, accuracy, alignment, and user outcomes.
  • Participate in a structured AI benchmarking training track, gaining expertise in profiling, telemetry, and performance tuning.
  • Collaborate with AI and Infra teams to instrument model-serving systems (vLLM, Triton, Ray, or Kubernetes-based deployments).
  • Develop data and telemetry pipelines that capture model performance, latency, and quality.
  • Contribute to DevRev’s open benchmarking frameworks, defining how intelligent SaaS software is measured globally.
  • Work with cross-functional teams to ensure benchmarks reflect real-world customer value and system impact.
What you'll bring
  • 3–10+ years of experience as a Software Engineer in backend, systems, or infrastructure roles.
  • Strong foundation in Python or similar backend languages — Go, Java, C#, or Node.js.
  • Hands-on experience in building APIs and cloud-native architectures (AWS, GCP, or Azure; Kubernetes a plus).
  • Understanding of metrics, profiling, or observability tools (Grafana, Prometheus, ELK, or similar).
  • Curiosity and commitment to learn AI benchmarking, LLM evaluation, and performance frameworks.
  • Strong analytical and product intuition — you care about what the benchmark represents, not just what it measures.
  • Bonus: exposure to vector search, model tuning (LoRA, DPO), or open-source contributions.

 

Why This Role Matters

Agents are not APIs — they reason, adapt, and learn. But with that power comes ambiguity in how we measure success. At DevRev, we believe the benchmarks of the past aren’t enough for the software of the future.This role is your opportunity to design the KPIs of the AI-native enterprise — to bring rigor to systems that reason, and structure to software that thinks.Join us to shape how intelligence is measured in SaaS 2.0.

DevRev is an equal opportunity employer and does not discriminate on the basis of race, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition, or any other basis protected by law.

Apply once. Then go straight to the hiring manager.

After you apply, unlock the direct contact details of the people who actually make the call. A quick follow-up makes you 5x more likely to land an interview.

MR

Marcus Rivera

Chief Revenue Officer

m.rivera@company.com
linkedin.com/in/marcusrivera
Unlocked after you apply
·

Software Engineer Related jobs

Other jobs at DevRev

Premium

Reach out to the hiring manager directly.

Gain access to the contact details of the hiring managers who actually decide, and reach out to network with them directly. That, plus more when you upgrade:

  • Full match report with fit score and gaps
  • Career diagnostics on how recruiters read you
  • Curated company matches and warm intros
  • 48h early access to new roles

Cancel anytime.