Staff Product Manager (Evals)

About Workato

Workato is the leading Control and Execution Platform for Enterprise AI — the neutral platform enterprises trust to put AI to work across their business. Workato unifies data, applications, and processes into a single platform so AI can reliably orchestrate business processes in production at enterprise scale. Built on more than a decade of running mission-critical processes for over half the Fortune 500 — including Nasdaq, Amazon, Cisco, Vodafone, Atlassian, and Lucid Motors — Workato turns over 14,000 enterprise systems AI needs to act on into one governed execution layer. For more information, visit workato.com .

Why join us?

Ultimately, Workato believes in fostering a flexible, trust-oriented culture that empowers everyone to take full ownership of their roles . We are driven by innovation and looking for team players who want to actively build our company.

But, we also believe in balancing productivity with self-care . That’s why we offer all of our employees a vibrant and dynamic work environment along with a multitude of benefits they can enjoy inside and outside of their work lives.

If this sounds right up your alley, please submit an application. We look forward to getting to know you!

Also, feel free to check out why:

Business Insider named us an “enterprise startup to bet your career on”

Forbes’ Cloud 100 recognized us as one of the top 100 private cloud companies in the world

Deloitte Tech Fast 500 ranked us as the 17th fastest growing tech company in the Bay Area, and 96th in North America

Quartz ranked us the #1 best company for remote workers

Responsibilities

We're looking for a Staff Product Manager to own evaluations for AI agents at Workato — both the internal framework that helps our teams ship better AI features, and the customer-facing tools that let builders assess and improve the agents they create. This is a role with a dual mandate. Internally, you'll establish how Workato evaluates agent quality, starting with Agent Studio and expanding to other teams shipping AI capabilities. Externally, you'll build the evaluation experience that helps business technologists understand why their agents succeed or fail — and what to do about it. The right person for this role has actually written evals. You've built test suites, designed evaluation criteria, and debugged agent failures in the trenches. You know the gap between "eval theory" and "eval reality," and you can translate that practitioner knowledge into products that work for both technical teams and non-technical builders.

In this role, y ou will also be responsible to:

Define and own the evaluation framework for Workato's internal AI agent features, driving adoption across teams starting with Agent Studio

Build the customer-facing evaluation experience — how builders test, measure, and improve agents they create on Workato

Make hard calls about what evaluation complexity to expose versus abstract, balancing rigor with approachability

Partner closely with the Build Experience PM to ensure evaluation is integrated into the builder journey, not bolted on

Work with ML engineers and platform teams to ground the framework in technical reality while keeping it accessible

Establish metrics for what "good" looks like — both for internal agent quality and for customer evaluation adoption

Spend significant time with customers understanding where they struggle to assess agent performance and what mental models they bring

Requirements

Qualifications / Experience

7+ years in Product Management

Hands-on experience writing evaluations for AI/ML systems (agents, LLMs, or similar)

Track record of shipping technical products to both internal and external users

Experience driving adoption of frameworks or practices across engineering teams

Strong written and verbal communication skills

Bachelor's degree or equivalent experience

Practitioner depth in evaluations. You've written evals yourself — built test suites, designed rubrics, debugged why agents underperformed. You understand evaluation methodology not only from reading about it, but from doing it. You have opinions about what works, what doesn't, and where current approaches fall short.

Strong product management experience. You've shipped products, driven roadmaps, and led cross-functional teams. You know how to translate technical capabilities into user value and write specs that don't leave details to chance.

Technical translation ability. You can take complex evaluation concepts and make them accessible to business technologists without dumbing them down. You understand the difference between hiding complexity and organizing it.

Internal influence skills. You've driven adoption of frameworks, practices, or tools across teams. You can be a credible partner to ML engineers while advocating for what internal teams actually need.

Greenfield comfort. You've defined products from ambiguity — scoped v1s, made bets with incomplete information, and iterated based on what you learned. You don't need an existing playbook to be effective.

B2B product sensibility. You see enterprise conventions as problems to solve, not constraints to accept. You're drawn to products that make complex workflows feel elegant.

Nice to Have

Experience with agent architectures, RAG systems, or LLM application development

Background in ML engineering, solutions architecture, or technical program management before PM

Experience building developer tools or platform products

Familiarity with evaluation frameworks (e.g., human eval pipelines, automated benchmarks, red-teaming)

(REQ ID: 2538)