AI Product Engineer

About Replay

At its core, Replay was founded to help founders. We started by supporting startups through shutting down, but we have since expanded into unlocking a new revenue stream for all types of businesses.

In 2025, we had a unique insight: the data every company generates each day through collaboration, communication, and building is some of the most valuable training data in the world. Public and synthetic data can only get frontier models so far, so the next generation of model progress depends on real, proprietary data grounded in how actual businesses operate. We are a primary source of it, partnering directly with the frontier AI labs building what comes next.

Why Join Replay Now

We have scaled from $0 to a multi-eight-figure run rate in a matter of months

We have raised from top-tier investors, including Floodgate, Afore, Ludlow, and Hustle Fund

We are small enough that you will carry outsized responsibility and grow as quickly as the company does

You will partner with and build for some of the fastest and most important companies in the world

You will help build a massive, category-defining business from the ground floor

The Role

This is a production product-engineering role for someone who has shipped and operated LLM-backed software—not a prompt-engineering, research, or company-wide AI strategy position.

As our Senior AI Product Engineer, you will turn our early dissolution support agent into a trustworthy product that resolves well-bounded customer needs and establishes the right operating model for more complex workflows. You will move between customer experience, application code, retrieval, tool contracts, model behavior, evaluation, permissions, observability, rollout, and production learning. The goal is correct resolution and customer trust—not maximum deflection or autonomy.

What You'll Do

Own AI-assisted dissolution support from the customer problem through production behavior, measurement, and iteration

Design retrieval, context, structured outputs, tool contracts, orchestration, and deterministic boundaries for grounded, inspectable behavior

Build clear answer, status, no-action, and human-handoff experiences that remain useful when evidence or authority is incomplete

Create representative, versioned evaluations for routing, grounding, usefulness, safety, stability, and real customer outcomes

Ship with explicit permissions, tenant boundaries, privacy controls, auditability, canaries, rollback, and recovery

Instrument runtime and tool reliability, latency, cost, repeat contact, support effort, and serious failure modes

Turn production failures into durable product, evaluation, and system improvements

Determine whether complex workflows should be automated, AI-assisted, structured for a human, or deliberately remain human-owned, then build the approved product approach

Simplify, replace, or remove agentic components when deterministic software or a clearer product experience would work better

Work closely with Product, Support, Security, domain experts, and full-stack engineers who own the surrounding Dissolution product

What Success Looks Like

Customers get correct, useful resolution for a meaningful set of dissolution needs—not merely fewer human replies

Unsupported claims and unsafe actions remain inside explicit launch guardrails, with sensitive failures treated as stop-ship issues

Human handoffs are timely, accurate, and carry enough context to help the customer rather than restart the conversation

New intents move from evidence design through safe release using repeatable evaluation, tool, observability, and rollout infrastructure

Quality, runtime reliability, latency, cost, privacy, and customer effort remain visible as the product grows

At least one valuable multi-step workflow has an evidence-backed operating model—automated, AI-assisted, or deliberately human-owned—with clear authority, auditability, and recovery

You Might Thrive Here If

You have personally owned a production software product, including an LLM-backed capability beyond a prototype

You are a strong product engineer who can build across customer experience, application code, backend systems, AI behavior, and production operations

You understand retrieval, context selection, structured outputs, tool use, orchestration, evaluation, and observability—and know when simpler, deterministic software is the better tool

You can turn ambiguous user needs into explicit evidence, state, authority, and failure boundaries

You have designed for unsupported claims, uncertainty, permissions, privacy, human escalation, rollback, and recovery

You use representative evidence to make ship, revise, or stop decisions rather than optimizing demos or one aggregate score

You enjoy learning a consequential domain and working directly with Product, Support, Security, and engineering partners

You use modern AI development tools fluently and verify their output with the same rigor you apply to product behavior

This Role May Not Be for You If

You want to focus primarily on model research, prompt iteration, or AI infrastructure without owning the complete customer and production outcome

You believe more autonomy, more model calls, or a more sophisticated agent framework is inherently better

You prefer to hand off evaluation, security, observability, or production operation after a prototype works

You want a company-wide AI charter rather than focused ownership of the Dissolution product

Bonus

Experience building customer-support, operations, or multi-step workflow agents

Experience with LangGraph, LangChain, or comparable orchestration approaches

Experience with typed tool protocols, retrieval systems, golden datasets, offline evaluation, shadow deployments, or model-based judges

Experience with privacy-sensitive, multi-tenant, audited, legal, financial, or other high-trust products

Strong Python plus TypeScript, React, Node.js, or comparable full-stack experience