hirly

Judgmentlabs

Applied AI Engineer

San Francisco

See how you match this job — and similar ones. Free.

Upload your resume and hirly scores it against this role at Judgmentlabs first, then against similar open jobs, and shows where you fit and why.

PDF or DOCX, up to 12MB. No sign-up to see your matches.

Get past the screening software and onto a recruiter's desk

hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.

  • Keywords matched to this posting
  • Fit score before you apply
  • Cover letter included
Tailor my resume for this job →

Apply from your AI assistant

Connect hirly to Claude and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending. ChatGPT: manual setup today.

Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.

hirly's read of this role

Seniority
Mid level
Country
US
Work mode
On-site / unstated
First seen by hirly
1 Sept 2026

Derived automatically from the posting. Upload your resume above to see how the role scores against it.

the posting

The Role:

We are looking for Applied AI Engineers to build AI systems that use agent interaction data to understand how agents behave, evaluate them at scale, and improve them through learning and feedback.

Your research will not live on a whiteboard. You’ll work directly with real-world agent data, apply frontier methods in production, and see your work ship into the product. By making agent behavior measurable and debuggable, your systems will support teams deploying agents across finance, legal, operations, and other high-stakes workflows. You will own projects end-to-end, with significant autonomy, and work closely with the team to build self-improving agent systems.

What You'll Do:

Build AI systems to aggregate, index, and analyze large-scale long-running agent interaction data in order to extract meaningful signals

Design and implement post-training and optimization workflows to improve agents, both internally and for customers

Build agent platform infrastructure, including orchestration, runtimes, and developer tools that help teams define, test, deploy, and iterate on complex agent workflows

Build internal tools and infrastructure that support rapid experimentation, analysis, and training

Work closely with product to integrate agents into customer-facing workflows

Collaborate with external companies and research partners on frontier AI research

What We're Looking For

Every hire clears three bars, no exceptions:

Agency. You are intellectually curious, self-directed, and stay up to date with the latest research, blogs, trends, and ideas.

Depth of thought. You can reason clearly about abstract systems, and ideally have experience working on agents, RL, or the infrastructure that supports them.

Ownership. You own outcomes, not just tasks. You use freedom to experiment responsibly, make business-driven decisions, and focus first on work that moves the company forward.

More specifically, you should bring strength in at least one of the following areas:

Data quality, evaluation, benchmarking, and hands-on work with messy production data

Agent systems built or evaluated in real-world or production settings

Reinforcement learning, post-training, agents, or machine learning fundamentals

Infrastructure and systems work across training, data pipelines, evaluation, or model serving

Translating research into product while balancing customer constraints, technical tradeoffs, and business impact

Turning ambiguous problems into clear, well-designed plans

Original posting on Judgmentlabs's site ↗

Listed on hirly, a job board. hirly is not the employer: Judgmentlabs is hiring for this role.

Browse similar roles

Want this one?

Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.

Tailor my resume for this job