hirly

Doctolib

Staff AI Engineer - Agent Evaluation Platform (x/f/m)

Paris

hirly Pro

Get in front of the people hiring at Doctolib

  • Reach the hiring manager at Doctolib on LinkedIn, from your own account
  • hirly applies to this job for you, with unlimited applications
  • A resume and cover letter written for this job, unlimited
  • The hiring manager and recruiter at Doctolib, with contact details
Get Pro for this job →

$49 a month · ₹1,999 for 30 days in India · cancel any time

See how you match this job — and similar ones. Free.

Upload your resume and hirly scores it against this role at Doctolib first, then against similar open jobs, and shows where you fit and why.

PDF or DOCX, up to 12MB. No sign-up to see your matches.

Get past the screening software and onto a recruiter's desk

hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.

  • Keywords matched to this posting
  • Fit score before you apply
  • Cover letter included

Matched against 2.3M live jobs from 220,000+ employers in 200+ countries.

Tailor my resume for this job →

Apply from your AI assistant

Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.

Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.

hirly's read of this role

Role family
Engineering
Seniority
Lead / management
Country
FR
Work mode
Remote-friendly
First seen by hirly
21 Nov 2026

Derived automatically from the posting. Upload your resume above to see how the role scores against it.

the posting

Join a team of passionate and hardworking entrepreneurs to transform healthcare!

Working in the tech team at Doctolib involves building innovative products and features to improve the daily lives of care teams and patients. We work in feature teams in an agile environment, while collaborating with product, design, and business teams.

What you'll do

We are looking for a Staff AI Software Engineer to join the AI Evaluation team (EvalX) at Doctolib and help every AI product team measure, understand and improve the quality, reliability and safety of their systems.

Our AI products span patient-facing conversational agents, AI-powered phone assistants and clinical assistants supporting healthcare professionals. EvalX builds the platform, tools and methods that let these teams evaluate continuously, and our ambition is to make evaluation a core engineering practice.

In this role, you will work at the intersection of platform engineering, applied AI and product teams: you build shared evaluation capabilities, and you work directly with the teams that use them so they actually get adopted.

In this role, you will:

Design, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations) used by AI product teams across Doctolib

Work hands-on with product teams to assess how they evaluate today, define improvement plans, and help them own their evaluation quality

Design evaluation strategies for agentic systems, combining deterministic checks, LLM-as-a-judge, simulation-based testing, and human review

Build intelligent capabilities for production sampling, dataset representativeness monitoring, distribution-shift detection, and test case generation

Help teams define meaningful quality metrics, representative datasets, and reliable quality gates throughout the development lifecycle

Turn recurring product needs into reusable, self-service platform capabilities, and measure success through adoption, team autonomy, and evaluation coverage

Contribute to cross-functional initiatives that shape Doctolib's AI platform and best practices for building and evaluating agentic systems

Who you are

Before you read on, if you don't have the exact profile described below, but you feel this job description matches your skill set, we still encourage you to apply! You could be our next team mate if you:

Have 6+ years of experience as a software engineer, with significant experience using Python; or as a machine learning engineer

Have built or operated evaluation for LLM-powered or agentic systems: datasets, LLM-as-a-judge, human review, production-driven evaluation

Understand and promote software development best practices

Are product-oriented and enjoy working directly with other teams, challenging assumptions, and turning complex needs into concrete solutions

Can influence engineering practices and drive change without hierarchical authority: your job is not done until your solution is adopted

Are comfortable speaking and writing in English in your day-to-day with the team

Now, it would be fantastic if you:

Have experience with applied ML techniques such as embeddings, clustering, statistical sampling, and distribution monitoring

Have experience building developer platforms or internal tooling, or in a Forward Deployed Engineering role

Have experience in the medical domain or another regulated environment

What we offer

Free comprehensive health insurance (basic package) for you and your children

25 days of paid vacation per year, plus up to 14 days of RTT

Free mental health and coaching services through our partner Moka.care

Work from abroad for up to 10 days per year thanks to our flexibility days policy

Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib

A subsidy from the work council to refund part of the membership to a sport club or a creative class

50% reimbursement of your public transport subscription

Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth

ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib

For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support

The interview process

A first interview with a member of the Talent Acquisition team

A coding interview

An agentics system design

A behavioural interview

Reference check

Job details

Permanent position

Full Time

Workplace: Levallois

Start date : As soon as possible

If you would like to find out more about tech life at Doctolib, feel free to read our latest Medium blog articles!

At Doctolib, we believe in improving access to healthcare for everyone - regardless of where you come from, what you look like. This translates into our recruitment process: Doctolib is an equal opportunity employer. We don't just accept diversity at Doctolib, we respect and celebrate it!

The more diverse ideas are heard, the more our product will truly improve healthcare for all. You are welcome to apply to Doctolib, regardless of your gender, religion, age, sexual orientation, ethnicity, disability, or place of origin. If you have a disability, let us know if there's any way we can make the interview process smoother for you!

All the information transmitted via this form is processed by Doctolib for the purpose of managing applications. For more information on how Doctolib processes your application data, click here.

If you wish to exercise your rights or if you have any questions about the processing of your data, you can write to us at ----- Find Jobs in France on Arbeitnow

Original posting on Doctolib's site ↗

Listed on hirly, a job board. hirly is not the employer: Doctolib is hiring for this role.

Browse similar roles

Want this one?

Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.

Tailor my resume for this job