AI Product Manager, AI Platform - Quality and Evaluation

Are you passionate about building AI products that customers can trust by making quality measurable rather than subjective? Do you enjoy partnering with engineers and researchers to evaluate large language models, improve AI system performance, and turn data into product decisions? Are you excited by the opportunity to define the future of AI quality, observability, and evaluation platforms that power intelligent applications? If so, we invite you to be a part of our innovative team.

As a Product Manager on Ridgeline's AI Platform team, you'll define and execute the strategy for the systems that ensure our AI capabilities are accurate, reliable, and continuously improving. You'll lead initiatives spanning LLM evaluation frameworks, AI model benchmarking, evaluation harnesses, prompt and model experimentation, AI quality platforms, and observability capabilities that help teams understand, measure, and improve production AI performance. You'll work closely with engineering, AI researchers, design, and product teams to deliver platform capabilities that enable every AI-powered experience across Ridgeline. You'll leverage cutting-edge AI technologies and tools in a fast-moving, creative, progressive work environment while helping establish best practices for developing and operating enterprise AI products.

At Ridgeline, how we work matters as much as what we build. Ridgeliners act like owners, choose growth over comfort, and communicate with transparency. We assume positive intent, bias toward action, and bring solutions—not just problems. We celebrate wins, learn from setbacks, and thrive in a resilient, collaborative, high-performing culture. If this excites you, we’d love to meet you!

You must be work authorized in the United States without the need for employer sponsorship.

The impact you will have

Define and execute the product strategy and roadmap for Ridgeline's AI Quality Platform

Build scalable LLM evaluation frameworks and automated evaluation harnesses that enable continuous model validation

Establish standardized offline and online evaluation methodologies, golden datasets, and quality benchmarks across AI products

Define the roadmap for AI observability, enabling teams to monitor quality, latency, cost, reliability, and user outcomes in production

Partner with engineering and AI researchers to build continuous regression testing and experimentation capabilities for generative AI systems

Drive product decisions using AI quality metrics, customer feedback, and quantitative evaluation data

Collaborate with engineering teams to deliver platform capabilities that enable trusted, measurable, and continuously improving AI experiences across Ridgeline

Translate advances in foundation models, evaluation techniques, and AI tooling into platform capabilities that create meaningful customer value

Prioritize product investments using customer insights, product analytics, and measurable business outcomes

Communicate product vision, roadmap, and priorities clearly across technical and executive stakeholders

Foster a collaborative environment rooted in ownership, continuous learning, experimentation, and transparency

What we look for

5+ years of product management experience building highly technical platform, infrastructure, or developer-focused products

Demonstrated experience shipping production AI products powered by large language models

Experience defining products that incorporate LLM evaluation frameworks, AI quality measurement, and continuous model evaluation

Deep understanding of offline and online evaluation methodologies, model benchmarking, regression testing, and AI quality metrics

Experience with LLM observability platforms such as Datadog LLM Observability, Langfuse, LangSmith, Arize AI, Braintrust, Galileo, or similar AI quality and evaluation solutions

Familiarity with prompt engineering, experimentation frameworks, evaluation datasets, and LLM-as-a-judge techniques for generative AI applications

Experience partnering effectively with software engineers, ML engineers, AI researchers, and cross-functional stakeholders

Strong analytical skills with the ability to translate AI performance signals into product decisions

Excellent written and verbal communication skills

Willingness to learn about cutting-edge technologies while cultivating expertise in investment management workflows and enterprise AI

An aptitude for problem solving

Ability to communicate effectively

Serious interest in having fun at work

Bonus

Experience building products around retrieval-augmented generation (RAG), AI agents, MCP, or model orchestration frameworks

Experience with synthetic data generation and automated evaluation pipelines

Experience building internal developer platforms or ML infrastructure products

Experience in enterprise SaaS, developer platforms, or financial technology

About Ridgeline

Ridgeline is headquartered in Lake Tahoe, with offices in New York, Reno, the Bay Area, Dublin Ireland. Ridgeline is recognized by Fast Company as a “Best Workplace for Innovators,” by Frost & Sullivan as a “Technology Innovation Leader,” and by The Software Report as a “Top 100 Software Company.”

Ridgeline is proud to be a community-minded, discrimination-free equal opportunity workplace.

Ridgeline processes the information you submit in connection with your application in accordance with the Ridgeline Candidate Privacy Policy . Please review the Ridgeline Candidate Privacy Policy in full to understand our privacy practices and contact us with any questions.

This posting is for an existing vacancy.

Compensation and Benefits

The typical starting salary range for new hires in this role is listed below. In select locations (including, the San Francisco Bay Area, CA, and the New York City Metro Area), an alternate range may apply as specified below.

The typical starting salary range for this role is: $136,000-$170,000.

The typical starting salary range for this role in the select locations listed above is: $149,500-$187,000.

Final compensation amounts are determined by multiple factors, including candidate experience and expertise, and may vary from the amount listed above.

As an employee at Ridgeline, you’ll have many opportunities for advancement in your career and can make a true impact on the product.

In addition to the base salary, Ridgeline employees can participate in our Company Stock Plan subject to the applicable Stock Option Agreement. We also offer rich benefits that reflect the kind of organization we want to be: one in which our employees feel valued and are inspired to bring their best selves to work. These include unlimited vacation, educational and wellness reimbursements, and $0 cost employee insurance plans. Please check out our Careers page for a more comprehensive overview of our perks and benefits.