Principal Data Scientist

About the team

At Zillow Group, our mission is to help more people get home. The CoreTech Org delivers on this mission by building the foundational platforms, data, and developer capabilities that power every consumer, partner, and employee experience at Zillow.

You will work most closely with the Connections Platform (CP) — the engine that turns consumer interest into a connection with the right partner, owning the conversational experience, the lead funnel and follow-up workflows, and the shared agent services that sit underneath both. Increasingly, that experience is powered by LLM-based and agentic systems: agents that qualify leads, summarize and extract intelligence from conversations for partner teams, and drive proactive outreach. Beyond CP, you will be a measurement partner for the broader CoreTech Org — the foundational platforms every consumer, partner, and employee experience at Zillow is built on — wherever a hard data science problem needs an owner.

These are platforms with hundreds of touchpoints and dozens of dependent teams — and they have historically been among the hardest places at Zillow to measure well. CP itself is mid-transformation, shifting from NLP to LLM and agent-based systems, with agents increasingly built and relied on in production but no rigorous way to evaluate them yet. That combination — high dependency, hard-to-measure, and changing fast — is exactly where rigorous measurement creates the most leverage.

About the role

We are seeking a Principal Data Scientist to be a measurement and impact partner in CoreTech. This is a senior individual-contributor role embedded with the Data & Infrastructure Platform team and who will partner tightly with Product leadership.

You will own four outcomes: (1) make the impact of platform work legible — quantifying what shipped, what moved, and what didn’t, in terms leaders can act on; (2) define the metrics that matter for CP and the other platforms you take on — what belongs on a progress dashboard, what is a leading vs. lagging indicator, and what is vanity; (3) build the evaluation layer CP is missing — standing up rigorous, repeatable evals for its LLM and agentic systems (including LLM-as-a-judge) so the team can tell whether an agent actually works, not just that it shipped; and (4) operationalize and instrument those metrics and evals in partnership with engineering — getting the events emitted, the pipelines stable, the dashboards live, and the data trusted.

The defining challenge of this role is twofold. The first is measuring causal impact : CP has live, expensive, executive-facing questions that hinge on making a credible impact claim when a clean A/B test isn’t available — conversational and agent-driven flows that span a lead’s journey across sessions and channels, and changes that roll out to everyone at once. The second is establishing an evaluation feedback loop : much of CP now runs on LLM-based and agentic systems that were built by engineers with no rigorous way to measure whether they actually work. You will own how we evaluate these systems — defining the signals that matter, standing up eval and experimentation frameworks (including LLM-as-a-judge), and separating activity and adoption from genuine quality and outcomes. The same question recurs across CoreTech in many different areas, and you will need to help prove whether new features or infrastructure actually pay off, rather than trusting activity or adoption as proof. Owning these questions is the heart of the job.

This is a high-leverage, cross-Zillow role: your work will set the measurement bar for platforms every product team at Zillow depends on, and your point of view will directly shape how Data & Infrastructure Platform OKRs are written and reviewed at the executive level.

You will:

• •Define and operationalize the measurement strategy for the Connections Platform and the broader Data & Infrastructure Platform team — owning the throughline from raw events to executive dashboards and causal inference attribution.
• •Own the evaluation of CP’s LLM-based and agentic systems: design the evals, define the quality signals, and build the frameworks (including LLM-as-a-judge and using tooling such as Braintrust) that tell us whether an agent is actually working.
• •Own the causal inference and experimentation program for these teams: design and analyze A/B tests where you can, and where you can’t, build the quasi-experimental causal inference design (switchbacks, difference-in-difference, synthetic control, interrupted time-series) that produces a defensible estimate of impact. Know which method the situation calls for, what assumptions it rests on, and make the call for the best method to answer each hard question.
• •Be the credible authority on “what to measure” and “what to experiment on” for these teams — pushing back on vanity metrics, identifying the leading indicators we’ve been missing, and knowing which signals actually reflect the customer experience.
• •Own the impact dimension that platform measurement usually lacks — connecting adoption of CP’s agents and other platform investments to business and revenue outcomes, not just activity, speed, and quality signals.
• •Partner with engineering ICs to instrument the metrics: events, schemas, and pipelines needed to get quality measurements. Engineers will wire up the engine; you make the data infrastructure for these surfaces something teams can trust, and you shape what gets emitted in the first place.
• •Translate complex measurement realities into clear narratives for VP and C-level partners, including the trade-offs they need to consider.
• •Set the bar for how data science is practiced inside platform teams at Zillow , treating the platform itself as a product whose adoption, quality, and impact get measured as rigorously as any consumer product (Platform-as-a-Product measurement).
• •Help decipher the impact of product changes that are multi-dimensional at once – touching consumers, partners, and developers, and carrying real business and revenue stakes.

This role has been categorized as a Remote position. “Remote” employees do not have a permanent corporate office workplace and, instead, work from a physical location of their choice, which must be identified to the Company. U.S. employees may live in any of the 50 United States, with limited exceptions.

In California, Connecticut, Maryland, Massachusetts, New Jersey, New York, Washington state, and Washington DC the standard base pay range for this role is $182,100.00 - $290,900.00 annually. This base pay range is specific to these locations and may not be applicable to other locations.

In Colorado, Hawaii, Illinois, Maine, Minnesota, Nevada, Ohio, Rhode Island, Vermont, and Virginia the standard base pay range for this role is $173,000.00 - $276,400.00 annually. The base pay range is specific to these locations and may not be applicable to other locations.

In addition to a competitive base salary this position is also eligible for equity awards based on factors such as experience, performance and location. Actual amounts will vary depending on experience, performance and location. Employees in this role will not be paid below the salary threshold for exempt employees in the state where they reside.

Who you are

• •Senior data scientist with internal-customer and/or platform measurement experience. 10+ years in Data Science, UX Research, or Applied Science, including meaningful time measuring products where the customer is a developer, an internal team, or a platform. You have led the measurement of platform products — where adoption, partner teams, and downstream outcomes all matter — and you know why that’s harder, not easier.
• •Fluent in evaluating complex user-facing systems. You have designed evals for products where quality is hard to pin down – using the right tool for the job, from labeled test sets and human review to LLM-as-a-judge. You can separate adoption and