Lead Databricks Engineer (Senior Software Engineer II)

Job Description

Strategic Mandate

As the Data Platform Lead , you are the technical owner and steward of Nielsen's Databricks Lakehouse for Corporate function. You will drive the strategy, architecture, and operational maturity of our Databricks ecosystem end to end — from workspace and Unity Catalog design to cost governance and platform reliability. This role sits at the intersection of platform engineering and data governance: you will define the guardrails, standards, and reusable patterns that let every engineering and analytics team build confidently on a secure, scalable, and cost-efficient foundation. You will act as the "Platform Authority" for Databricks, ensuring the lakehouse scales with the business while remaining governed, observable, and audit-ready. Critically, you will build an AI-driven data platform — architecting AI-native capabilities (feature stores, vector search, GenAI-ready pipelines) directly into the platform, while also using AI-assisted and agentic engineering practices to build, manage, and scale the platform itself faster and more reliably