Freshfields
Senior Data Platform Engineer
Raleigh
hirly Pro
Get in front of the people hiring at Freshfields
- Reach the hiring manager at Freshfields on LinkedIn, from your own account
- hirly applies to this job for you, with unlimited applications
- A resume and cover letter written for this job, unlimited
- The hiring manager and recruiter at Freshfields, with contact details
$49 a month · ₹1,999 for 30 days in India · cancel any time
Get past the screening software and onto a recruiter's desk
hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.
- Keywords matched to this posting
- Fit score before you apply
- Cover letter included
Matched against 2.8M live jobs from 220,000+ employers in 200+ countries.
Tailor my resume for this job →Apply from your AI assistant
Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.
Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.
hirly's read of this role
- Role family
- Engineering
- Seniority
- Senior
- Country
- US
- Work mode
- On-site / unstated
- First seen by hirly
- 20 Oct 2026
Derived automatically from the posting. Upload your resume above to see how the role scores against it.
the posting
Overview of the Function
We are seeking an experienced and forward-thinking Senior Data Platform Engineer to manage, optimize, and evolve the cloud-native platforms used by the Data Office. The role combines deep data engineering capability with platform engineering accountability across Microsoft Azure, Databricks, Unity Catalog, integration and ingestion services, Collibra, and Immuta.
The successful candidate will ensure that the Data Office platform estate is secure, reliable, scalable, observable, and cost-effective. This includes optimizing performance, workloads, storage, compute, access, and platform configurations, while also designing and delivering robust data pipelines and lakehouse solutions that support analytics, business intelligence, AI, and machine learning.
This is a hands-on role working across engineering, architecture, governance, security, and operations to provide a trusted enterprise data platform and a strong developer and consumer experience.
Role Summary / Purpose of Job
The Senior Data Platform Engineer will be responsible for the day-to-day engineering and technical management of the Data Office platform ecosystem. The role will maintain platform health, availability, security, performance, and cost efficiency, while supporting platform upgrades, configuration, automation, monitoring, access management, and incident resolution.
In parallel, the role will deliver senior-level data engineering outcomes across ingestion, integration, transformation, orchestration, governance, and data access. The engineer will work with data architects, engineers, analysts, governance teams, security teams, and business stakeholders to ensure that data products are reliable, discoverable, controlled, and fit for purpose.
Key Responsibilities and Deliverables
- Manage the Data Office platform estate across Azure, Databricks, Unity Catalog, integration and ingestion services, Collibra, and Immuta, ensuring operational reliability and alignment with enterprise standards.
- Monitor and optimize platform performance, workloads, clusters, SQL warehouses, jobs, storage, and compute to improve reliability, scalability, user experience, and cost efficiency.
- Own capacity and workload management, including resource allocation, concurrency, job scheduling, autoscaling, workload isolation, and the assessment of serverless and cluster-based compute options.
- Administer platform configuration, workspaces, environments, service connections, secrets, policies, and technical integrations across development, test, and production.
- Manage data access controls and entitlements across Azure, Databricks, Unity Catalog, Collibra, and Immuta, applying least-privilege, role-based, attribute-based, and policy-based access principles.
- Implement and maintain Unity Catalog structures, including catalogs, schemas, permissions, data objects, lineage, and controlled data-sharing patterns.
- Support Collibra integration and adoption for cataloging, metadata, business glossaries, lineage, ownership, stewardship, and governance workflows.
- Design, build, and maintain secure, scalable lakehouse solutions using Azure Data Lake Storage, Databricks, Delta Lake, Spark, PySpark, Python, and SQL.
- Develop and optimize batch and streaming ingestion, integration, ETL, and ELT pipelines using Azure Data Factory, Databricks Workflows, APIs, event-based services, and appropriate orchestration tools.
- Engineer reusable integration and ingestion patterns for structured, semi-structured, and unstructured data from enterprise applications, files, databases, APIs, and event streams.
- Implement observability across platforms and pipelines, including monitoring, logging, alerting, performance metrics, usage reporting, cost reporting, failure analysis, and operational dashboards.
- Lead technical incident investigation, root-cause analysis, remediation, and problem management for platform, workload, access, and data pipeline issues.
- Automate platform deployment and configuration through infrastructure as code, CI/CD, testing, release management, and version control using tools such as Terraform, Git, Azure DevOps, or GitHub Actions.
- Implement data quality, reconciliation, validation, exception handling, recovery, and audit controls to improve reliability and trust in enterprise data products.
- Plan and execute platform upgrades, feature adoption, migrations, configuration changes, resilience testing, backup and recovery activities, and lifecycle management.
- Maintain technical standards, runbooks, architecture diagrams, support procedures, data dictionaries, operating documentation, and knowledge articles.
- Provide technical leadership, coaching, design review, and best-practice guidance to data engineers and other platform users.
- Collaborate with architecture, cloud operations, cyber security, risk, governance, analytics, and business teams to translate requirements into secure and sustainable engineering solutions.
Key Requirements
Experience and Background
- 5–10 years of hands-on experience in data engineering, data platform engineering, cloud engineering, or a closely related field.
- Proven experience managing and supporting enterprise data platforms in production, including performance, workloads, storage, compute, access, availability, and cost.
- Proven experience designing and implementing cloud data lake or lakehouse architectures at scale.
- Experience operating in a Microsoft Azure enterprise environment and working with cross-functional platform, security, governance, and engineering teams.
- Experience working in a regulated environment, preferably legal or financial services, with an understanding of security, privacy, audit, and compliance obligations.
Core Platform and Engineering Skills
- Advanced hands-on experience with Databricks, including Spark, PySpark, Delta Lake, SQL, Workflows, clusters, SQL warehouses, jobs, performance tuning, and cost optimization.
- Hands-on experience with Unity Catalog, including catalogs, schemas, permissions, lineage, secure data access, and governance administration.
- Strong experience with Microsoft Azure services, including Azure Data Lake Storage Gen2, Azure Data Factory, Azure Functions, Logic Apps, Key Vault, networking, identity, monitoring, and related services.
- Strong Python and SQL skills, with the ability to develop, review, troubleshoot, and optimize production-grade code and queries.
- Experience designing and supporting integration and ingestion patterns across databases, enterprise applications, files, APIs, and streaming or event-based sources.
- Experience with Collibra or a comparable data catalog and governance platform, including metadata, lineage, stewardship, ownership, and workflow integration.
- Experience with Immuta or a comparable data security platform, including policy-based access, sensitive data controls, auditability, and integration with Databricks.
- Knowledge of identity, access management, role-based access control, attribute-based access control, service principals, managed identities, secrets, and least-privilege design.
- Experience with platform monitoring, observability, capacity management, incident response, root-cause analysis, resilience, backup, and recovery.
- Experience with CI/CD, infrastructure as code, source control, automated testing, and release management using Terraform, Git, Azure DevOps, GitHub Actions, or equivalent tooling.
Desirable Skills
- Experience with dbt, Apache Airflow, Azure Event Hubs, Spark Structured Streaming, or comparable transformation and orchestration technologies.
- Experience with Azure Synapse Analytics, Microsoft Purview, containerization, Kubernetes, or enterprise API management.
- Knowledge of FinOps practices for data platforms, including attribution, budgeting, forecasting, chargeback or showback, and cost-control policies.
- Understanding of data quality, metadata management, lineage, retention, classification, privacy
Listed on hirly, a job board. hirly is not the employer: Freshfields is hiring for this role.
Similar jobs
- Sr Platform Engineer IIPacificlife · Newport Beach CA-700First seen 6d ago
- Senior Data Platform EngineerSpoton · Chicago, ILFirst seen 6d agoremote
- Sr Platform Engineer, DataEnsemblehp · Remote - NationwideFirst seen 6d agoremote
- Senior Platform Engineer, Embedded Linux & SecuritySwan · San FranciscoFirst seen 6d ago
- Senior Data Platform Engineer - Weights & Biases Coreweave · Livingston, NJ / New York, NY / San Francisco, CA / Sunnyvale, CA / Bellevue, WA / Remote - USFirst seen 6d agoremote
Browse similar roles
Want this one?
Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.
Tailor my resume for this job