Databricks Solution Architect

Description

We are seeking a Lead Data Engineer, Master Data Management to help shape the future of data at Clearstead. This role offers the opportunity to join our Data & AI team at a foundational stage, helping build the data pipelines, integrations, governance practices, and master data capabilities that bring our enterprise data together.

You’ll work at the intersection of data engineering, master data management, governance, and AI, helping ensure that critical business data is accurate, consistent, secure, and accessible. This is an ideal opportunity for someone who enjoys being hands-on with data while also helping establish the standards and practices that support a strong, scalable enterprise data environment.

Key Responsibilities

Data Architecture & Platform Management

Lead the architecture and development of the Azure Databricks platform, including Delta Lake, Unity Catalog, and Medallion Architecture.

Establish technical standards, security controls, and scalable data integration frameworks.

Oversee platform performance, monitoring, operational readiness, and cost optimization.

Manage the transition of platform ownership from external implementation partners.

Data Engineering & Integration

Design, build, and maintain production data pipelines using PySpark, SQL, and Databricks Workflows.

Integrate data from Salesforce, Orion, Addepar, PARis, and other enterprise systems.

Develop data transformation, reconciliation, and quality assurance processes.

Implement automated testing, deployment, and infrastructure management practices.

AI Enablement & Data Innovation

Develop data infrastructure that supports AI applications, intelligent agents, and advanced analytics.

Support document ingestion, vector search, model serving, and retrieval-augmented generation.

Collaborate with the AI Engineer and Data Management Lead to deliver secure, scalable solutions.

Technical Leadership & Collaboration

Serve as the technical design authority for the enterprise data platform.

Evaluate vendor solutions and oversee implementation partner deliverables.

Partner with business and technology leaders to translate requirements into effective solutions.

Mentor developing engineers and promote engineering best practices.

Requirements

Bachelor's degree in Data Management, Information Systems, Computer Science, Business Analytics, or a related field, or an equivalent combination of education and relevant professional experience.

10+ years of experience in data engineering, data architecture, or a related discipline, including 4+ years designing and managing production Databricks environments.

Expert knowledge of Azure Databricks, Unity Catalog, Delta Lake, and modern data architecture principles.

Strong hands-on experience with PySpark, SQL, data pipelines, and enterprise data integration.

Experience supporting AI or machine learning workloads within Databricks.

Knowledge of Azure security, identity management, Terraform, and CI/CD practices.

Demonstrated experience managing enterprise data platforms and external technology partners.

Strong communication skills and the ability to translate technical concepts for business stakeholders.

Preferred Qualifications

Advanced degree in Information Systems, Data Analytics, or a related discipline.

Databricks Data Engineer Associate or Professional, Certified Data Management Professional (CDMP), or comparable industry certification.

Experience in financial services, wealth management, or another regulated data environment.

Familiarity with Salesforce Financial Services Cloud, Orion, Addepar, Power BI, Databricks AI/BI Genie, or Delta Sharing.

Core Competencies

Operational excellence and attention to detail

Systems and process design mindset

Strong judgment and discretion

Ability to manage competing priorities

Clear, professional communication

Enterprise data architecture and platform ownership

Security, reliability, and operational discipline

Cross-functional partnership and vendor leadership

Hands-on engineering and technical problem-solving

Strategic judgment and solution design

Clear, business-focused communication

Work Environment

This is a remote position that requires prolonged computer use and effective collaboration across teams and locations. Limited travel may be required for training, team meetings, or other business-related activities. Occasional flexibility in work hours may be needed to support system implementations, production incidents, or critical business operations.

General Statement

This job description summarizes the primary duties and requirements of the position. It is not intended to be an exhaustive list of every responsibility or task. Responsibilities may be adjusted based on business needs, consistent with applicable law. Nothing in this job description creates a contract of employment or changes the at-will employment relationship where applicable.

About Clearstead

Clearstead is a financial advisory firm serving wealthy families and leading institutions. Our mission is to enable wealthy families and leading institutions to meet their financial objectives and achieve their aspirations, building stronger futures for their families, their communities, and themselves.

We fulfill our mission by creating a unified investment framework for our clients and integrating financial planning, tax planning and compliance, trust administration services with investment management to address complex financial situations.