Director, Site Reliability Engineer

Job Description:

AssetMark is a leading strategic provider of innovative investment and consulting solutions serving independent financial advisors. We provide investment, relationship, and practice management solutions that advisors use in helping clients achieve wealth, independence, and purpose.

The Opportunity

We are seeking a strategic and technically strong Director of Service/Site Reliability Engineering and Observability to lead the reliability, monitoring, performance, and operational excellence of an enterprise environment hosted exclusively on Microsoft Azure.

This leader will own the strategy and operating model for SRE and observability across both Azure Infrastructure-as-a-Service (IaaS) and native Azure managed services. The scope includes real-time applications, APIs, virtual machines, containers, serverless workloads, integrations, databases, messaging platforms, data pipelines, and batch-processing systems.

The role requires a combination of enterprise leadership, Azure architecture knowledge, application-development experience, production troubleshooting expertise, and the ability to build scalable practices that improve availability, performance, resilience, security, and customer experience.

The Director will partner closely with application development, cloud infrastructure, security, architecture, data, service management, and business teams to ensure Azure-hosted systems are observable, supportable, secure, resilient, and ready for production.

At AssetMark, reliability is a first-order expression of client obsession. Our SRE team plays a critical role in delivering the consistent, trusted technology experience that advisors depend on to run their businesses.

We can only consider candidates for this position who are able to accommodate a hybrid work schedule and are close to our Charlotte, NC office.

Key Responsibilities

SRE and Observability Strategy

• •Define and execute an enterprise-wide SRE and observability strategy aligned with business priorities, technology objectives, and risk-management requirements.
• •Establish standards for metrics, logs, traces, events, dashboards, alerting, instrumentation, service ownership, telemetry quality, and data retention.
• •Develop a multi-year roadmap for Azure observability platforms, reliability engineering, automation, resilience, and operational maturity.
• •Define service-level indicators, service-level objectives, service-level agreements, error budgets, availability targets, latency objectives, and performance standards.
• •Establish operational-readiness requirements for new applications, major releases, infrastructure changes, Azure service implementations, and third-party integrations.
• •Evaluate and govern observability platforms, tools, vendors, integrations, licensing, access, security, and operating costs.
• •Promote consistent reliability and observability practices across Azure subscriptions, resource groups, environments, and service teams.

Azure Platform Reliability and Observability

• •Lead reliability and observability practices across Azure IaaS and native Azure services, including virtual machines, scale sets, containers, serverless workloads, managed databases, storage, messaging, integration services, and data platforms.
• •Establish Azure-native monitoring and telemetry standards using capabilities such as Azure Monitor, Log Analytics, Application Insights, Azure Managed Grafana, Activity Logs, Resource Health, and Azure Service Health.
• •Ensure end-to-end visibility across application, infrastructure, platform, network, identity, security, data, and third-party dependencies.
• •Define standards for Azure diagnostic settings, centralized logging, metric collection, distributed tracing, correlation IDs, alert routing, dashboard design, and telemetry retention.
• •Partner with cloud platform and architecture teams to improve the reliability of Azure subscriptions, management groups, networking, identity, access controls, policies, and shared services.
• •Ensure Azure-hosted services are designed with appropriate availability, scalability, recoverability, monitoring, and operational support requirements.

Real-Time Monitoring and Operational Excellence

• •Lead the design and operation of real-time monitoring for business-critical applications, customer-facing services, APIs, Azure infrastructure, databases, networks, messaging systems, and integrations.
• •Improve the quality, relevance, and actionability of alerts while reducing alert fatigue and unnecessary escalations.
• •Create executive, operational, and engineering dashboards that clearly communicate system health, business impact, trends, and emerging risks.
• •Establish effective on-call, escalation, incident response, major incident management, and service-restoration practices.
• •Participate in root-cause analysis, problem management, post-incident reviews, and corrective-action tracking.
• •Drive proactive detection of reliability, capacity, performance, security, and availability risks.
• •Develop runbooks, playbooks, automated remediation, self-healing capabilities, and other mechanisms that reduce manual intervention and accelerate recovery.
• •Use Azure telemetry and operational data to identify trends, dependencies, recurring failures, and opportunities for continuous improvement.

Application Development and Troubleshooting

• •Partner with application-development teams to improve instrumentation, diagnosability, performance, resilience, and production supportability.
• •Provide technical leadership during complex production issues involving application code, APIs, microservices, middleware, databases, queues, files, integrations, Azure services, and infrastructure.
• •Apply a strong understanding of software-development practices to review application behavior, identify failure patterns, and support code-level troubleshooting.
• •Promote observability as part of the software-development lifecycle, including design, development, testing, deployment, and post-release validation.
• •Support performance engineering, profiling, capacity planning, release readiness, and production validation.
• •Troubleshoot issues across Azure compute, networking, identity, storage, messaging, databases, application runtimes, and service dependencies.
• •Use telemetry and operational data to distinguish application defects, infrastructure issues, data problems, configuration errors, and dependency failures.

Batch Systems and Data Processing

• •Ensure reliable operation and monitoring of scheduled jobs, batch processes, data pipelines, file transfers, reporting processes, and other time-sensitive workloads hosted on Azure.
• •Establish visibility into batch dependencies, job duration, success and failure rates, late starts, missed schedules, data-quality issues, retries, restarts, and downstream impacts.
• •Define recovery, restart, reconciliation, and escalation procedures for failed or delayed batch processes.
• •Monitor and improve Azure-based batch and data services, including scheduled workloads, serverless processing, messaging, integration services, and data-platform components.
• •Partner with application, data, and business teams to protect batch-processing service levels and ensure timely delivery of critical data and outputs.

Reliability, Resilience, and Risk Management

• •Improve system availability, scalability, recoverability, and fault tolerance across Azure-hosted enterprise platforms.
• •Lead capacity planning, performance monitoring, resilience testing, disaster-recovery readiness, and business-continuity improvements.
• •Evaluate resiliency across Azure regions, availability zones, virtual-machine workloads, managed services, and application dependencies.
• •Identify single points of failure, operational dependencies, unsupported services, configuration risks, and other sources of reliability risk.
• •Partner with security and compliance teams to ensure monitoring, logging, access, retention, and incident practice