Lead AI/MLOPs Infrastructure Engineer - Senior Vice President

Key Responsibilities:

Platform & Infrastructure

Design and support scalable ML infrastructure for training, inference, and feature management.

Own the end-to-end AI/ML platform stack, including orchestration, compute, storage, and model serving.

Assess and implement cloud infrastructure for AI workloads, including GPU clusters and TPUs.

Define infrastructure-as-code standards for AI/ML environments, with Terraform as the primary tool.

MLOps & CI/CD for ML

Design, build, operate, and maintain MLOps pipelines across the full ML lifecycle, including training, validation, versioning, and deployment.

Partner with AI/ML engineers to productionize models and establish consistent deployment patterns.

Enable production AI/ML and Generative AI workloads, including LLM-based services.

Build and manage AWS-based, cloud-native infrastructure using Kubernetes and containerized workloads.

Monitoring & Reliability

Implement monitoring, drift detection, and alerting for model performance and system health.

Own SLAs and SLOs for model serving and inference endpoints.

Lead incident response for production AI/ML failures, including root cause analysis and continuous improvement.

Advance reliability engineering practices, including chaos testing and inference load testing.

Document standards, best practices, and reference architectures for MLOps and AI infrastructure.

Tooling & Developer Experience

Develop internal tools that improve productivity for ML engineers and data scientists.

Standardize development environments through containerization, reproducibility, and dependency management.

Evaluate and integrate third-party MLOps tools such as MLflow, Kubeflow, Weights & Biases, and Ray.

Security, Compliance & Cost

Apply data governance and access controls across ML systems.

Optimize cloud and compute spend for training and inference workloads.

Ensure model pipelines comply with applicable data privacy requirements.

Leadership & Cross-functional

Set the MLOps roadmap, standards, and best practices.

Mentor engineers on infrastructure and MLOps patterns.

Collaborate with ML engineers, data scientists, and platform teams.

Translate business needs into practical infrastructure architecture decisions.

Required Qualifications

15+ years of experience in DevOps, SRE, or Platform Engineering, with AWS as a primary cloud platform.

Experience supporting production machine learning systems, including deployment and monitoring.

Hands-on experience with MLOps platforms and tooling, including model registries, experiment tracking, and feature stores.

Exposure to production Generative AI and LLM workloads, including AWS Bedrock-based use cases.

Proven ability to build and operate AI/MLOps pipelines.

Strong proficiency with Terraform and scripting or programming in Python or similar languages.

Solid Linux, systems, and troubleshooting fundamentals.

Excellent communication skills and ability to collaborate across teams.

Preferred / Nice-to-Have

Hands-on experience with Kubernetes, containerized workloads, and cloud networking.

Experience with production Generative AI or LLM workloads.

Experience working in regulated or fintech environments.

Background optimizing costs for compute-intensive workloads.

Benefits

The base salary range for this role is $180,000 to $230,000 depending on experience. iCapital offers a compensation package which includes salary, equity for all full-time employees, and an annual performance bonus. Employees also receive a comprehensive benefits package that includes an employer matched retirement plan, generously subsidized healthcare with 100% employer paid dental, vision, telemedicine, and virtual mental health counseling, parental leave, and unlimited paid time off (PTO).

We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office Monday-Thursday, with the flexibility to work remotely on Friday.

For additional information on iCapital, please visit https://www.icapitalnetwork.com/about-us Twitter: @icapitalnetwork | LinkedIn: https://www.linkedin.com/company/icapital-network-inc | Awards Disclaimer: https://www.icapitalnetwork.com/about-us/recognition/

iCapital is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, gender, sexual orientation, gender identity, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics