Principal Cloud Platform Engineer

SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide. At the core of SambaNova's technology is the RDU (Reconfigurable Dataflow Unit) — a chip built on a dataflow architecture rather than the traditional GPU model. Its decode performance is especially strong for agentic workloads like multi-turn agents, code generation, and long-running applications. RDUs are packaged into SambaRack, rack-scale hardware that lets customers deploy state-of-the-art models with better performance, greater energy efficiency, and faster time to value.

About the Team

The Cloud Platform team runs SambaNova’s production inference service. We handle capacity planning, deployment, monitoring, and incident response across regions in the United States, Asia, Europe, and Latin America.

About the Role

As a Principal Cloud Platform Engineer, you will design and build the software that deploys, schedules, and scales model workloads across our shared accelerator hardware, staying hands-on in the code while leading the technical design. You will set uptime and latency targets for our inference endpoints and share in the on-call rotation that keeps them running.

Responsibilities

Shared ownership of the production inference service across regions, covering availability, latency, performance, change management, and capacity planning

Designing and building platform services, Kubernetes operators, and controllers that deploy, schedule, and scale model workloads

Designing scheduling for shared accelerator capacity, including quotas, priority, and fairness between customers

Rolling out upgrades without disrupting running workloads, and defining how the service recovers when a node or worker fails

Designing platform security, including tenant isolation, workload identity, and secrets management

Standing up new regions with infrastructure as code in Terraform and Ansible, and building CI/CD pipelines for model and service deployments

Building monitoring and alerting in Prometheus, Grafana, and Datadog, and defining SLOs and SLIs that drive reliability work

Finding and eliminating performance bottlenecks, and designing auto-scaling for variable inference loads

Leading incident response and postmortems as part of a shared primary/secondary on-call rotation

Forecasting infrastructure needs and working with finance to manage cloud spend

Required Qualifications

B.S. in Computer Science, Computer Engineering, or related field

8+ years of experience in Site Reliability Engineering, platform engineering, or distributed systems roles

Experience supporting a large-scale, customer-facing service in a public cloud environment (AWS, GCP, Azure)

Strong programming skills in Python, Go, Rust, or Java, with recent hands-on coding

Experience designing and building a platform service, control plane, or scheduler used by multiple teams or customers

Kubernetes experience such as writing operators or controllers, running cluster upgrades, or configuring the scheduler

Experience with multi-tenant systems, including resource quotas, isolation, and fair sharing of capacity

Experience designing for failure, including retries, idempotency, and safe rollouts

Deep understanding of monitoring and observability principles and tools (e.g., Prometheus, Grafana, ELK Stack, Datadog)

Experience with Infrastructure as Code (e.g., Terraform, CloudFormation)

Experience with CI/CD principles and tools (e.g., Jenkins, GitHub Actions, ArgoCD)

Strong Linux/Unix system administration fundamentals

Preferred Qualifications

Experience in a hybrid environment bridging cloud and on-premise/data center infrastructure

Direct experience supporting ML/AI inference services in production

Knowledge of model serving frameworks like vLLM, SGLang, or Ray, including batching and KV cache management

Familiarity with GPU-accelerated computing and optimizing workloads for NVIDIA GPUs for purposes of mapping to RDUs

Experience with batch or job scheduling systems such as Slurm, Volcano, or Kueue

Experience with Kubernetes internals such as etcd, control plane high availability, or the scheduler

Understanding of MLOps principles and practices

Experience managing and tuning databases (SQL or NoSQL) and caching systems (Redis, Memcached)

Base Salary Range:

Base Pay Range

$210,000 — $280,000 USD

• Submission Guidelines
• Please note that in order to be considered an applicant for any position at SambaNova Systems, you must submit an application form for each position for which you believe you are qualified.

• EEO Policy
• SambaNova Systems is an Equal Opportunity/Affirmative Action Employer. All qualified applicants will receive consideration for employment without regard basis of age (40 and over), color, disability, gender identity, genetic information, marital status, military or veteran status, national origin/ancestry, race, religion, creed, sex (including pregnancy, childbirth, breastfeeding), sexual orientation, and any other applicable status protected by federal, state, or local laws.

• Benefits Summary for US-Based, Full-Time Employment Positions
• SambaNova offers a competitive total rewards package, including the base salary, plus equity and benefits. We cover 95% premium coverage for employee medical insurance, and 77% premium coverage for dependents and offer a Health Savings Account (HSA) with employer contribution. We also offer Dental, Vision, Short/Long term Disability, Basic Life, Voluntary Life, and AD&D insurance plans in addition to Flexible Spending Account (FSA) options like Health Care, Limited Purpose, and Dependent Care. Our library of well-being benefits available to you and your dependents includes a full subscription to Headspace, Gympass+ membership with access to physical gyms, One Medical membership, counseling services with an Employee Assistance Program, and much more.