hirly

DigitalOcean

Principal Engineer, Inference Memory and Storage Systems

Seattle

hirly Pro

Get in front of the people hiring at DigitalOcean

  • Reach the hiring manager at DigitalOcean on LinkedIn, from your own account
  • hirly applies to this job for you, with unlimited applications
  • A resume and cover letter written for this job, unlimited
  • The hiring manager and recruiter at DigitalOcean, with contact details
Get Pro for this job →

$49 a month · ₹1,999 for 30 days in India · cancel any time

See how you match this job — and similar ones. Free.

Upload your resume and hirly scores it against this role at DigitalOcean first, then against similar open jobs, and shows where you fit and why.

PDF or DOCX, up to 12MB. No sign-up to see your matches.

Get past the screening software and onto a recruiter's desk

hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.

  • Keywords matched to this posting
  • Fit score before you apply
  • Cover letter included

Matched against 2.8M live jobs from 220,000+ employers in 200+ countries.

Tailor my resume for this job →

Apply from your AI assistant

Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.

Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.

hirly's read of this role

Role family
Engineering
Seniority
Lead / management
Stated salary
$249,600 – $312,000 per year
Country
US
Work mode
Remote-friendly
First seen by hirly
9 Oct 2026

Derived automatically from the posting. Upload your resume above to see how the role scores against it.

the posting

Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world.

We are looking for a Principal Engineer to own the technical vision and roadmap for the memory and storage layer underneath DigitalOcean's inference platform.

DigitalOcean is the Inference Cloud. The Inference Platform team builds the serving stack that runs frontier open models on our GPU fleet at production scale—model orchestration, disaggregated prefill and decode, request routing, and the engine integrations (vLLM, SGLang, TensorRT-LLM) that make it all go. As context lengths grow and agentic traffic patterns become the norm, the single biggest lever on cost, TTFT, and throughput is where the KV cache lives and how efficiently we can move it.

That layer is currently a collection of good decisions made independently. This role exists to make it one system: a unified, multi-tier memory and storage substrate spanning GPU HBM, host DRAM, local NVMe, and remote object storage, exposed through a coherent interface to every serving engine and router we run. The selected candidate will set direction across teams, contribute meaningfully to the open source projects we depend on, and translate hard systems work into unit economics our customers feel.

What You'll Be Doing:

Defining and evolving a unified memory layer spanning GPU memory, pinned host memory, RDMA-accessible memory, local NVMe tiers, and remote object storage for large-scale LLM inference

Architecting deep integrations with the serving engines we run in production (vLLM, SGLang, TensorRT-LLM), focused on KV cache offload, reuse, eviction policy, and cross-node sharing

Designing the interfaces and protocols behind disaggregated prefill/decode, peer-to-peer KV cache transfer, and cache-aware routing—including how the router, the engine, and the cache tier agree on what is resident and where

Owning the eviction and admission story across tiers (LRU and its successors, cost-aware policies, pluggable backends) and the metrics that prove those policies work under real traffic

Partnering with our GPU infrastructure, networking, and platform teams to exploit GPUDirect, RDMA, NVMe-oF, and NVLink for low-latency cache access across heterogeneous accelerator pools

Driving the unit economics: modeling and validating the effect of cache hit ratio, prefix caching, and tiering on $/token and $/GPU-hr, and making the tradeoffs legible to product and pricing partners

Setting technical direction and raising the bar through design review, mentorship of senior and staff engineers, and sponsorship of the initiatives that follow from this roadmap

Representing DigitalOcean externally—upstream contributions, conference talks, and customer-facing technical deep dives

What You'll Add to DigitalOcean:

15+ years building large-scale distributed systems, high-performance storage, or ML systems infrastructure, with a track record of delivering and operating production services

Deep fluency in memory hierarchies (GPU HBM, host DRAM, NVMe, remote/object storage) and experience designing systems that span tiers for both performance and cost

Experience with distributed caching or key-value systems, particularly designs optimized for low latency under high concurrency

Hands-on work with networked I/O and RDMA / NVMe-oF / NVLink-class technologies, and familiarity with aggregated and disaggregated deployment topologies for AI clusters

Strong systems programming skills in C/C++, Go, Rust, or Python, and comfort reading and modifying serving-engine internals

Rigor in profiling and optimization across CPU, GPU, memory, and network, using measurement to drive architectural decisions and to validate improvements in TTFT, ITL, and throughput

Excellent written and verbal communication, and a history of leading cross-functional efforts with product, infrastructure, and customer-facing teams

Bonus:

Contributions to open source LLM serving or inference infrastructure projects—vLLM, SGLang, llm-d, NVIDIA Dynamo, LMCache, or similar—especially on KV cache offload, compression, or reuse

Experience designing a unified memory or storage layer that presents a single logical KV or object model across GPU, host, SSD, and cloud tiers in a hyperscale or public cloud environment

Familiarity with Kubernetes-based GPU orchestration, including DRA, MIG/MPS partitioning, and gateway/inference-extension routing patterns

Publications or patents in LLM systems, memory-disaggregated architectures, RDMA-based data planes, or CDN-like caching systems for ML workloads

Compensation Range:

$249,600 - $312,000

Non-sales roles : In addition to base salary, this role is eligible to participate in DigitalOcean’s annual bonus plan and may be eligible for a new hire equity grant, subject to plan terms and local eligibility. Bonus amounts depend on company and individual performance.

Sales roles (sales compensation eligible) : In addition to base salary, this role may be eligible to earn incentive compensation under the applicable sales compensation plan and may be eligible for a new hire equity grant, subject to plan terms and local eligibility.

*This is a hybrid role

JR: 2026-8160

#LI-Hybrid

Why You’ll Like Working for DigitalOcean

We innovate with purpose. You’ll be a part of a cutting-edge technology company with a high-growth trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As part of our team, you’ll embody our "Shark" mindset: bold, ambitious, and scrappy. You'll lead with a bias for action and take true ownership of our products, customer success, and team culture.

We prioritize career development. At DigitalOcean, you’ll do the best work of your career. You will join a community of exceptional talent, collaborating closely with some of the brightest and most interesting people in the industry. We are a high-performance and fast-paced organization that will always challenge you to think big. Our talent development team prioritizes your professional development and provides resources to support your growth. We provide employees with reimbursement for relevant conferences, training, and education, subject to company policy.

We care about your well-being. We take a comprehensive approach to our total rewards packages because we know different things drive different people. Our benefits are designed to support you and may include flexible time off, local team meetups, and mental health and well being support. Our benefits philosophy is consistent globally, but specific benefits and eligibility vary by location to reflect local needs and regulations.

We reward our employees. The salary range for this position is based on market data, relevant years of experience, and skills. Eligible employees may also receive equity compensation, which may include a new hire equity grant and, in participating countries, the opportunity to participate in our Employee Stock Purchase Program. Equity awards are subject to approval and the terms of the applicable plan.

Non-sales roles : In addition to base salary, this role is eligible to participate in DigitalOcean’s annual bonus plan and may be eligible for a new hire equity grant, subject to plan terms and local eligibility. Bonus amounts depend on company and individual performance.

Sales roles (sales compensation eligible) : In addition to base salary, this role may be eligible to earn incentive compensation under the applicable sales compensation plan and may be eligible for a n

Original posting on DigitalOcean's site ↗

Listed on hirly, a job board. hirly is not the employer: DigitalOcean is hiring for this role.

Browse similar roles

Want this one?

Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.

Tailor my resume for this job