NVIDIA
Principal Architect, AI Inference Networking - NIXL and Dynamo
China, Shanghai · China, Shenzhen
hirly Pro
Get in front of the people hiring at NVIDIA
- Reach the hiring manager at NVIDIA on LinkedIn, from your own account
- hirly applies to this job for you, with unlimited applications
- A resume and cover letter written for this job, unlimited
- The hiring manager and recruiter at NVIDIA, with contact details
$49 a month · ₹1,999 for 30 days in India · cancel any time
Get past the screening software and onto a recruiter's desk
hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.
- Keywords matched to this posting
- Fit score before you apply
- Cover letter included
Matched against 2.8M live jobs from 220,000+ employers in 200+ countries.
Tailor my resume for this job →Apply from your AI assistant
Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.
Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.
hirly's read of this role
- Seniority
- Lead / management
- Country
- CN
- Work mode
- On-site / unstated
- First seen by hirly
- 23 Sept 2026
Derived automatically from the posting. Upload your resume above to see how the role scores against it.
the posting
Inference has become a networking problem. With disaggregated serving, where prefill runs on one set of GPUs, decode on another, and KV cache streams between them, performance depends on how fast you can move tensors between GPUs, across nodes, and in and out of storage. Performance is only half of it. These deployments are elastic, so the data path needs dynamic APIs that let workers join and leave, connections form and tear down, and transfers reroute at runtime without stalling the pipeline. NIXL, NVIDIA's open source Inference Xfer Library, is the layer that solves this, and it sits underneath NVIDIA Dynamo and a growing list of inference engines.
China is where some of the world's most demanding inference deployments are being built. We are hiring a senior architect to work directly with the platform teams building them, understanding what breaks at scale and turning that into upstream code and roadmap. You will work daily with core NIXL and Dynamo engineers in Israel and the US.
What you'll be doing:
- Drive NIXL adoption in China, from the first architecture conversation through prototype and benchmark to production rollout and upstream contribution.
- Profile customer inference stacks and fix what's costing throughput and TTFT: KV cache movement, memory registration, transfer scheduling.
- Write NIXL backends and plugins, extend Dynamo integrations, and contribute optimizations upstream to NIXL and GPUDirect class data paths.
- Help define the APIs that make the data path dynamic, supporting elastic scaling, worker churn and runtime rerouting in production inference clusters.
- Bring the region's requirements into NVIDIA's roadmap with enough technical authority to change it.
- Grow the local community around NIXL through talks, benchmarks and reference architectures.
What we need to see:
- M.Sc. or Ph.D. in CS/EE/CE, or equivalent depth earned in industry.
- 12+ years building or optimizing large scale distributed systems: AI inference or training infrastructure, communication libraries, high performance networking, or HPC runtimes.
- Strong systems programming in C++ and Python, with performance critical code shipped.
- Hands on experience with RDMA networking (InfiniBand or RoCE) and GPU memory movement.
- Working knowledge of modern LLM serving, including prefill/decode disaggregation, KV cache management, tensor and pipeline parallelism, and at least one serving stack (Dynamo, TensorRT-LLM, vLLM, SGLang, Triton).
- Strong communication skills, with the technical credibility to be taken seriously by customer engineering teams and by NVIDIA's core library maintainers.
- Fluent Mandarin and professional English, and comfortable working across China, Israel and US time zones.
Ways to stand out from the crowd:
- Open source contributions to NIXL, Dynamo, NCCL, vLLM, SGLang, TensorRT-LLM or Mooncake. We will read your commits.
- Inference at genuine scale: thousands of GPUs, multi tenant clusters, strict SLOs.
- CUDA, and comfort reading kernel and driver level code when the answer is down there.
- A community you've built: a project people adopted, a talk people still cite.
You will help shape how NIXL is adopted across China, working alongside the engineers who design the transport layer itself. The work is public, and disaggregated inference is being defined right now.
NVIDIA is committed to fostering a diverse work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Listed on hirly, a job board. hirly is not the employer: NVIDIA is hiring for this role.
Similar jobs
- Principal Architect - CybersecurityLogicalis · 2 LocationsFirst seen 6d ago
- Principal Architect, Source to PayWorkday · 2 LocationsFirst seen 6d ago
- Architect, Technology Solutions and Digital ProductsAPLL · Shanghai Shi, ChinaFirst seen 33d ago
- Senior Architect, Image QualityGehc · BeiJingFirst seen 51d ago
- Architect, AIAia · Guangzhou, CN-AIA Shared Services ChinaFirst seen 55d ago
Browse similar roles
Want this one?
Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.
Tailor my resume for this job