Etchedai
Inference Software Engineer
Cupertino, CA
hirly Pro
Get in front of the people hiring at Etchedai
- Reach the hiring manager at Etchedai on LinkedIn, from your own account
- hirly applies to this job for you, with unlimited applications
- A resume and cover letter written for this job, unlimited
- The hiring manager and recruiter at Etchedai, with contact details
$49 a month · ₹1,999 for 30 days in India · cancel any time
Get past the screening software and onto a recruiter's desk
hirly rewrites your resume for this job — matching the keywords and skills in the posting, moving your most relevant experience to the top, and writing a cover letter to fit. About 30 seconds.
- Keywords matched to this posting
- Fit score before you apply
- Cover letter included
Matched against 2.2M live jobs from 220,000+ employers in 200+ countries.
Tailor my resume for this job →Apply from your AI assistant
Connect hirly to Claude or ChatGPT and ask it to apply to this job. hirly tailors your resume, fills the employer’s form and asks before sending.
Some employer sites stop an application at a CAPTCHA or sign-in and hand it back with a link. Applying needs a paid plan. Works with any assistant that supports MCP.
hirly's read of this role
- Role family
- Engineering
- Seniority
- Mid level
- Country
- US
- Work mode
- Remote-friendly
- First seen by hirly
- 4 Sept 2026
Derived automatically from the posting. Upload your resume above to see how the role scores against it.
the posting
About Etched
Etched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200. With Etched ASICs, you can build products that would be impossible with GPUs, like real-time video generation models and extremely deep & parallel chain-of-thought reasoning agents.
Key responsibilities
Contribute to the architecture and design of the Sohu host software stack
Implement high-performance, modular code across the complete Etched software stack, consisting of a mix of Rust, C++ and Python.
Interface with firmware and drivers teams delivering highest-performance HW/SW stack.
Work with AI model researchers and product-facing teams building out the Etched serving front-end.
Representative projects
Build scheduling logic for handling continuous batching and real time inference
Implement inference-time acceleration techniques such as speculative decoding, tree search, KV cache sharing, etc.
Implement distributed networking primitives for efficient multi-server inference
You may be a good fit if you have
Experience with C++ and Python
Familiarity with transformer model architectures and inference serving stacks (vLLM, SGLang, etc.) or experience working in distributed inference/training environments
Experience working cross-functionally in large software and hardware organizations
Strong candidates may also have
Experience with Rust
Familiarity with GPU kernels, the CUDA compilation stack and related tools, or other hardware accelerators
Understanding of distributed systems, networking, and parallel programming
Benefits
Full medical, dental, and vision packages, with 100% of premium covered
Housing subsidy of $2,000/month for those living within walking distance of the office
Daily lunch and dinner in our office
Relocation support for those moving to Cupertino
How we’re different
Etched believes in the Bitter Lesson . We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware. Larger and larger training runs encourage companies to consolidate around fewer model architectures, which creates a market for single-model ASICs.
We are a fully in-person team in Cupertino, and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.
Listed on hirly, a job board. hirly is not the employer: Etchedai is hiring for this role.
Similar jobs
- Intermediate Software EngineerGfs · Atlanta, GeorgiaFirst seen today
- Intermediate Software EngineerGfs · 2 LocationsFirst seen today
- Software Engineer II - EmbeddedGlobalhr · US-CA-SAN JOSE-826 ~ 200 Holger Way ~ BLDG 826First seen today
- Avionics Software EngineerCae · Arlington, TXFirst seen today
- Software Engineer, Agents for GoodBlackbaud · Remote - Anywhere - USAFirst seen todayremote
Browse similar roles
Want this one?
Upload your resume and hirly rewrites it for this job and writes the cover letter — in about thirty seconds, before you sign up.
Tailor my resume for this job