Senior Principal Model Bring-up Engineer
$208k - $312k • Remote • Austin, Texas, United States; San Jose, California, United States
Posted 1d ago
Job Location
Austin, Texas, United States; San Jose, California, United States
Tech Stack
Remote Work Policy
Fully remote
Categories
LLM Engineer
About the job
SambaNova is a leader in next-generation AI infrastructure, delivering a full-stack inference platform for customers worldwide. At the core of SambaNova's technology is the RDU (Reconfigurable Dataflow Unit) — a chip built on a dataflow architecture rather than the traditional GPU model. This role involves bringing frontier open-weight models from Hugging Face to production serving on SambaNova's RDU dataflow accelerators. This includes adapting models, mapping them across chips and hosts, compiling them, and validating them numerically against the reference. You will also build inference features like continuous batching, speculative decoding, and FP8/FP4 as shared modules to reduce the cost of bringing up new models.
Responsibilities
- Bring up new LLM architectures end-to-end, including model adaptation, checkpoint conversion, multi-chip/multi-host mapping, compilation, numerical validation, and production serving.
- Develop shared inference features such as continuous batching, prompt caching, chunked/incremental prefill, dynamic tensor shapes, speculative decoding, FP8/FP4, and tensor/expert parallelism.
- Debug issues at the intersection of models, compilers, and hardware, ensuring fixes are integrated into the correct shared components.
Requirements
- B.S. in Computer Science, Electrical Engineering, or related field.
- 8+ years of industry experience in ML systems, inference, or compiler engineering.
- Strong Python and PyTorch internals experience, with some C++ proficiency.
- Deep understanding of transformer inference, including KV caching, GQA, sliding window attention, MoE routing, prefill vs decode, batching, and speculative decoding.
- Hands-on experience with accelerator bring-up, ML compilers (MLIR, XLA, TVM, Triton, torch-mlir), or inference engines (vLLM, SGLang, TensorRT-LLM).
- Proficiency in numerical debugging.
- Familiarity with Bazel, Slurm, and large monorepos.
- Excellent writing and communication skills for presenting evidence-based claims.
Benefits
- Equity
- 95% premium coverage for employee medical insurance
- 77% premium coverage for dependents
- Health Savings Account (HSA)