Research Engineer, Latency & Throughput
Paris • Full-time
Posted 1mo ago
About the job
Mistral is seeking a Research Engineer to join the Latency & Throughput team, focusing on advancing LLM inference. This role involves researching and prototyping techniques to reduce latency and increase throughput, drawing from recent literature and implementing proof-of-concepts. You will bridge the gap between academic research and production impact, working on areas like speculative decoding, quantization, and new attention mechanisms. The position requires a strong ML background, hands-on experience with Python and PyTorch, and a proven track record of conducting experiments and drawing conclusions. You will collaborate with inference engineers and research scientists to integrate validated techniques into Mistral's serving stack.
Responsibilities
- Research and prototype LLM inference techniques such as speculative decoding, quantization, KV compression, watermarking, and new model architectures.
- Design and execute experiments to validate hypotheses, iterating quickly on small-scale setups.
- Demonstrate clear latency or throughput gains without unacceptable quality loss.
- Collaborate with the Inference Foundation team to integrate validated techniques into the vLLM-based serving stack.
- Partner with pretraining and RL teams when techniques require co-training or retraining models.
- Build tooling and validation platforms to facilitate AI-driven inference research.
Requirements
- Strong ML background with solid knowledge of LLM training techniques (architecture, tokenization, fine-tuning, evaluation).
- Hands-on experience with Python and PyTorch; JAX is acceptable with strong Python proficiency.
- Research experience (thesis, publications, dedicated research project, or efficiency-oriented work).
- Track record of conducting experiments: forming hypotheses, designing controlled comparisons, and drawing clear conclusions.
- Familiarity with the state of the art in LLM inference.
- Self-starter, low-ego, and collaborative, comfortable with ambiguity and negative results.
Benefits
- Healthcare coverage
- Parental leave
- Retirement plans
- Relocation support
- Wellness programs
- Meal and transportation allowances
