New: upload your resume and get matched to AI jobs. Free to start →

Research Engineer, Latency & Throughput

Paris • Full-time

Posted 1mo ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

Mistral is seeking a Research Engineer to join the Latency & Throughput team, focusing on advancing LLM inference. This role involves researching and prototyping techniques to reduce latency and increase throughput, drawing from recent literature and implementing proof-of-concepts. You will bridge the gap between academic research and production impact, working on areas like speculative decoding, quantization, and new attention mechanisms. The position requires a strong ML background, hands-on experience with Python and PyTorch, and a proven track record of conducting experiments and drawing conclusions. You will collaborate with inference engineers and research scientists to integrate validated techniques into Mistral's serving stack.

Responsibilities

  • Research and prototype LLM inference techniques such as speculative decoding, quantization, KV compression, watermarking, and new model architectures.
  • Design and execute experiments to validate hypotheses, iterating quickly on small-scale setups.
  • Demonstrate clear latency or throughput gains without unacceptable quality loss.
  • Collaborate with the Inference Foundation team to integrate validated techniques into the vLLM-based serving stack.
  • Partner with pretraining and RL teams when techniques require co-training or retraining models.
  • Build tooling and validation platforms to facilitate AI-driven inference research.

Requirements

  • Strong ML background with solid knowledge of LLM training techniques (architecture, tokenization, fine-tuning, evaluation).
  • Hands-on experience with Python and PyTorch; JAX is acceptable with strong Python proficiency.
  • Research experience (thesis, publications, dedicated research project, or efficiency-oriented work).
  • Track record of conducting experiments: forming hypotheses, designing controlled comparisons, and drawing clear conclusions.
  • Familiarity with the state of the art in LLM inference.
  • Self-starter, low-ego, and collaborative, comfortable with ambiguity and negative results.

Benefits

  • Healthcare coverage
  • Parental leave
  • Retirement plans
  • Relocation support
  • Wellness programs
  • Meal and transportation allowances

About Mistral AI