Senior IA Inference Engineer (x/f/m)
Paris • Full-time
Posted 1mo ago
Job Location
Paris
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
We are seeking a Senior Inference Engineer to join our ML Platform team. Your primary mission is to power Doctolib's AI-driven healthcare products by owning the self-hosted LLM serving platform. You will build and operate the shared foundations that AI feature teams rely on, deploying, tuning, and scaling open-weight models across our GPU fleet. This role directly contributes to faster, safer, and more cost-efficient AI for care teams and patients across Europe. You will work on innovative products and features to improve the daily lives of care teams and patients.
Responsibilities
- Build the shared LLM serving foundations with MLOps engineers, defining the serving stack, GPU capacity planning, and deployment automation for open-weight models on Ray Serve/Anyscale, evolving towards Kubernetes-based orchestration.
- Own the model registry and promotion path, establishing clear processes for versioning, validating, and promoting models into production.
- Benchmark performance and output quality under realistic workloads, and maintain the curated model catalog.
- Optimize inference performance and efficiency by tuning serving engines to reduce latency and cost per token through batching, caching, quantization, and parallelism strategies.
- Own production reliability and disaster recovery, defining SLOs, running load tests, implementing safe rollouts/rollbacks, and participating in on-call and incident response.
- Build observability tools (Datadog dashboards, alerting, cost reporting) to track performance and identify resource efficiency opportunities.
- Ensure the inference stack is secure, compliant, and evolving, working with Security and Privacy teams and assessing new models and serving engines.
Requirements
- 5+ years of software or computer engineering experience.
- Hands-on production experience with at least one LLM serving engine (vLLM, SGLang, TensorRT-LLM, TGI, or Triton), ideally having chosen, introduced, tuned, or scaled one.
- Proficiency in Python and Terraform.
- Solid understanding of transformer inference mechanics (prefill vs. decode, KV cache, batching strategies, compute vs. memory bound).
- Strong experience running GPU workloads on Kubernetes or a managed Ray/Anyscale platform.
- Performance engineering mindset with experience in designing reproducible benchmarks and analyzing profiler output.
- Comfortable owning production reliability, including observability, incident response, and on-call.
- Excellent communication and collaboration skills, high agency, and a pragmatic build-vs-buy approach.
- Experience with distributed serving frameworks (KServe, llm-d, NVIDIA Dynamo) is a plus.
- GPU kernel experience or deep profiling skills (CUDA, Triton, Nsight) is a plus.
- Experience in regulated environments (healthcare, finance) and with frameworks like GDPR, HDS, ISO 27001, or SOC 2 is a plus.
Benefits
- Free comprehensive health insurance (basic package) for you and your children
- 25 days of paid vacation per year, plus up to 14 days of RTT
- Free mental health and coaching services
- Work from abroad for up to 10 days per year
