New: upload your resume and get matched to AI jobs. Free to start →

Senior IA Inference Engineer (x/f/m)

Paris • Full-time

Posted 1mo ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

We are seeking a Senior Inference Engineer to join our ML Platform team. Your primary mission is to power Doctolib's AI-driven healthcare products by owning the self-hosted LLM serving platform. You will build and operate the shared foundations that AI feature teams rely on, deploying, tuning, and scaling open-weight models across our GPU fleet. This role directly contributes to faster, safer, and more cost-efficient AI for care teams and patients across Europe. You will work on innovative products and features to improve the daily lives of care teams and patients.

Responsibilities

  • Build the shared LLM serving foundations with MLOps engineers, defining the serving stack, GPU capacity planning, and deployment automation for open-weight models on Ray Serve/Anyscale, evolving towards Kubernetes-based orchestration.
  • Own the model registry and promotion path, establishing clear processes for versioning, validating, and promoting models into production.
  • Benchmark performance and output quality under realistic workloads, and maintain the curated model catalog.
  • Optimize inference performance and efficiency by tuning serving engines to reduce latency and cost per token through batching, caching, quantization, and parallelism strategies.
  • Own production reliability and disaster recovery, defining SLOs, running load tests, implementing safe rollouts/rollbacks, and participating in on-call and incident response.
  • Build observability tools (Datadog dashboards, alerting, cost reporting) to track performance and identify resource efficiency opportunities.
  • Ensure the inference stack is secure, compliant, and evolving, working with Security and Privacy teams and assessing new models and serving engines.

Requirements

  • 5+ years of software or computer engineering experience.
  • Hands-on production experience with at least one LLM serving engine (vLLM, SGLang, TensorRT-LLM, TGI, or Triton), ideally having chosen, introduced, tuned, or scaled one.
  • Proficiency in Python and Terraform.
  • Solid understanding of transformer inference mechanics (prefill vs. decode, KV cache, batching strategies, compute vs. memory bound).
  • Strong experience running GPU workloads on Kubernetes or a managed Ray/Anyscale platform.
  • Performance engineering mindset with experience in designing reproducible benchmarks and analyzing profiler output.
  • Comfortable owning production reliability, including observability, incident response, and on-call.
  • Excellent communication and collaboration skills, high agency, and a pragmatic build-vs-buy approach.
  • Experience with distributed serving frameworks (KServe, llm-d, NVIDIA Dynamo) is a plus.
  • GPU kernel experience or deep profiling skills (CUDA, Triton, Nsight) is a plus.
  • Experience in regulated environments (healthcare, finance) and with frameworks like GDPR, HDS, ISO 27001, or SOC 2 is a plus.

Benefits

  • Free comprehensive health insurance (basic package) for you and your children
  • 25 days of paid vacation per year, plus up to 14 days of RTT
  • Free mental health and coaching services
  • Work from abroad for up to 10 days per year