Staff Software Engineer, Inference / Compute Infrastructure Engineering London or Amsterdam
Salary not listed • London & Amsterdam • Full-time
Posted 2mo ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
Job Location
London & Amsterdam
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware pool. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple platform users from operational complexities while increasing usable capacity from existing hardware. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless decoupling, alongside event-driven systems for health monitoring, remediation, and utilization management without manual intervention. A product mindset is crucial, with experience building internal platforms or APIs for other engineering teams and a focus on developer experience and ownership of delivered software, including production support.
Responsibilities
- Design and implement the provisioning state machine to model the full lifecycle of a physical host, from discovery and inference bring-up to GPU driver/CUDA stack, health validation, and decommission, as explicit, versioned states and transitions.
- Develop a self-service API and control plane enabling the inference team to request, scale, and tear down inference clusters with a single API call, eliminating the need for human intervention.
- Automate self-healing processes by detecting degraded or failed nodes, safely draining them, triggering repair or replacement, and automatically reintroducing healthy capacity.
- Ensure the reliability of the provisioning pipeline through idempotency, retries, rollback, and drift detection, making the system as dependable as any other production service.
- Collaborate with the inference/ML platform team to understand their cluster requirements (topology, interconnect, scheduling constraints) and represent them as first-class abstractions within the platform.
- Engineer infrastructure code with a focus on software development best practices, including strong typing, automated testing, code review, versioning, and CI/CD.
Requirements
- Strong software engineering background in Go, Python, Rust, or similar languages.
- Experience with durable workflow orchestration tools like Temporal or Cadence for managing long-lived, manifest-driven workflows.
- Experience building software control planes or orchestration systems that model state and reconcile it over time (e.g., Kubernetes controllers/operators, custom reconciliation loops, workflow engines).
- Experience with event-driven systems, designing and building software around message queues, event streams, or pub/sub (e.g., Kafka, NATS, SQS).
- A product mindset, with experience building internal platforms or APIs consumed by other engineering teams and a focus on developer experience.
- Exposure to bare-metal provisioning (PXE/iPXE, Redfish/IPMI, BMC) and/or networking fundamentals (VLANs, BGP, fabric design).
- Familiarity with GPU/accelerator infrastructure.
- Experience with GPU cluster software stacks (NCCL, CUDA, InfiniBand/RoCE).
- Prior work at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization.
- Systems programming experience in Rust or Go.