Senior Engineering Manager, Kernel and Virt
$201k - $251k • Seattle • Full-time
Posted 4mo ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
DigitalOcean is seeking a Senior Engineering Manager to lead their Inference Orchestration team. This role will drive the strategy, execution, and scaling of the company's Kubernetes-based AI infrastructure. The manager will be responsible for balancing business needs with technical excellence, ensuring high throughput, optimal GPU utilization, and robust fault tolerance for next-generation disaggregated inference, fine-tuning, and training workloads. The position involves team leadership, project execution, cross-functional collaboration, and ensuring the operational health of services.
Responsibilities
- Recruit, mentor, and coach engineers, fostering a culture of ownership and technical excellence.
- Own team project execution, translating business goals into technical roadmaps and ensuring on-time delivery.
- Collaborate with Product Management and other engineering teams to align priorities and manage dependencies.
- Ensure the production health, stability, and on-call rotation of owned services.
- Define the technical roadmap and oversee the architecture of high-throughput scheduling systems for large Kubernetes clusters.
- Implement strategies to eliminate GPU waste through fractional GPU allocation and fairshare scheduling.
- Manage and orchestrate disaggregated AI inference pipelines using frameworks like NVIDIA Grove.
- Deploy topology-aware scheduling to optimize pod placement based on hardware dimensions.
- Drive initiatives to enhance cluster performance, including scheduling latency and API server load.
- Orchestrate efficient model weight distribution and implement Checkpoint/Restore for training fault recovery.
- Define and enforce security best practices for AI workloads, ensuring multi-layered isolation environments.
Requirements
- Proven track record of managing and growing high-performing engineering teams, preferably in distributed systems or infrastructure.
- Deep expertise in Kubernetes at scale and a strong understanding of AI workload orchestration, scheduling, and resource management.
- Strategic knowledge of GPU architectures (NVIDIA and/or AMD), interconnects, and hardware topology.
- Experience in balancing performance against cost, applying principles like Dominant Resource Fairness (DRF).
- Familiarity with container runtime internals, system isolation, and security contexts.
- Strong understanding of modern LLM serving architectures, disaggregation patterns, and common serving engines (e.g., vLLM, Triton, SGLang).
- Expertise in defining, tracking, and operationalizing infrastructure and inference metrics to meet service level objectives.
Benefits
- Annual bonus plan
- New hire equity grant
- Simplifying cloud and AI