Staff Engineer, Serverless Inference
$191k - $239k • Seattle • Full-time
Posted 1mo ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
DigitalOcean is seeking a Staff Engineer to design and optimize their Serverless Inference infrastructure and APIs. This role will focus on tackling large-scale AI workloads, emphasizing throughput, GPU utilization, and fault tolerance to support the next generation of AI-native enterprises. You will be instrumental in building scalable, multi-tenant services for AI inference and intelligent routing, while also strengthening platform resiliency through enhanced observability, capacity management, and automation. This position requires close collaboration with platform, GPU infrastructure, and product engineering teams to deliver robust, highly available systems and APIs.
Responsibilities
- Design and build scalable, multi-tenant services for AI inference and intelligent routing.
- Enhance platform resiliency via improved observability, capacity management, automation, and tooling.
- Collaborate with platform, GPU infrastructure, and product engineering teams.
- Raise the engineering bar through strong software design, operational discipline, and continuous improvement.
- Contribute to architecture decisions for traffic management, service orchestration, reliability, and scalability.
- Participate in on-call rotations and lead efforts to reduce operational pain and prevent incidents.
- Provide technical leadership by breaking down ambiguous problems and guiding execution plans.
- Support junior and mid-level engineers through coaching and feedback.
- Foster a culture of ownership, accountability, and continuous learning.
- Lead by example in incident response and production-quality engineering practices.
Requirements
- 7+ years of experience building and operating multi-tenant platforms or distributed backend systems.
- Strong experience operating high-scale distributed services in production.
- Deep understanding of SRE principles (observability, incident management, reliability, capacity planning, automation).
- 1+ years of hands-on experience with Go/Golang in production systems.
- 2+ years of experience with Kubernetes.
- Strong understanding of cloud-native architectures, microservices, and distributed systems.
- Experience debugging performance, scalability, and reliability issues in production.
- Proficiency in tracking infrastructure and inference metrics (e.g., TTFT, TPOT, GPU utilization).
Benefits
- Compensation Range: $191,200 - $239,000
- Equity compensation
- Annual bonus plan
- Reimbursement for relevant conferences, training, and education
- Flexible time off
- Local team meetups
- Mental health and well-being support