Staff Software Engineer, Inference Cloud
Remote • Sunnyvale, CA • FullTime
Posted 2y ago
Job Location
Sunnyvale, CA
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Cerebras Systems is seeking a Staff Engineer to take ownership of significant aspects of their Inference Cloud Platform architecture. This role focuses on the cloud layer supporting the Inference Service, with key responsibilities for availability, latency, reliability, and global scale. It's a hands-on individual contributor position ideal for an engineer passionate about tackling complex distributed systems challenges, including multi-region traffic architecture, handling bursty AI workloads, optimizing performance at high QPS, and maintaining a fast, available platform under load. The engineer will be involved in writing code, making critical architectural decisions, debugging production issues, and influencing technical direction across related teams.
Responsibilities
- Shape the technical direction and roadmap for the Inference Cloud Platform, including multi-region topology, failure domains, and system evolution.
- Design and build core platform components like service discovery, request routing, load balancing, caching, batching, and traffic management for AI inference.
- Architect active-active systems with rapid failover, graceful degradation, and clear SLOs to improve latency, throughput, capacity efficiency, and resilience.
- Define platform mechanisms for admission control, quota management, rate limiting, and differentiated quality of service.
- Write and review production code in critical platform areas, making high-consequence architectural decisions and setting technical standards.
- Lead resolution of complex production issues and cross-system bottlenecks, driving observability, incident response, capacity planning, and post-incident improvements.
- Partner with ML, Product, Infrastructure, and Platform teams to translate requirements into scalable system designs and align on technical decisions.
- Mentor senior engineers through design feedback, pairing, and establishing clear technical standards.
Requirements
- 8+ years of software engineering experience, with significant individual contributor experience in large-scale distributed systems or cloud infrastructure.
- Deep expertise in distributed systems architecture within cloud environments, including networking, compute orchestration, container platforms, and multi-region production services.
- Proven track record of making sound architectural decisions for highly available, latency-sensitive systems at scale.
- Experience optimizing latency, throughput, and efficiency in high-QPS systems; TTFT and tail-latency reduction experience is a strong plus.
- Strong proficiency in backend or systems languages like Go, C++, or Python, capable of contributing production code.
- Experience designing and implementing observability and reliability practices (metrics, logging, tracing, alerting, incident response, SLOs).
- Ability to influence senior engineers and cross-functional partners through technical credibility, communication, and judgment.
Benefits
- Build a breakthrough AI platform beyond the constraints of the GPU.
- Publish and open source cutting-edge AI research.
- Work on one of the fastest AI supercomputers in the world.
- Job stability with startup vitality.
- Simple, non-corporate work culture that respects individual beliefs.