Senior Systems Engineer, Workers AI
In-Office
Posted 10d ago
Remote Work Policy
On-site
Categories
Applied AI Engineer
About the job
You will design and build the core infrastructure that powers AI inference across Cloudflare's global network, handling real-time voice, frontier open LLMs, and customer-deployed models on a heterogeneous fleet of GPUs and next-generation accelerators. This role involves solving complex problems in distributed systems and high-performance computing, such as sub-second model cold starts, multi-accelerator workload scheduling, efficient KV cache management, and a model deployment platform. We are building a novel AI inference platform embedded in the internet's fabric and are seeking high-agency systems engineers who are passionate about foundational infrastructure and defining how AI operates at the network edge.
Responsibilities
- Develop and maintain core components of the serverless inference platform for high availability and scalability.
- Optimize the model scheduling system for increased efficiency and resource utilization.
- Implement improvements to request routing logic to reduce end-user latency.
- Drive measurable improvements in platform reliability and resilience.
- Expand and refine the observability stack (metrics, logging, tracing) and tune alerts.
- Lead complex, cross-functional technical projects from concept to deployment.
- Mentor junior engineers and contribute to a collaborative engineering culture.
Requirements
- Proven experience in systems engineering, focusing on distributed, high-performance systems.
- Expert proficiency in Rust programming, especially in asynchronous environments.
- Deep understanding and hands-on experience with networking and application protocols (e.g., TCP, HTTP, WebSocket).
- Solid experience with scaling and performance optimization techniques, including load balancing and caching in distributed environments.