Senior Systems Engineer, AI Gateway
$194k - $266k • In-Office
Posted 5d ago
Job Location
In-Office
Tech Stack
Remote Work Policy
On-site
Categories
AI Infrastructure Engineer
About the job
AI inference is becoming core infrastructure, and AI Gateway is Cloudflare's solution to provide developers with reliable, observable, secure, and cost-controlled access to multiple AI models across various providers, all at the network edge. This is a hands-on individual contributor role focused on building this critical layer. You will work across the entire stack, from distributed systems and high-throughput APIs to developer-facing tools like SDKs and dashboards, shaping how developers build and operate AI applications on Cloudflare. The ideal candidate thrives in the fast-paced AI ecosystem, building durable infrastructure amidst rapidly changing APIs and models, and is excited about using AI to improve the software development process itself.
Responsibilities
- Build core AI Gateway systems for high-volume inference traffic across providers and models.
- Design and implement APIs, routing, observability, and developer workflows for AI Gateway.
- Collaborate with Workers AI, Agents, and Developer Platform teams to create a cohesive AI platform.
- Translate product and technical ideas into prototypes, designs, and production features.
- Enhance reliability, performance, cost efficiency, and debuggability for AI workloads.
- Develop software, including agents, tests, harnesses, evaluation loops, and operational tools.
- Focus on and improve the developer experience through error messages, SDKs, dashboards, and documentation.
- Mentor other engineers through design and code reviews, and provide technical leadership.
- Contribute strong opinions and judgment, and adapt views based on data and reality.
Requirements
- 5+ years of experience building and operating production systems, preferably in AI infrastructure, developer platforms, cloud infrastructure, edge/proxy systems, high-scale APIs, or distributed systems.
- Strong systems engineering fundamentals including networking, APIs, reliability, observability, data modeling, performance, and operational tradeoffs.
- Experience designing or scaling distributed systems with a focus on latency, throughput, availability, and correctness.
- Ability to work across the full stack, from API design and product surfaces to backend services, queues, storage, control planes, and performance-critical code.
- Strong understanding of developer tools, including experience with APIs, SDKs, dashboards, CLIs, and internal platforms.
- Excellent technical communication skills, including writing design docs, explaining tradeoffs, and reviewing architecture.
- A bias towards building working software, with the ability to prototype quickly and ship incrementally.