Senior Systems Engineer, AI Gateway
$194k - $266k • In-Office
Posted 10d ago
Remote Work Policy
On-site
Categories
Applied AI Engineer
About the job
AI inference is becoming core infrastructure, and AI Gateway is Cloudflare's solution to provide this layer at the network edge. This role is for a Senior Systems Engineer to help build this critical component. You will be a hands-on individual contributor, translating product ideas into production-ready systems. Your work will span distributed systems, high-throughput APIs, developer workflows, SDKs, and documentation, shaping how developers build and operate AI applications on Cloudflare. You should be comfortable navigating the rapidly evolving AI ecosystem, building durable infrastructure amidst constant change. The role also emphasizes using AI-native tools and agents to enhance the engineering process and build internal tools for team efficiency.
Responsibilities
- Build core AI Gateway systems for high-volume inference traffic across providers, models, and Cloudflare's network.
- Design and implement APIs, routing, observability, controls, and developer workflows for AI Gateway.
- Collaborate with Workers AI, Agents, and the Developer Platform team to create a cohesive AI platform.
- Translate ambiguous ideas into prototypes, designs, features, and production improvements.
- Enhance reliability, performance, cost efficiency, and debuggability for AI workloads.
- Develop software and supporting tools like agents, tests, harnesses, and operational tools.
- Focus on developer experience through error messages, SDKs, dashboards, docs, and API design.
- Mentor other engineers through technical leadership and reviews.
- Contribute strong opinions and judgment, with a willingness to adapt based on data and reality.
Requirements
- 5+ years of experience building and operating production systems, preferably in AI infrastructure, developer platforms, cloud infrastructure, edge/proxy systems, high-scale APIs, or distributed systems.
- Strong systems engineering fundamentals including networking, APIs, reliability, observability, data modeling, performance, and operational tradeoffs.
- Experience designing or scaling distributed systems where latency, throughput, availability, and correctness are critical.
- Ability to work across the full stack, from API design and product surfaces to backend services, queues, storage, control planes, and performance-critical code.
- Strong understanding of developer tools and experience with APIs, SDKs, dashboards, CLIs, docs, and internal platforms.
- Clear technical communication skills for design docs, tradeoff explanations, architecture reviews, and aligning teams.
- A bias towards action with the ability to prototype quickly, ship incrementally, and improve systems in production.