Software Engineer, AI Gateway
$196k - $294k • Remote • Hybrid - San Francisco
Posted 23d ago
Job Location
Hybrid - San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Infrastructure Engineer
About the job
As a member of the AI Gateway Team, you will be responsible for developing and enhancing our AI Gateway platform, which provides a unified API for accessing hundreds of AI models from multiple providers. You will collaborate with a remote and distributed team of engineers to build reliable, low-latency systems that handle rate limiting, intelligent failovers, and seamless integrations, enabling developers to ship AI-powered applications without managing provider-specific complexities. This role focuses on ensuring production-ready reliability for AI workloads, including automatic fallbacks during outages and consistent performance across providers like OpenAI, Anthropic, Google, and more.
Responsibilities
- Contribute to the design, implementation, and maintenance of the AI Gateway platform, focusing on unified API endpoints, rate limit management, and intelligent failover mechanisms.
- Write clean, efficient, and well-documented code with thorough testing to ensure low-latency responses and stability for high-volume AI inference requests.
- Collaborate with cross-functional teams to integrate new AI providers and models, addressing scalability and performance challenges.
- Engage with the open-source community and contribute to AI SDK and related projects.
- Gather user feedback and analytics to drive innovations in AI Gateway, such as enhanced billing unification and provider-agnostic authentication.
Requirements
- 5+ years of relevant experience
- Strong proficiency in JavaScript/TypeScript
- Experience with backend development, APIs, and cloud infrastructure
- Experience with AI/ML integrations, API gateways, distributed systems, or handling high-throughput services (e.g., rate limiting, caching, failovers)
- Experience contributing to or participating in open source projects
- Excellent communication skills and ability to work effectively in a collaborative team environment
Benefits
- Competitive compensation package, including equity
- Inclusive Healthcare Package
- Mentorship and opportunities to attend events for networking and skill-building
- Flexible Time Off
- Provision of necessary gear and a WFH budget