Staff Engineer, API Platform
Bengaluru • FullTime
Posted 3mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Sarvam is building India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make AI work for India. We partner with leading enterprises and public institutions, backed by prominent venture capital firms. Our model APIs, handling millions of daily calls, are crucial for developers and enterprises shipping products on our foundation models. Currently built with FastAPI and Python, we are undertaking a significant rewrite in Go. We are seeking a Staff Engineer to take end-to-end ownership of this critical platform, encompassing its architecture, reliability, performance, and standards.
Responsibilities
- Lead the end-to-end design and evolution of the API platform, from request ingress to response egress.
- Drive the Python to Go rewrite, defining architecture, setting patterns, and ensuring a reliable migration.
- Engineer audio pipelines, including TTS chunking, sample-rate adjustments, format encoding, and VAD-based silence detection for ASR.
- Develop vision pipelines for orchestrating OCR, layout detection, and VLM harnesses for structured data extraction.
- Build streaming infrastructure for WebSocket connections, queue-based batch processing, back pressure, and low-latency model invocation.
- Develop the commercial layer, including metering, billing, prepaid wallet management, and rate-limiting.
- Ensure performance and reliability for a multi-tenant platform with sub-second latency SLOs.
- Implement robust observability (logging, metrics, tracing) for efficient production debugging.
- Develop integration testing infrastructure to enable rapid, safe deployments.
- Collaborate closely with Inference and MLOps teams, mentoring them on production engineering best practices.
- Foster a design-first, documentation-first engineering culture, emphasizing RFCs and written decision-making.
Requirements
- 7+ years of experience building production backend systems at scale.
- Strong proficiency in Go, with experience designing, building, and operating Go services under real load.
- Comfort working with Python, particularly FastAPI.
- Proven track record of designing and operating high-scale, low-latency, multi-tenant distributed systems.
- Hands-on experience with real-time/streaming systems, including WebSockets, long-lived connections, and backpressure.
- Strong fundamentals in PostgreSQL and Redis, including schema design, query performance, and caching strategies.
- Experience running services on Kubernetes in production.
- Demonstrated judgment as a senior individual contributor, including build vs. buy decisions, optimization strategies, and mentoring capabilities.
Benefits
- High ownership and high impact from day one.
- Opportunity to work on problems with population-scale impact.
- Work alongside a fast-moving, high-talent-density team.
- AI-first approach to building and problem-solving.