Software Engineer - Model API Platform
Salary not listed • Bengaluru • Full-time
Posted 18d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Sarvam is building India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make AI work for India. We partner with leading enterprises and public institutions, backed by top venture capital firms. We are seeking a Software Engineer to own major parts of our model APIs platform end-to-end. This platform handles millions of daily API calls for services like ASR, TTS, Vision, LLM, and Translate, including features like metering, billing, and rate-limiting. You will be responsible for the reliability, performance, and low latency of critical components, working closely with our Staff Engineer.
Responsibilities
- Own one or more major parts of the model APIs platform, including building, operating, and improving them.
- Translate loosely defined goals into technical specifications, project plans, and shipped results.
- Participate in on-call rotations, incident response, and root-cause analysis for platform issues.
- Optimize services and network paths to minimize latency and improve performance.
- Build and operate streaming infrastructure, including WebSocket connections and queue-based processing.
- Engineer audio pipelines for Text-to-Speech (TTS) and Automatic Speech Recognition (ASR), including VAD-based silence detection.
- Ensure services meet sub-100ms latency Service Level Objectives (SLOs).
- Design and implement systems with a focus on latency budgets, throughput, cost, failure modes, and tenant isolation.
- Integrate observability tools (logging, metrics, tracing) for efficient debugging.
- Strengthen integration testing harnesses to enable fast and safe deployments.
- Collaborate with Inference and MLOps teams to optimize model serving.
- Mentor junior engineers through code reviews and pairing sessions.
- Promote a documentation-first engineering culture with RFCs preceding code development.
Requirements
- 4-6 years of experience building and operating production backend systems.
- Proven experience owning a service end-to-end under significant load, including incident management and resolution.
- Strong understanding of distributed systems fundamentals, including concurrency, networking, queues, caching, consistency, and failure modes.
- Experience with low-latency or high-throughput services, with the ability to reason about p95 and p99 latency.
- Solid fundamentals with relational databases (e.g., PostgreSQL) and caches (e.g., Redis), including schema design and query performance.
- Fluency in at least one backend language (Python or Golang) and willingness to work in both.
- Experience running services using containers, CI/CD pipelines, and production monitoring tools.
- Demonstrated daily use of AI tools in engineering workflows, with concrete examples of automation or speed improvements.
- A bias for action, making fast reversible decisions and careful irreversible ones.
- Excellent writing skills, particularly for RFCs and design documents.