LLMops Engineer
India • Full-time
Posted 1mo ago
Job Location
India
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
LLM Engineer
About the job
Coursera and Udemy have merged to create a powerful learning platform, driven by AI. This role is for an AI Platform and LLMOps Engineer within a fast-paced innovation team focused on enabling AI across the organization and for enterprise customers. The team builds and maintains the infrastructure and tooling for AI adoption, creating a platform for internal teams to deploy AI tools and transforming business functions into AI-native operating models. Additionally, the team develops custom AI-powered solutions for enterprise clients to support their AI adoption and upskilling journeys.
You will be responsible for the operational backbone of agentic AI systems, ensuring reliability in production. This includes managing prompt and pipeline versioning, evaluation, monitoring, incident, and cost management. You will collaborate with Software Engineers, AI Specialists, and Product Managers to transform AI prototypes into reliable, observable, and cost-effective production systems.
Responsibilities
- Build and maintain operational tooling for LLM-powered systems, including prompt/pipeline versioning, evaluation, and CI/CD workflows for AI outputs.
- Implement monitoring and observability for production LLM systems, tracking latency, token usage, cost, output quality, and drift.
- Design and run automated evaluation suites to detect regressions and quality degradation.
- Manage RAG pipelines and vector store infrastructure for performance and accuracy.
- Implement safety and compliance guardrails for content filtering, PII redaction, and access controls.
- Manage cost governance for LLM usage, including caching strategies and model routing.
- Collaborate with Product Managers and Senior Engineers to define and implement operational requirements.
- Participate in agile development processes like sprint planning, technical grooming, and retrospectives.
Requirements
- Bachelor's or Master's degree in Computer Science, Computer Engineering, or a related field.
- 4+ years of experience in a software engineering role, with a strong background in Backend Engineering and DevOps.
- Hands-on experience operating or supporting LLM-powered systems in production (APIs, RAG, frontier/open weight/fine-tuned models).
- Proficiency in a backend language like Python, Java, TypeScript, or Go.
- Familiarity with Docker and container orchestration (Kubernetes).
- Experience implementing APIs, working with SQL and NoSQL databases, and writing automated tests.
- Working knowledge of at least one cloud platform (AWS, GCP, or Azure).
- Familiarity with infrastructure-as-code (e.g., Terraform) and CI/CD tooling (e.g., GitHub Actions, Jenkins).
- Comfort debugging production issues in non-deterministic systems.
- Strong belief in engineering quality and building leverage-creating tooling.
Benefits
- Work at the operational core of Coursera's AI transformation.
- Solve problems related to production AI system reliability, safety, and cost-effectiveness.
- Build hands-on expertise in a fast-growing engineering discipline.
- Join a supportive, innovative team with a culture of continuous learning.
- Be part of a mission-driven company transforming global access to education and upskilling.
