Software Engineer, Evals
Bangalore, India
Posted 1mo ago
About the job
Glean is seeking backend and infrastructure engineers to build platforms that measure, explain, and improve AI quality at scale. This role involves owning core systems for large-scale evaluations, processing traces, powering observability workflows, and providing engineers with clear signals about assistant and agent behavior. You will work on systems that need to be scalable, secure, permissions-aware, fast, and cost-efficient, directly shaping how AI products are shipped. This is an opportunity to work at the intersection of distributed systems, data infrastructure, applied AI, and product quality, helping to build the systems that determine if new models, prompts, retrieval strategies, and agent workflows are ready for enterprise customers.
Responsibilities
- Design and build large-scale evaluation pipelines to measure assistant and agent quality across thousands of real user and synthetic workflows.
- Evaluate frontier model releases and the latest OSS model drops, building infrastructure and quality signals for understanding regressions, tradeoffs, and launch readiness.
- Build agent observability infrastructure, including trace enrichment, durable telemetry pipelines, dashboards, and debugging workflows.
- Own backend systems from architecture and design through production rollout, reliability, monitoring, and iteration.
- Partner with product, ML, and infrastructure engineers to integrate evals into the AI feature shipping process.
- Improve the quality loop by connecting eval results, customer feedback, regression analysis, and engineering workflows to product improvements.
- Build systems that balance speed, reliability, enterprise security, and cost in modern cloud-native environments.
- Mentor other engineers and contribute to the engineering culture of the AI quality team.
Requirements
- 6+ years of software engineering experience building backend systems, infrastructure, distributed systems, or data platforms.
- Strong coding skills in Go, Python, Java, C++, or similar languages, with an emphasis on reliability, scale, and well-tested components.
- Comfort working with distributed data pipelines, production services, and observability tools.