AI Engineer - Quality and Post-Training
$200k - $280k • Remote • San Francisco Bay Area
Posted 11h ago
Job Location
San Francisco Bay Area
Tech Stack
Remote Work Policy
Fully remote
Categories
LLM Engineer
About the job
Glean is seeking an AI Engineer focused on Quality and Post-Training to enhance their Work AI platform. This role is crucial for improving the reliability and usefulness of AI agents by focusing on measurement and quality improvements. You will work at the intersection of applied research, quality assurance, and product engineering to refine Glean's AI assistants and autonomous agents, ensuring they can reason, plan, and act effectively across complex workflows. The position involves optimizing the entire AI stack, including agent harnesses, model routing, prompts, and retrieval, while balancing quality with performance and cost.
Responsibilities
- Improve the AI stack (agent harness, model routing, prompts, retrieval) using eval results and production traces, balancing quality with latency and cost.
- Perform post-training on models using SFT and preference/reward-based optimization (DPO, PPO, GRPO), including data generation.
- Develop RL environments for simulating enterprise workflows, including evalsets, harness support, and LLM-based verifiers for benchmarking and hill-climbing.
- Enhance the quality of evalsets, LLM judges, and agentic verifiers, calibrating them against human graders for reliable release decisions and model training.
- Collaborate with product, platform, and research teams to implement quality improvements in production.
Requirements
- 5+ years of industry experience in AI.