Staff Software Engineer
$207k - $300k • Mountain View, CA, USA; New York, NY, USA • Full-time
Posted 11d ago
About the job
At Google DeepMind, we are building the world's first general-purpose learning agent. This role focuses on measuring the intelligence of our AI prototypes, working with cutting-edge AI agents developed by our machine learning and neuroscience research scientists. You will create systems for agent testing in various environments, develop test problems, visualize results, build agent leaderboards, and test new algorithms on robots. The GenAI Eval Platform team provides a rigorous, high-velocity, and scalable evaluation backbone for Google's frontier AI models, working closely with research teams to steer AI progress by assessing model capability, safety, and reasoning depth. We are a pioneering AI lab focused on advancing AI development to solve complex global challenges and accelerate high-quality product innovation for billions of users, always prioritizing safety and ethics.
Responsibilities
- Design and optimize distributed evaluation execution engines for high-throughput, low-latency inference across TPU and GCU pools.
- Build foundational abstractions for evaluating complex LLM agent loops, tool use, and automated LLM-as-a-judge rating systems.
- Design robust error classification, automated retry policies, and observability dashboards to meet strict SLOs for evaluation pipeline success rates.
- Collaborate with research scientists and data science teams to translate frontier model evaluation requirements into scalable infrastructure solutions.
- Mentor engineers, uphold high code quality standards (Python in Google3), and advocate for rigorous testing and system design practices.
Requirements
- Bachelor's degree in Computer Science, Software Engineering, Mathematics, or a related STEM field, or equivalent practical experience.
- 5 years of experience coding in one or more general purpose languages (e.g., Python, Java, C or C++).
- 5 years of experience designing and launching large-scale production software products or foundation AI systems.
- 4 years of experience in software development.
- 3 years of experience leading software design and system architecture for high-throughput distributed systems, foundation model runtimes, or client applications.
- Master's degree or PhD in Computer Science, Artificial Intelligence, Robotics, or a related technical field (preferred).
- 5 years of experience in a technical leadership role setting technical direction and guiding project teams (preferred).
- 3 years of experience driving complex cross-functional initiatives across research, engineering, product management, and UX design (preferred).
- Experience developing, evaluating, or serving large language models, agentic workflows, multimodal pipelines, or AI runtime APIs (e.g., PyTorch, JAX, Triton, vLLM, Gemini API) (preferred).
Benefits
- 20% bonus target
- equity
- benefits
