Senior Software Engineer, GenAI Evaluation
$174k - $252k • New York, NY, USA • Full-time
Posted 11d ago
Job Location
New York, NY, USA
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Engineer
About the job
Google is seeking experienced software engineers to develop and enhance the next generation of technologies that connect, explore, and interact with information. This role involves working on critical projects at a massive scale, extending beyond web search. Engineers will contribute fresh ideas across various domains including AI, NLP, and large-scale systems. The position offers opportunities to switch teams and projects, requiring versatility, leadership, and enthusiasm for tackling new challenges across the full stack.
Responsibilities
- Design, build, and maintain reusable evaluation infrastructure components like streaming evaluation, autorater pipelines, and metric aggregation.
- Enhance the scalability, reliability, and efficiency of large-scale evaluation workflows across compute, storage, and serving backends.
- Onboard product teams, troubleshoot their issues, and develop self-serve features for recurring needs.
- Implement secure per-customer data isolation and access control for evaluation data and results.
- Facilitate the transition of teams from offline evaluations to online evaluations, and subsequently to live experiments and production.
Requirements
- Bachelor's degree or equivalent practical experience.
- 5 years of experience coding in Python.
- 3 years of experience developing large-scale infrastructure, distributed systems, networks, or compute technologies, storage, or hardware architecture.
- 3 years of experience working with Generative AI agents.
- 2 years of experience with multithreading.
- 2 years of experience with large-scale data processing, storage systems, or workflow orchestration.
- 2 years of experience with two or more of the following: distributed and parallel systems, machine learning, information retrieval, natural language processing, Unix/Linux environments, or security and access control.
- Experience with ML infrastructure, model evaluation, or LLM and GenAI workflows (e.g., autoraters, metrics, or experimentation platforms).
- Experience developing in Python and C++.
Benefits
- 15% bonus target
- Equity
- Benefits
