Senior Staff Software Engineer, ML Performance, GPU
$262k - $364k • Sunnyvale, CA, USA • Full-time
Posted 1mo ago
Job Location
Sunnyvale, CA, USA
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Google's Core ML organization is seeking software engineers for the ML Performance team to drive GPU performance optimizations for Gemini models. Working in close partnership with DeepMind and XLA, the team also writes custom kernels in CUDA, Triton, and Pallas/Mosaic to achieve peak efficiency. The ML Performance team focuses on performance analysis such as headroom/roofline analysis, benchmarking, and hill-climbing, along with advanced optimization techniques including sharding, compiler optimizations, quantization, custom kernels, and agentic optimizations. Beyond delivering immediate model-specific improvements, the team achieves scalable results by promoting these point optimizations into principled, stack-level enhancements across compilers and kernel libraries to boost out-of-the-box performance.
Responsibilities
- Identify and maintain LLM training and serving benchmarks to drive XLA:GPU/Triton performance and guide XLA releases.
- Collaborate with DeepMind to solve ML model performance challenges, onboard new LLM models and products on GPU hardware, and enable efficient LLM training and serving at scale.
- Run architecture-level simulations on GPU designs, perform roofline analysis, and explore novel model/data efficiency techniques.
- Analyze performance and efficiency metrics to identify bottlenecks and design/implement solutions at Google fleet-wide scale.
- Collaborate with tooling and fleet metrics sub-teams to build tools for tracking performance and efficiency and extracting metrics from Google's running workloads.
Requirements
- 8 years of experience in software development.
- 7 years of experience leading technical project strategy, ML design, and optimizing end-to-end Machine Learning.
- Experience with modern GPU architectures, memory hierarchies, and performance bottlenecks.
- Experience with low-level GPU programming (CUDA, Triton, CUTLASS, etc.) and performance engineering techniques.
- Experience with modern LLMs and their deployment on AI accelerators.
- Bachelor’s degree or equivalent practical experience.
Benefits
- 25% bonus target
- Equity
- Benefits
