Senior Software Engineer, Machine Learning, TPU Workload Optimization
Singapore • Full-time
Posted 18d ago
Job Location
Singapore
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Google is seeking a Senior Software Engineer to optimize TPU workloads for machine learning training and inference. This role is part of the AI Engine team, focusing on providing cutting-edge infrastructure and framework support for serving ML models on Cloud GPUs and TPUs. You will work with external customers to onboard and optimize their ML workloads, contributing to an emergent product with the potential to revolutionize how customers utilize Google's infrastructure for machine learning. This is a dynamic and exciting environment offering significant growth opportunities, where you will empower Google customers with breakthrough capabilities and insights at unparalleled scale, efficiency, and reliability.
Responsibilities
- Lead the onboarding of stable ML stacks and optimize training/inference solutions on new TPUs.
- Architect frameworks for training and inference onboarding and optimization tailored to customer needs.
- Collaborate with ML research, performance, and tooling teams on optimization efforts.
- Deliver stable solutions for customer training and inference optimization on TPUs.
- Prepare and optimize large reference models for state-of-the-art single-host and multi-host inference at scale.
Requirements
- Bachelor's degree or equivalent practical experience.
- 5 years of software development experience in one or more programming languages.
- 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, evaluation, data processing, debugging, fine-tuning).
- 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).
