New: upload your resume and get matched to AI jobs. Free to start →

Senior Software Engineer, Machine Learning, TPU Workload Optimization

Singapore • Full-time

Posted 18d ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

Google is seeking a Senior Software Engineer to optimize TPU workloads for machine learning training and inference. This role is part of the AI Engine team, focusing on providing cutting-edge infrastructure and framework support for serving ML models on Cloud GPUs and TPUs. You will work with external customers to onboard and optimize their ML workloads, contributing to an emergent product with the potential to revolutionize how customers utilize Google's infrastructure for machine learning. This is a dynamic and exciting environment offering significant growth opportunities, where you will empower Google customers with breakthrough capabilities and insights at unparalleled scale, efficiency, and reliability.

Responsibilities

  • Lead the onboarding of stable ML stacks and optimize training/inference solutions on new TPUs.
  • Architect frameworks for training and inference onboarding and optimization tailored to customer needs.
  • Collaborate with ML research, performance, and tooling teams on optimization efforts.
  • Deliver stable solutions for customer training and inference optimization on TPUs.
  • Prepare and optimize large reference models for state-of-the-art single-host and multi-host inference at scale.

Requirements

  • Bachelor's degree or equivalent practical experience.
  • 5 years of software development experience in one or more programming languages.
  • 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, evaluation, data processing, debugging, fine-tuning).
  • 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).

About Google