New: upload your resume and get matched to AI jobs. Free to start →

Senior Staff Software Engineer, AI/ML, Google Cloud Compute Infrastructure

$262k - $364k • Sunnyvale, CA, USA • Full-time

Posted 17d ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

This role focuses on shaping the future of AI/ML hardware acceleration, specifically driving cutting-edge TPU (Tensor Processing Unit) technology that powers Google's most demanding AI/ML applications. You will be part of a team developing custom silicon solutions and contributing to the innovation behind products used by millions worldwide. The position involves verifying complex digital designs with a focus on TPU architecture and its integration within AI/ML-driven systems, bridging the gap between model architecture innovation and next-generation hardware design. Operating at the intersection of cutting-edge AI research and infrastructure engineering, you will define strategic goals and architectural roadmaps for future machine learning training and serving capabilities, leading the integration of foundational ML research with advanced architectures to deliver industry-defining, high-performance, and power-efficient accelerators.

Responsibilities

  • Lead the technical goal, roadmap, and software architecture for hardware/software co-design, optimizing large-scale ML inference and training systems.
  • Design, develop, and oversee simulation frameworks, cycle-accurate performance models, and system analysis tools to evaluate architectural trade-offs.
  • Architect scalable software solutions, kernels, and execution pipelines that bridge frontier ML model architectures with next-generation custom silicon accelerators.
  • Drive system-level performance optimization across distributed ML infrastructure, balancing compute, memory bandwidth, and inter-chip networking requirements.
  • Provide technical leadership and mentorship across hardware, compiler, and ML research teams to align on goals and transition co-design innovations into production.

Requirements

  • Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, a related technical field, or equivalent practical experience.
  • 8 years of experience in software development, systems engineering, or hardware/software co-design.
  • 5 years of experience with machine learning systems, architecture, performance modeling, or simulation frameworks.
  • Experience with ML frameworks (e.g., JAX, PyTorch, TensorFlow) and large language model (LLM) inference or training workloads.
  • Master’s degree or PhD in Engineering, Computer Science, or a related technical field (preferred).
  • 5 years of experience in a technical leadership role leading project teams and setting technical direction (preferred).
  • Experience driving multi-generational hardware/software co-design or performance optimizations for massive-scale ML inference and training (preferred).
  • Experience with accelerator programming, ML compilers, and deep learning execution models (preferred).
  • Experience analyzing system-level performance across compute, memory, interconnects, and packaging trajectories (preferred).

Benefits

  • 25% bonus target
  • Equity
  • Benefits

About Google