New: upload your resume and get matched to AI jobs. Free to start →

Software Engineering Manager, AI/ML Infrastructure and Performance Engineering

Bengaluru, Karnataka, India • Full-time

Posted 11d ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

As a Software Engineering Manager, you will lead a team of engineers, providing technical leadership and strategic direction for major projects. Your role involves not only optimizing your own code but also empowering your team to optimize theirs. You will manage project goals, contribute to product strategy, and foster team development. This position is within the ML, Systems, & Cloud AI (MSCA) organization, which designs, implements, and manages the hardware, software, machine learning, and systems infrastructure for all Google services and Google Cloud. The team focuses on driving continuous improvements to the machine learning software/hardware stacks through insightful performance debugging for workloads and custom kernels, summarizing profile data from various sources.

Responsibilities

  • Gain an intuitive understanding of data collection, analysis, and visualization workflows across frameworks, XLA, and the runtime stack.
  • Contribute to the end-to-end stack and analysis tools to support new ML paradigms like horizontal scaling for upcoming TPU chips.
  • Collaborate with ML Stack leads to understand model optimization use cases and integrate debugging into third-party environments (VSCode, Cursor, Grafana).
  • Work with open-source ML inference frameworks (e.g., vLLM, SGLang) to identify performance improvement opportunities.
  • Partner with teams owning parts of the ML stack to understand performance optimization use cases and provide insights into performance bottlenecks using frameworks like TorchTPU, vLLM, and SGLang.
  • Build agentic workflows for performance debugging and optimization.

Requirements

  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience in software engineering, machine learning infrastructure, computer architecture, distributed computing, people management, debugging tools, and communication.
  • Experience in people management and building infrastructure to improve ML system and application performance.
  • Experience with performance optimization, GPU programming, High Performance Computing, and Large Language Models.
  • Experience with ML infrastructure and frameworks, and GPU or TPU performance analysis.
  • Experience with ML frameworks such as TensorFlow, JAX, PyTorch, and Keras.
  • Experience with ML inference frameworks such as vLLM, SG Lang, and Pathways.
  • Experience in open-source software development, including releasing and supporting open-source projects.

About Google