New: upload your resume and get matched to AI jobs. Free to start →

Senior Software Engineer, ML Performance, GenAI

$174k - $252k • Mountain View, CA, USA • Full-time

Posted 15d ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Engineer

About the job

At Google DeepMind, we are seeking a Senior Software Engineer to focus on ML Performance within GenAI. This role is central to our mission of building the world's first general-purpose learning agent, with a key focus on measuring the intelligence of our prototypes. You will work with cutting-edge AI agents developed by our exceptional team of Machine Learning and Neuroscience research scientists. Your responsibilities will span creating systems for agent testing in various environments, developing test problems within physics simulators, visualizing results, building competitive agent leaderboards, and testing new algorithms on robots. We emphasize identifying and achieving performance improvements in production and research ML workloads, particularly LLM inference (latency, throughput, cost per token) and agentic workflows on current and next-generation TPU platforms. This involves working across the stack from model architecture to Pallas/Mosaic kernels and XLA, and partnering with Gemini modeling, Deployment, Compiler, and TPU hardware teams to influence future accelerator capabilities.

Responsibilities

  • Optimize Gemini inference cost per token by improving throughput, accelerator utilization, batching, scheduling, and KV cache efficiency.
  • Build agentic workflows enabling AI agents to discover, implement, and validate performance improvements at scale.
  • Characterize and tune Gemini on new TPU generations, optimizing kernels, sharding, and serving configurations for production readiness.
  • Influence future TPU hardware and compiler roadmaps by sharing performance insights.
  • Implement model compression techniques like quantization and sparsity to reduce cost-per-token for production Gemini models.
  • Evaluate new architectures, optimizers, and training methods to achieve target quality with reduced compute requirements.
  • Create systems for agent testing using 2D and 3D games.
  • Develop test problems within physics simulators.
  • Create graphical visualizations of results.
  • Build competitive agent leaderboards.
  • Test new algorithms on robots.

Requirements

  • Bachelor's degree in Computer Science, Software Engineering, Mathematics, or a related STEM field, or equivalent practical experience.
  • 5 years of experience in software development, systems programming, or applied ML engineering.
  • 3 years of experience with performance optimization, systems data analysis, visualization tools, or debugging.
  • 3 years of experience with Generative AI, Large Language Models (LLMs), Machine Learning, and related frameworks.
  • 1 year of experience designing and launching production software products or foundation AI systems.
  • Master's or PhD in Computer Science, Artificial Intelligence, Robotics, or related technical field (preferred).
  • 3 years of experience in a technical leadership role setting architectural direction and guiding project teams (preferred).
  • Experience developing, evaluating, or serving large language models, agentic workflows, multimodal pipelines, or AI runtime APIs (e.g., PyTorch, JAX, Triton, vLLM, Gemini API) (preferred).
  • Expertise or strong interest in performance debugging/optimization, performance tools design, compiler design, code optimization, high-performance software development, concurrent programming, or multi-core computer architectures (preferred).

Benefits

  • 15% bonus target
  • Equity
  • Benefits

About Google DeepMind