TPU Performance Engineer
$280k - $850k • San Francisco, CA | New York City, NY | Seattle, WA
Posted 6d ago
Job Location
San Francisco, CA | New York City, NY | Seattle, WA
Tech Stack
Remote Work Policy
On-site
Categories
Machine Learning Engineer
About the job
As a TPU Kernel Engineer, you will be responsible for identifying and addressing performance issues across various ML systems, including research, training, and inference. A significant part of this role involves designing and optimizing kernels specifically for the TPU. You will also provide crucial feedback to researchers regarding how model changes affect performance. The ideal candidate will have a proven history of solving large-scale systems problems and demonstrating expertise in low-level optimization.
Responsibilities
- Identify and address performance issues across ML systems (research, training, inference).
- Design and optimize kernels for TPUs.
- Provide feedback to researchers on the performance impact of model changes.
- Implement low-latency, high-throughput sampling for large language models.
- Adapt existing models for low-precision inference.
- Build quantitative models of system performance.
- Design and implement custom collective communication algorithms.
- Debug kernel performance at the assembly level.
Requirements
- Significant experience optimizing ML systems for TPUs, GPUs, or other accelerators.
- Results-oriented with a bias towards flexibility and impact.
- Proactive in taking on tasks outside of the immediate job description.
- Experience with high-performance, large-scale ML systems.
- Experience designing and implementing kernels for TPUs or other ML accelerators.
- Deep understanding of accelerators, potentially with a background in computer architecture.
- Familiarity with ML framework internals.
- Experience with language modeling using transformers.