TPU Kernel Engineer
San Francisco, CA | New York City, NY | Seattle, WA
Posted 16d ago
Remote Work Policy
On-site
Categories
Applied AI Engineer
About the job
Anthropic's mission is to create reliable, interpretable, and steerable AI systems that are safe and beneficial for users and society. As a TPU Kernel Engineer, you will be responsible for identifying and addressing performance issues across various ML systems, including research, training, and inference. A significant part of this role involves designing and optimizing kernels specifically for the TPU, and providing feedback to researchers on how model changes affect performance. We are looking for individuals with a strong background in solving large-scale systems problems and performing low-level optimizations.
Responsibilities
- Identify and address performance issues in ML systems (research, training, inference).
- Design and optimize kernels for TPUs.
- Provide feedback to researchers on the performance impact of model changes.
- Implement low-latency, high-throughput sampling for large language models.
- Adapt existing models for low-precision inference.
- Build quantitative models of system performance.
- Design and implement custom collective communication algorithms.
- Debug kernel performance at the assembly level.
Requirements
- Significant experience optimizing ML systems for TPUs, GPUs, or other accelerators.
- Results-oriented with a bias towards flexibility and impact.
- Proactive in taking on tasks outside of the immediate job description.
- Experience with high-performance, large-scale ML systems.
- Experience designing and implementing kernels for TPUs or other ML accelerators.
- Deep understanding of accelerators, potentially with a background in computer architecture.
- Familiarity with ML framework internals.
- Experience with language modeling using transformers.
Benefits
- Annual compensation range: $280,000 - $850,000 USD
- Visa sponsorship available