Software Engineer, GPU Performance
Salary not listed • Los Angeles, Palo Alto, San Francisco, Toronto, Singapore • Full-time
Posted 1mo ago
Job Location
Los Angeles, Palo Alto, San Francisco, Toronto, Singapore
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
HeyGen is building AI applications including Avatar IV, Photo Avatar, Interactive Avatar, and Video Translation. We are looking for a Software Engineer focused on GPU performance to make the systems behind these experiences faster and more efficient. You will work across model execution and inference infrastructure, using profiling and measurement to improve latency, throughput, and GPU cost. This role is a fit for an engineer who enjoys understanding how software uses the hardware beneath it.
Responsibilities
- Investigate GPU utilization, kernel execution, memory bandwidth, and CPU–GPU data movement using profiling tools.
- Identify and measure bottlenecks across model execution, preprocessing, and inference serving.
- Improve performance through batching, scheduling, memory management, and better GPU utilization.
- Develop or integrate high-performance GPU kernels when existing implementations limit performance.
- Build benchmarks and automated checks to catch performance regressions.
- Collaborate with AI researchers and infrastructure engineers to implement optimizations.
- Measure the impact of changes on latency, throughput, cost, and output quality.
Requirements
- Experience optimizing GPU-based AI workloads or high-performance computing systems.
- Proficiency in Python and experience with PyTorch or a similar machine learning framework.
- Strong understanding of GPU hardware, including memory bandwidth, cache behavior, tensor cores, and data movement.
- Experience using profiling tools to identify and validate performance improvements.
- Ability to translate performance experiments into production changes and communicate tradeoffs.
Benefits
- Competitive salary and benefits package.
- Dynamic and inclusive work environment.
- Opportunities for professional growth and advancement.
- Collaborative culture that values innovation and creativity.
- Access to the latest technologies and tools.