Machine Learning Performance Engineer
Salary not listed • London, England, United Kingdom • Full-time
Posted 2y ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
Job Location
London, England, United Kingdom
Also open in Hong Kong, Hong Kong, New York, New York, United States
Remote Work Policy
On-site
Employment Type
Full-time
Categories
Machine Learning Engineer
About the job
We are seeking an engineer with expertise in low-level systems programming and optimization to join our expanding ML team. Machine learning is a crucial component of Jane Street's global business, and our dynamic trading environment provides a unique platform for rapid ML experimentation. Your primary role will be to optimize the performance of our machine learning models, encompassing both training and inference. This involves ensuring efficient large-scale training, low-latency inference for real-time systems, and high-throughput inference for research. The work requires a holistic systems approach, considering storage, networking, and host/GPU-level factors, down to the most granular details of hardware performance.
Responsibilities
- Optimize the performance of machine learning models for both training and inference.
- Improve large-scale training efficiency.
- Achieve low-latency inference in real-time systems.
- Ensure high-throughput inference for research applications.
- Address whole-systems performance considerations including storage, networking, and host/GPU levels.
- Investigate and optimize performance at the lowest hardware levels, such as L2 cache access.
- Debug and optimize training run performance end-to-end.
- Utilize low-level GPU knowledge (PTX, SASS, warps, Tensor Cores, memory hierarchy) for optimization.
- Apply debugging and optimization tools like CUDA GDB, NSight Systems, and NSight Compute.
- Leverage knowledge of libraries such as Triton, CUTLASS, CUB, Thrust, cuDNN, and cuBLAS.
- Understand and optimize latency and throughput characteristics of CUDA graph launches, tensor core arithmetic, warp-level synchronization, and asynchronous memory loads.
- Utilize networking technologies like Infiniband, RoCE, GPUDirect, PXN, rail optimization, and NVLink for GPU cluster connectivity.
- Understand collective algorithms for distributed GPU training using NCCL or MPI.
- Propose innovative approaches and critically evaluate existing methods and tools.
Requirements
- Experience in low-level systems programming and optimization.
- Understanding of modern ML techniques and toolsets.
- Systems knowledge to debug training performance end-to-end.
- Low-level GPU knowledge including PTX, SASS, warps, cooperative groups, Tensor Cores, and memory hierarchy.
- Debugging and optimization experience with tools like CUDA GDB, NSight Systems, and NSight Compute.
- Familiarity with libraries such as Triton, CUTLASS, CUB, Thrust, cuDNN, and cuBLAS.
- Intuition about latency and throughput characteristics of CUDA graph launch, tensor core arithmetic, warp-level synchronization, and asynchronous memory loads.
- Background in networking technologies for GPU clusters: Infiniband, RoCE, GPUDirect, PXN, rail optimization, NVLink.
- Understanding of collective algorithms for distributed GPU training (NCCL or MPI).
- Inventive approach and willingness to question existing methods and tools.
- Fluency in English.