Deep Learning Model and System Optimization Engineering Intern – 2027
Salary not listed • China, Shanghai • Internship
Posted 8d ago
Remote Work Policy
On-site
Employment Type
Internship
Categories
LLM Engineer
About the job
Our team develops GPU-accelerated software for deep learning inference, focusing on improving the performance, memory efficiency, scalability, and deployment cost of large language models on NVIDIA GPUs. We work across model architectures, algorithms, numerical representations, inference systems, and GPU architectures to achieve these goals. This role will contribute to inference frameworks like TensorRT-LLM, helping to bring state-of-the-art models to production efficiently. We are seeking passionate interns interested in large language model inference, low-precision computing, GPU performance optimization, and hardware-software co-design.
Responsibilities
- Analyze and optimize inference performance across model architecture, numerical precision, GPU kernels, memory systems, and end-to-end serving frameworks.
- Develop low-bit quantization techniques for model weights, activations, attention operands, and KV caches.
- Explore numerical formats and mixed-precision strategies for emerging low-precision formats while preserving model accuracy.
- Design and evaluate techniques for KV cache compression, sparsification, hierarchical storage, and reuse.
- Optimize the inference of various model architectures including MHA, GQA, MQA, MLA, and attention variants.
- Investigate model-system co-optimization techniques such as speculative decoding and state reuse.
- Build performance models and optimization strategies for long-context inference and agentic workloads, focusing on throughput, time to first token, latency, memory efficiency, and cost.
- Collaborate with model, framework, kernel, library, system software, and GPU architecture teams to influence future designs.
Requirements
- Pursuing an MS or PhD in a relevant field with an expected graduation date in 2028.
- Available for at least three months, at least four days per week.
- Strong understanding of Transformer architectures and large language model inference.
- Experience with at least one LLM inference framework (e.g., TensorRT-LLM, vLLM, SGLang).
- Proficiency in Python and PyTorch, with strong experimental design, data analysis, debugging, and problem-solving skills.
- Experience with at least one GPU programming technology (e.g., CUDA C/C++, Triton).
- Understanding of GPU architecture, memory hierarchy, Tensor Cores, parallel computing, and performance analysis.
- Familiarity with low-precision computing concepts, including quantization error, scaling, and mixed-precision computation.
- Ability to evaluate optimization techniques across model accuracy, kernel performance, and end-to-end inference performance.
- Good communication, technical writing, organization, and cross-functional collaboration skills.