Research Intern, Inference (Winter 2027)

The Inference Research team is dedicated to building the next generation of efficient, scalable, and reliable serving systems for large foundation models, advancing open and transparent AI. This role operates at the intersection of model architectures, high-performance systems engineering, and hardware optimization, focusing on co-designing software, algorithms, and models to reduce the cost and latency of AI systems. As a research intern, you will explore distributed inference, compiler-aware optimization, and novel inference-time computation strategies, co-designing and implementing cross-layer optimizations across models, systems, and hardware, with a focus on KV cache design and large-scale serving architectures. Projects aim to unlock unprecedented performance and scale for foundation models, enabling faster serving, larger model deployment, and robust evaluation under realistic serving workloads.