Member of Technical Staff, Model Efficiency
Remote • New York • FullTime
Posted 8mo ago
About the job
Cohere is seeking a Member of Technical Staff focused on Model Efficiency to join a fast-growing team of researchers and engineers. This role is instrumental in pushing the boundaries of LLM inference efficiency, developing techniques to improve model execution in production for lower latency and higher throughput. You will work across the inference stack, identifying bottlenecks and developing optimizations, collaborating with modeling and systems teams to ship meaningful improvements. Opportunities exist to build expertise in advanced performance techniques like GPU/CUDA optimizations and model execution strategies for large-scale architectures.
Responsibilities
- Improve core performance metrics by diving deep into model execution.
- Identify and resolve performance bottlenecks across the model execution stack.
- Develop innovative optimizations for LLM inference.
- Collaborate with modeling and systems teams to experiment, measure, and ship improvements.
- Gain expertise in GPU/CUDA optimizations, kernel-level improvements, and model execution strategies for MoE and large-scale architectures.
Requirements
- 5+ years of experience writing high-performance, production-quality code.
- Strong programming skills in C++ or Python (Rust/Go also welcome).
- Experience working with large language models and familiarity with the LLM inference ecosystem (e.g., vLLM, SGLang).
- Ability to diagnose and resolve performance bottlenecks across the model execution stack.
- A strong bias for action: ship fast, measure impact, and iterate.
Benefits
- Weekly lunch stipend of $75/£75 or equivalent.
- Full health and dental benefits, including a separate budget for mental health.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months.
- Annual enrichment benefits (Arts & culture, fitness/wellness, quality time, workspace improvement).
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days).
- Budget for traveling to other offices if remote.
- Annual company offsite.
- Co-working benefit for those not near an office.
- $500 home office stipend.