Member of Technical Staff, Inference & RL Systems
$275k - $550k • San Francisco • FullTime
Posted 7mo ago
About the job
As a Research Engineer on the Inference & RL Systems team, you will design and operate the distributed systems that serve our models in production and power large-scale post-training workflows. This role sits at the boundary between model execution and distributed infrastructure, focusing on systems that determine inference latency, throughput, stability, and the reliability of RL and post-training training loops. You will own the infrastructure that makes both production inference and large-scale RL iteration fast and reliable, addressing demanding execution constraints introduced by Magic’s long-context models.
Responsibilities
- Design and scale high-performance inference serving systems
- Optimize KV-cache management, batching strategies, and scheduling
- Improve throughput and latency for long-context workloads
- Build and maintain distributed RL and post-training infrastructure
- Improve reliability of rollout, evaluation, and reward pipelines
- Automate fault detection and recovery for serving and RL systems
- Profile and eliminate performance bottlenecks across GPU, networking, and storage layers
- Collaborate with Kernels and Research to align execution systems with model architecture
Requirements
- Strong software engineering and distributed systems fundamentals
- Experience building or operating large-scale inference or training systems
- Deep understanding of GPU execution constraints and memory trade-offs
- Experience debugging performance issues in production ML systems
- Ability to reason about system-level trade-offs between latency, throughput, and cost
- Track record of owning critical production infrastructure
Benefits
- Equity is a significant part of total compensation, in addition to salary
- 401(k) plan with 6% salary matching
- Generous health, dental and vision insurance for you and your dependents
- Unlimited paid time off
- Visa sponsorship and relocation stipend to bring you to SF, if possible