Research Engineer - Inference
$150k - $350k • Remote • San Francisco • FullTime
Posted 1d ago
About the job
Prime Intellect is building the open superintelligence stack, a platform designed to unify compute, environments, evaluations, and deployment for frontier-scale AI development. We are creating open-source models trained for long-horizon tasks and the full-stack platform our research team uses to build them. We are looking for individuals passionate about building at the intersection of frontier research, real infrastructure, and go-to-market for a nascent category.
Responsibilities
- Extend vLLM integration to incorporate new model architectures into RL training, with validation before larger runs.
- Enhance rollout throughput for long-horizon agents by tuning concurrency and KV-cache reuse as context lengths increase.
- Optimize multi-node MoE inference, including expert parallelism and tradeoffs in separating prefill from decode.
- Reduce overhead for trainer-to-inference weight updates through sharding-aware transfer and communication overlap, ensuring rollout workers use the correct policy version.
- Build upon FP8 and evaluate lower precision, using trainer/sampler mismatch checks to detect numerical regressions and measure their impact on RL training.
- Collaborate with model researchers on architecture choices that improve inference efficiency, balancing model quality with execution cost on target hardware.
Requirements
- Experience building or optimizing GPU inference systems, with an understanding of inference engine internals.
- Strong Python skills and experience with PyTorch or a comparable ML framework.
- Ability to profile end-to-end workloads and translate measurements into verified performance improvements.
- Experience debugging numerical or integration issues in ML systems.
Benefits
- Cash Compensation Range of $150-350k
- Equity incentives
- Flexible work arrangements (remote or in-person)
- Visa sponsorship
- Relocation assistance
- Quarterly team off-sites
- Hackathons
- Conferences and learning opportunities