Rust Engineer
San Francisco • FullTime
Posted 2mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
LiteLLM is seeking a Rust Engineer to lead the critical migration of its AI Gateway from Python to Rust. This high-visibility role involves owning the entire migration process, from planning with the CTO and restructuring the codebase to executing a stability-first, gradual rollout. The primary focus is on significantly improving performance, aiming to make LiteLLM the lowest-latency, highest-throughput AI gateway in the market without compromising reliability for enterprise customers. This is a single-owner mandate where your decisions will directly shape the platform's future performance and efficiency.
Responsibilities
- Own the end-to-end migration of LiteLLM from Python to Rust, including architecture and rollout.
- Plan the migration with the CTO, sequencing work to minimize risk to live traffic.
- Design and execute a gradual rollout prioritizing stability.
- Restructure the existing codebase for a clean, incremental move to Rust.
- Drive performance gains in latency, throughput, and resource efficiency for over 1 billion requests per month.
- Collaborate cross-functionally to align the migration with backend priorities in performance, reliability, and security.
Requirements
- Deep, long-standing expertise in Rust with years of production experience.
- Obsessive focus on performance, latency, and throughput.
- Strong instinct for stability and a track record of balancing change with reliability.
- Prior experience leading codebase migrations from another language to Rust is a major bonus.
- Demonstrated history of shipping performance-critical systems is a major bonus.
- Familiarity with backend infrastructure, databases (PostgreSQL, Redis), and high-volume production systems.
Benefits
- Own one of the most important technical bets in the company's roadmap.
- Define the architectural foundation for the entire platform.
- Fast-paced growth environment with deep technical ownership.
- Competitive salary
- Health benefits
- Dental benefits
- Vision benefits