Inference Engineering and Product Lead
$300k - $350k • San Francisco • Full-time
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Modal is building the next-generation infrastructure layer for AI, enabling companies to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. Our platform provides instant GPU access, sub-second container starts, and native storage, supporting category-defining customers. We are seeking a leader to own the direction and execution of our LLM inference platform, a critical component of our offering that delivers frontier performance with best-in-class elasticity and developer experience. This is a hands-on leadership role where you will split your time between technical contributions, product shaping, and people management, setting direction, removing blockers, and fostering a strong engineering culture.
Responsibilities
- Recruit, hire, and grow a high-performing team of engineers, focusing on coaching and career development.
- Set clear performance expectations and build an environment where engineers can do their best work.
- Foster a culture of ownership, accountability, customer obsession, and continuous improvement.
- Drive technical and product decisions through design reviews, code reviews, and architectural discussions.
- Lead efforts to support customers with novel or frontier workloads on the Modal platform.
- Translate customer learnings into a roadmap for internal optimization platforms and user-facing products.
- Establish standards for reliability and product excellence, ensuring end-to-end project ownership.
- Partner with business operations and compute strategy on compute purchase strategies.
- Collaborate with Go-to-Market teams on product launches and improving inference opportunities.
- Guide the roadmap for teams building the underlying inference infrastructure and adjacent product teams.
Requirements
- 10+ years of industry experience, with at least 3 years in a leadership role.
- Proven track record of building high-performance systems at scale.
- Strong background in cloud infrastructure.
- Deep knowledge of low-level OS foundations, including Linux kernel, file systems, and containers.
- Experience with LLM inference in production is a plus.
- Familiarity with LLM inference concepts such as engines, kernels, routing, KV cache management, and speculative decoding is a plus.