Member of Technical Staff (TPM, Inference)
$170k - $265k • Remote • San Francisco • FullTime
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Perplexity is seeking a Technical Program Manager to act as the central point of contact between our model providers, engineering, and product teams, driving the advancement of our core inference platform. This role is crucial for managing one of the industry's highest-throughput inference stacks, serving diverse traffic across a dynamic portfolio of models. You will operate at the nexus of product, engineering, and finance, coordinating the smooth integration of new models and capacity while executing the roadmap for the inference platform itself. The ideal candidate possesses strong technical acumen, excels at coordinating diverse teams and external partners with competing deadlines, and is motivated by establishing operating models for functions with limited precedent.
Responsibilities
- Execute the roadmap for inference platform request handling, rate limits, quotas, usage controls, and reliability/observability.
- Serve as the liaison between model providers and Perplexity's engineering/product teams for onboarding, launch readiness, and rollout of new models and capacity.
- Drive improvements in latency, throughput, uptime, and cost-efficiency, proactively surfacing tradeoffs.
- Manage the operating model for model release and optimization programs, including day-zero launches, across performance engineering, infrastructure, and product teams.
- Lead cross-functional delivery for inference stack changes from planning through post-launch validation.
- Develop mechanisms for predictable release rituals, dashboards, and launch checklists to ensure low-risk inference releases at scale.
- Collaborate with GPU capacity and compute teams to align execution decisions with cost, capacity, and vendor constraints.
Requirements
- Significant experience in technical program management or product management within infrastructure, distributed systems, or ML/model-serving products.
- Direct experience with production LLM or ML inference, understanding the nuances of fast, reliable, and cost-effective serving.
- Proficiency in orchestrating across external partners and internal engineering teams with competing priorities.
- Experience with data and metrics, with the ability to identify and communicate difficult tradeoffs between latency, throughput, uptime, and cost.
- Ability to thrive in a small, agile team, demonstrating initiative and ownership in ambiguous environments.
- 6+ years of combined technical program management or product management experience.