Senior Staff Applied AI Inference Engineer
$250k - $300k • San Francisco, CA - US • Full-time
Posted 1mo ago
About the job
Crusoe is seeking a Senior Staff Applied AI Inference Engineer to accelerate the abundance of energy and intelligence by optimizing large language models for production environments. This role involves owning the inference stack end-to-end, from profiling performance and cost to implementing modern optimization techniques and diving deep into serving code. The work is hands-on, focusing on coding, profiling, and low-level optimization, with a customer-facing aspect to tailor deployments to specific needs and guide workloads from proof of concept to production. You will collaborate with customer engineering teams, shape fuzzy goals into clear specifications, and ensure engineered gains are realized by end-users.
Responsibilities
- Optimize inference techniques for production deployments.
- Design and optimize serving architectures, including prefill and decode disaggregation.
- Profile and analyze serving stack performance down to the kernel level to identify and resolve issues.
- Adapt and scale optimization methods for various ML models, with a focus on LLMs.
- Tune deployments for latency, throughput, and cost targets, ensuring reliability under real traffic.
- Tailor deployments to customer-specific models and constraints, partnering with their engineering teams.
- Build and support software and product features for the inference stack in production.
- Experiment quickly, shape goals into specs, run focused proofs of concept, and ship tested results.
- Own delivery end-to-end, from initial experiments to production deployment.
- Navigate ambiguity and make sound decisions on tradeoffs and tooling.
- Take ownership and accountability for work, and expect the same from colleagues.
Requirements
- Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field.
- Hands-on experience shipping production code in general-purpose languages, preferably Python.
- Familiarity with methods for optimizing LLMs for high throughput and low latency inference.
- Comfort with modern LLM serving frameworks like vLLM or SGLang.
- Proficiency in profiling and analyzing performance down to the kernel level.
- Solid understanding of GPU architecture and behavior.
- Demonstrated interest and hands-on experience with large language models.
- Working knowledge of AI/ML pipelines and the full model development and deployment lifecycle.
- Strong communication skills, especially for explaining complex technical topics.
Benefits
- Competitive compensation and equity packages
- Restricted Stock Units
- Paid time off, paid holidays & leave of absence programs
- Comprehensive health, dental & vision insurance
- Employer contributions to HSA account
- Paid parental leave
- Paid life insurance, short-term and long-term disability
- Professional development & tuition reimbursement
- Mental health & wellness support
- Commuter benefits (parking & transit)
- Cell phone stipend
