Inference Engineer
*HQ - San Francisco, CA • FullTime
Posted 1y ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Cartesia is seeking an Inference Engineer to advance its mission of building real-time multimodal intelligence. This role involves designing and building a low-latency, scalable, and reliable model inference and serving stack for cutting-edge foundation models. You will collaborate closely with research and product engineering teams to serve products efficiently and reliably, and design robust inference infrastructure with monitoring. This position offers significant autonomy to shape products and influence how AI is applied across various devices and applications.
Responsibilities
- Design and build low latency, scalable, and reliable model inference and serving stack for foundation models using Transformers, SSMs, and hybrid models.
- Work with research and product engineers to serve products in a fast, cost-effective, and reliable manner.
- Design and build robust inference infrastructure and monitoring for products.
- Shape products and impact how AI is applied across devices and applications.
Requirements
- Strong engineering skills, comfortable navigating complex codebases and writing clean, maintainable code.
- Experience building large-scale distributed systems with high demands on performance, reliability, and observability.
- Technical leadership with the ability to execute and deliver zero-to-one results amidst ambiguity.
- Background in or experience working on inference pipelines with machine learning and generative models.
- Experience implementing state-of-the-art Machine Learning models and research to applied problems.
- Experience with vLLM, SGLang, Continuous Batching, or other inference frameworks (preferred).
- Experience working in CUDA, Triton, or similar (preferred).
Benefits
- Competitive base salary
- Attractive equity package
- Fully covered medical insurance, dental, and vision for employee and family
- 9 weeks paternity leave
- 12 weeks maternity leave
- 401(k)
- Monthly commuter allowance
- Flexible PTO
- Provided meals and snacks
- Personal Yoshi