Software Engineer, Inference AI/ML
$92k - $135k • Sunnyvale, CA / Bellevue, WA
Posted 5d ago
Job Location
Sunnyvale, CA / Bellevue, WA
Tech Stack
Remote Work Policy
On-site
Categories
LLM Engineer
About the job
CoreWeave is seeking a Software Engineer to join the Inference team, focusing on shipping production features that enhance model serving on their GPU platform. This role involves improving latency, reliability, and cost-efficiency. As an IC1, you will implement well-scoped changes, learn operational practices, and grow with mentorship from experienced engineers. CoreWeave is the essential cloud for AI, providing a platform of technology, tools, and teams that enable innovators to build and scale AI with confidence.
Responsibilities
- Implement features and fixes in Python/Go/C++ for model-serving services like Triton, vLLM, TensorRT-LLM, and Ray Serve.
- Write tests, code comments, and design documentation, and participate in code reviews.
- Add basic metrics and dashboards, and assist with alarms and runbooks.
- Follow on-call runbooks and learn incident response through a guided rotation.
- Contribute to performance experiments, such as request batching, concurrency, and caching, with guidance.
Requirements
- BS/MS in CS, EE, or related field, or equivalent practical experience.
- Foundational knowledge in data structures, algorithms, and networked services.
- Experience with Python or Go (C++ is a plus) and Linux fundamentals.
- Familiarity with Git and CI basics.
- Exposure to containers and Kubernetes (coursework or projects are acceptable).
- Curiosity about GPU inference concepts like micro-batching, KV cache, and streaming.
Benefits
- Medical, dental, and vision insurance (100% paid by CoreWeave)
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- 401(k) with a generous employer match
- Flexible PTO
- Catered lunch each day
- Discretionary bonus
- Equity awards