Senior Software Engineer I, Inference
$139k - $204k • Remote • Sunnyvale, CA / Bellevue, WA
Posted 5d ago
Job Location
Sunnyvale, CA / Bellevue, WA
Tech Stack
Remote Work Policy
Fully remote
Categories
LLM Engineer
About the job
CoreWeave is seeking a Senior Software Engineer I, Inference to join their AI Cloud team. This role involves owning specific areas of the inference platform, leading design efforts, and improving latency, throughput, and reliability. You will collaborate with product, orchestration, and hardware teams to enhance a Kubernetes-native inference platform and meet strict service level agreements at scale. The position requires a strong focus on engineering standards, incident management, and mentoring junior engineers.
Responsibilities
- Lead design reviews and drive architecture within the team.
- Decompose multi-service work into clear milestones.
- Define and own service level indicators (SLIs) and objectives (SLOs).
- Ensure post-incident actions are implemented and reliability improves.
- Implement and quantify the impact of advanced inference optimizations.
- Strengthen incident posture through capacity planning, autoscaling, and traffic-shifting strategies.
- Mentor junior engineers and review cross-team designs.
- Elevate coding and testing standards across the team.
Requirements
- 3-8 years of industry experience building distributed systems or cloud services.
- Strong coding skills in Python or Go, with C++ as a plus.
- Deep familiarity with networked systems and performance.
- Hands-on experience with Kubernetes at production scale.
- Experience with CI/CD and observability stacks (Prometheus, Grafana, OpenTelemetry).
- Practical knowledge of inference internals like batching, caching, mixed precision, and streaming token delivery.
- Proven track record of improving tail latency and service reliability.
- Bachelor's or Master's degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience.
Benefits
- Discretionary bonus
- Equity awards
- Comprehensive benefits program
- Medical, dental, and vision insurance (100% paid by CoreWeave)