Software Engineer, Inference
$120k - $160k • Remote • Sunnyvale, CA / Bellevue, WA
Posted 14h ago
Job Location
Sunnyvale, CA / Bellevue, WA
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Infrastructure Engineer
About the job
CoreWeave is seeking a Junior Software Engineer to join their Inference team. In this role, you will be responsible for the end-to-end ownership of a production service, including feature development, maintenance of dashboards and alarms, and operational improvements. You will collaborate with cross-functional teams to enhance key service level indicators and contribute to the development of a cutting-edge inference platform that powers generative AI and real-time applications at scale.
Responsibilities
- Own one model-serving service end-to-end: design, implementation, tests, deployment, and operations.
- Add and tune observability (dashboards, alerts, traces); automate runbook steps to reduce toil.
- Improve latency/throughput via micro-batching, concurrency tuning, and caching.
- Participate in on-call rotations and lead post-incident action items for your service.
- Review peer pull requests and provide design feedback in team reviews.
Requirements
- 0-3 years of software engineering experience building cloud/distributed services in production.
- Proficiency in Python or Go.
- Familiarity with gRPC/HTTP APIs and containerized deployments.
- Working knowledge of Kubernetes (or Slurm/Ray) and CI/CD pipelines.
- Experience with metrics/alerting (Prometheus/Grafana/OpenTelemetry) and structured logging.
- Basic understanding of GPU inference concepts (KV caching, mixed precision, streaming tokens).
Benefits
- 100% employer-paid medical, dental, and vision insurance
- 401(k) with a generous match
- Flexible PTO
- Remote-friendly culture
- Company-paid Life Insurance
- Voluntary supplemental life insurance
- Short and long-term disability insurance
- Flexible Spending Account
- Health Savings Account
- Tuition Reimbursement
- Ability to Participate in Employee Stock Purchase Program (ESPP)
- Mental Wellness Benefits through Spring Health
- Family-Forming support provided by Carrot
- Paid Parental Leave
- Flexible, full-service childcare support with Kinside
- Catered lunch each day in office and data center locations
- Casual work environment
- Work culture focused on innovative disruption