Software Engineer - Evals
$175k - $275k • Palo Alto, CA
Posted 4d ago
Remote Work Policy
On-site
Categories
Machine Learning Engineer
About the job
SpaceXSI is building Super Intelligence to help humanity understand the universe, creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next. You will own and elevate our mission-critical research platform that directly accelerates SpaceXSI's frontier work. The evals team creates and executes instruments for measuring model capabilities and behaviors, used to clean datasets, provide feedback for experiments, and track growth throughout training. We also diagnose why agents fail and how to make improvements, managing execution infrastructure, designing data sets and grading schemes. You will design, build, and ship delightful and bulletproof systems that engineers love to use every day, valuing strong attention to detail, first principles reasoning, and great taste.
Responsibilities
- Create and execute instruments for measuring model capabilities and behaviors.
- Clean datasets using evaluation instruments.
- Provide feedback for experiments.
- Track model growth throughout training.
- Diagnose why agents fail and propose improvements.
- Manage execution infrastructure for evaluations.
- Design data sets and grading schemes.
- Design, build, and ship systems for engineers.
Requirements
- Software engineering proficiency.
- 3+ years of experience actively writing code.
- Track record of shipping high-quality projects.
- Strong sense of urgency.
- Exceptional product taste, with a sharp sense for output quality.
- Ability to translate fuzzy notions of quality, helpfulness, truthfulness, and safety into concrete, measurable criteria.
- Strong attention to detail.
Benefits
- Equity
- Comprehensive medical coverage
- Vision coverage
- Dental coverage
- Access to a 401(k) retirement plan
- Short-term disability insurance
- Long-term disability insurance
- Life insurance
- Various other discounts and perks