AI Engineer, Model Quality and Performance
Remote • Sunnyvale, CA • FullTime
Posted 5mo ago
About the job
Cerebras Systems is seeking an AI Engineer focused on Model Quality and Performance to define and measure the quality of AI models for Cerebras' inference offerings. This role involves building AI-driven systems to assess model performance at scale and translate these insights into actionable artifacts for customers and the product team. You will leverage AI agents to create custom evaluation suites tailored to specific customer use cases, extract representative test data, automate release qualification processes, and develop performance datasets and benchmarking workflows. The ideal candidate will have a strong inclination towards automating tasks using AI agents and will collaborate closely with engineering, product, and customer-facing teams.
Responsibilities
- Design evaluation suites with AI agents for model releases, including advanced, basic, long-context, and customer-specific evaluations.
- Build custom evaluations for target customers by orchestrating AI agents to mine workloads and synthesize representative evaluation sets.
- Automate end-to-end evaluation execution using AI-driven pipelines on standard tooling like Docker, Git, and CI.
- Build automations to forecast and benchmark model performance on Cerebras hardware for top customers.
- Develop product-quality tooling to synthesize quality and performance data into an easily usable view.
Requirements
- Experience building AI agents and shipping real systems where AI agents act as a force multiplier.
- Strong mathematical and statistical background.
- Comfort with Docker, Git, and standard automation tools.
- Experience in tooling design, having shipped user-friendly products.
- Experience building AI agents.
Benefits
- Build a breakthrough AI platform beyond the constraints of the GPU.
- Publish and open source cutting-edge AI research.
- Work on one of the fastest AI supercomputers in the world.
- Job stability with startup vitality.
- Simple, non-corporate work culture that respects individual beliefs.