AI Infrastructure Engineer, pAGI
$266k - $500k • Remote • San Francisco • FullTime
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
We are seeking an AI Systems Engineer to join our pAGI Infra team, which builds and operates the systems essential for large-scale model training and evaluation. This role involves owning projects from identifying bottlenecks and designing solutions to deployment and operation, focusing on distributed systems engineering, performance optimization, and collaboration with researchers. You will directly contribute to improving the speed and reliability of research by potentially building shared grading services, optimizing resource allocation, or integrating new training stacks into production.
Responsibilities
- Build and operate infrastructure for large-scale training and evaluation, enhancing reliability, throughput, and resource efficiency.
- Develop shared inference and grading platforms with automated capacity management, health monitoring, and performance visibility.
- Improve compute scheduling and resource allocation to minimize idle GPU time and ensure quick workload recovery from failures.
- Diagnose performance bottlenecks across training, inference, and orchestration, collaborating with teams to boost end-to-end performance.
- Create self-service tools, automated validation, and observability features to help researchers launch experiments, diagnose issues, and compare results with less manual effort.
Requirements
- Strong software engineering fundamentals and experience building or operating large-scale distributed systems.
- Experience in ML infrastructure, inference systems, GPU performance, or infrastructure tooling.
- Highly self-motivated with the ability to take ownership of open-ended problems.
- Proficiency in debugging across system boundaries and using measurements to drive performance and reliability improvements.