Software Engineer - Model Performance Systems
Remote • San Francisco • FullTime
Posted 8mo ago
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Baseten is seeking Software Engineers to join their team in a specialized, high-impact role at the intersection of high-performance computing (HPC) and Large Language Model (LLM) engineering. This position involves not only building automated performance monitoring and diagnostic tools for next-generation AI infrastructure but also defining the roadmap, driving key technical decisions, and taking full ownership of the future direction of this work. The role offers the opportunity to shape the platform that engineers use to deploy cutting-edge AI models into production.
Responsibilities
- Evaluate, run, and automate LLM quality benchmarks and custom performance suites.
- Develop and maintain internal GPU-enabled development environments for model experimentation.
- Build and contribute to open-source tools for automating model evaluation, benchmarking, and analysis.
- Use profilers to collect performance data, identify bottlenecks, and debug compute/networking stacks.
- Develop real-time dashboards and alerts for system health and performance monitoring.
- Automate performance testing via CI/CD pipelines to detect regressions.
- Build tools to identify optimal configurations balancing latency, cost, and quality for models.
Requirements
- Strong technical depth in systems and hardware, including GPU memory subsystems, InfiniBand, and cluster data movement.
- An automation mindset with a passion for scripting and stress-testing systems.
- Mathematical curiosity regarding Transformers and their computational requirements.
- Familiarity with Python and eagerness to master the NVIDIA software stack.
- C++ familiarity is a plus.
- Strong communication skills to drive cross-team efforts.
- Ability to navigate vague requirements.
- Mentorship capabilities.
Benefits
- Competitive compensation
- Meaningful equity
- 100% coverage of medical, dental, and vision insurance for employee and dependents
- Flexible PTO policy
- Company-wide Winter Break
- Paid parental leave
- Fertility and family-building stipend
- Company-facilitated 401(k)
- Exposure to a variety of ML startups