Staff+ Software Engineer, Inference Velocity
$405k - $485k • Remote • Remote-Friendly (Travel-Required) | San Francisco, CA | Seattle, WA | New York City, NY
Posted 1mo ago
About the job
Anthropic's Inference organization is responsible for delivering Claude to millions of users and enterprise customers with the speed, reliability, and efficiency that frontier AI demands. We are seeking a Staff engineer to lead the Inference Developer Productivity team. This team is dedicated to making every engineer in the organization significantly more effective at building, testing, and shipping inference software. This senior individual contributor role involves broad technical ownership, where you will set the technical direction for the team's toolchains, workflows, and feedback loops. You will be responsible for making critical decisions on architecture, prioritization, and trade-offs across heterogeneous accelerator platforms. You will partner closely with the Engineering Manager, who handles hiring and people development, while you drive the technical roadmap and execution. Collaboration with Anthropic's central Infrastructure organization is also key to ensure Inference's multi-accelerator reality is well-supported without duplicating efforts.
Responsibilities
- Set technical direction for Inference Developer Productivity, owning the architecture and roadmap for toolchains, dev environments, and CI/CD across GPU (CUDA), TPU, and Trainium platforms.
- Own accelerator toolchain management, ensuring compilers, drivers, libraries, and frameworks are current, compatible, and well-tested.
- Design and build infrastructure for efficient accelerator usage during development, including devbox environments, pre- and post-land validation automation, and shared tooling.
- Define and instrument productivity metrics for the Inference org, building dashboards and alerting to surface regressions and drive resolution.
- Proactively identify and eliminate bottlenecks, toil, and friction across Inference engineering workflows by designing and building systems.
- Act as the technical counterpart to Anthropic's central Infrastructure org, aligning on shared developer productivity initiatives and contributing Inference-specific requirements.
- Mentor engineers on the team through design and code reviews, and direct collaboration, raising the technical bar.
Requirements
- 8+ years of software engineering experience, with significant time as a technical lead or anchor on an infrastructure, platform, or developer productivity team.
- Deep background in systems engineering, build/test infrastructure, or ML infrastructure, with hands-on experience in toolchain issues, CI/CD pipelines, and developer workflow optimization.
- Experience owning toolchains or development environments for compute-intensive workloads (ML training or inference, HPC, large-scale distributed systems).
- In-depth knowledge of at least one accelerator ecosystem (CUDA/GPU, TPU, or Trainium/AWS Neuron) and a willingness to learn others.
- Proven track record of defining and using engineering metrics to drive improvement, including building dashboards, setting SLOs on developer workflows, or leading initiatives that improved engineering velocity.
- Experience driving technical alignment across organizational boundaries.
- Strong written and verbal communication skills, with the ability to influence technical direction without formal authority.