Staff Software Engineer, Inference
€295k - €355k • Dublin, IE
Posted 16d ago
Job Location
Dublin, IE
Tech Stack
Remote Work Policy
On-site
Categories
AI Infrastructure Engineer
About the job
Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.
Responsibilities
- Identify and address key infrastructure blockers to serve Claude to millions of users.
- Enable breakthrough AI research by providing high-performance inference infrastructure.
- Design intelligent routing algorithms to optimize request distribution across thousands of accelerators.
- Implement autoscaling for the compute fleet to dynamically match supply with demand.
- Build production-grade deployment pipelines for releasing new models.
- Integrate new AI accelerator platforms.
- Contribute to new inference features such as structured sampling and prompt caching.
- Support inference for new model architectures.
- Analyze observability data to tune performance based on production workloads.
- Manage multi-region deployments and geographic routing for global customers.
Requirements
- Significant software engineering experience, particularly with distributed systems.
- Familiarity with performance optimization.
- Familiarity with distributed systems.
- Familiarity with large-scale service orchestration.
- Familiarity with intelligent request routing.
- Familiarity with LLM inference optimization, batching strategies, and multi-accelerator deployments is highly encouraged.
- Experience with high-performance, large-scale distributed systems.
- Experience implementing and deploying machine learning systems at scale.
- Experience with load balancing, request routing, or traffic management systems.
- Experience with LLM inference optimization, batching, and caching strategies.
- Experience with Kubernetes and cloud infrastructure (AWS, GCP).
- Proficiency in Python or Rust.
- Results-oriented with a bias towards flexibility and impact.
- Ability to pick up slack, even if it goes outside your job description.
- Desire to learn more about machine learning systems and infrastructure.
- Thrive in environments where technical excellence directly drives business results and research breakthroughs.
- Care about the societal impacts of your work.
Benefits
- Annual compensation range: €295.000 — €355.000 EUR
- Visa sponsorship available