Staff Software Engineer, Observability & Profiling
London, UK
Posted 12d ago
Remote Work Policy
On-site
Categories
Applied AI Engineer
About the job
Anthropic is seeking Software Engineers to join its Observability team within the Infrastructure organization. This team is responsible for the monitoring and telemetry infrastructure used by all engineers and researchers at Anthropic, encompassing metrics, logging pipelines, distributed tracing, profiling, error analytics, alerting, and actionable dashboards. As Anthropic's infrastructure scales across massive GPU, TPU, and Trainium clusters, the complexity of operational data is increasing significantly. The team is building next-generation observability systems, including high-throughput telemetry pipelines, fleet-wide continuous profiling, eBPF-based tracing, and agentic diagnostic tools, to enable faster issue detection and resolution.
Responsibilities
- Design and build scalable telemetry ingest and storage pipelines for metrics, logs, traces, and error data.
- Build observability solutions for deep, low-overhead system visibility across the fleet.
- Own and evolve core observability platforms, driving improvements in reliability, cost, and scalability.
- Build instrumentation libraries, SDKs, and eBPF-based auto-instrumentation for high-quality telemetry.
- Reduce mean time to detection and resolution through cross-signal correlation and AI-assisted diagnostics.
- Drive fleet-wide efficiency by translating profiling and utilization telemetry into actionable optimization insights.
- Partner with Research, Inference, Product, and Infrastructure teams to meet observability needs.
Requirements
- Hands-on experience building and operating large-scale observability or monitoring infrastructure.
- Deep, hands-on experience with observability signals end-to-end (instrumentation to query and analysis).
- Understanding of high-throughput telemetry pipelines and their tradeoffs.
- Comfortable digging below the application layer into kernel, network stack, or hardware.
- Excellent communication skills and ability to partner with internal teams.
- Excited about building foundational infrastructure and comfortable navigating ambiguous technical challenges.
Benefits
- Annual compensation range: £325,000 — £390,000 GBP
- Hybrid work policy (at least 25% in office)