Senior Backend Engineer
San Francisco • FullTime
Posted 5mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
LiteLLM is seeking a Senior Backend Engineer to join their team and focus on building robust guardrails and observability tooling for their AI Gateway. This role involves owning the backend code responsible for consistent logging of guardrail calls, surfacing errors to users, and ensuring observability instrumentation performs under high-volume traffic. Your contributions will be crucial for maintaining user trust in security and compliance features by focusing on details like latency metrics, logging traceability, and backend guardrail registration.
Responsibilities
- Build and scale the product, ensuring performance, reliability, and continuous improvement.
- Ensure all guardrail and policy enforcement calls are properly logged and traceable.
- Build and design CPU-level guardrails to cover common attacks on LLM API's, MCP servers, and Agents.
- Identify and fix silent failures in guardrail creation, registration, and policy application.
- Work with observability integrations (Datadog, Splunk, Prometheus, OpenTelemetry) for monitoring and logging.
- Enhance observability integrations to handle over 1 billion requests per month with minimal latency and no memory leaks.
- Collaborate cross-functionally on backend engineering priorities including performance, reliability, and security.
Requirements
- Bachelor's or Master's in Computer Science or related field.
- 1+ years of experience with Python and backend frameworks (e.g. FastAPI, Flask).
- Understanding of logging best practices, error handling, and secure backend development.
- Exposure to monitoring, logging, or metrics platforms (Datadog, Splunk, Prometheus, OpenTelemetry).
- Familiarity with database integration and troubleshooting (PostgreSQL, Redis, etc.).
- Driven to deliver high-quality backend code with strong guardrails, auditing, and debugging capabilities.
- Eagerness to tackle hard bugs and ensure system transparency for end users.
Benefits
- Competitive salary
- Health, dental, and vision benefits