New: upload your resume and get matched to AI jobs. Free to start →

ML Engineer

$200k - $250k • San Francisco • Full-time

Posted 9d ago

Remote Work Policy

On-site

Employment Type

Full-time

Categories

LLM Engineer

About the job

LiteLLM is seeking an ML Engineer to join their team and contribute to the development of their AI Gateway. This role involves ensuring LiteLLM unifies the format for calling LLM APIs, specifically within the OpenAI and Anthropic specifications. You will be responsible for writing transformations to adapt API requests to various LLM provider formats and building provider-agnostic unification functionality. This is a critical role where you will work directly with the CEO and CTO on significant projects, including system design for gateway-level features and enhancing the developer experience through LiteLLM's Python SDK. You will also contribute to building across LLMs, MCPs, and Agents by maintaining an excellent LLM interoperability layer.

Responsibilities

  • Unify LLM API formats within the OpenAI and Anthropic specifications.
  • Develop transformations to convert API requests for various LLM providers.
  • Build provider-agnostic unification functionality for features like session management.
  • Design systems to support provider-level features at the gateway level (e.g., skills, compaction, batches).
  • Enhance the developer experience for users of LiteLLM's Python SDK.
  • Maintain an LLM interoperability layer supporting various MCP authentication flows and agent use cases.

Requirements

  • 1-2 years of backend/full-stack experience with production systems.
  • Passion for open source and user engagement.
  • Experience working with the OpenAI API, understanding its nuances.
  • Strong work ethic and ability to thrive in small teams.
  • Eagerness to engage with users and solve problems.

Benefits

  • Competitive salary
  • Equity
  • Health insurance
  • Dental insurance
  • Vision insurance