Head of Engineering

$200k - $300k • Remote • San Francisco • FullTime

Posted 1h ago

Remote Work Policy

Fully remote

Employment Type

FullTime

Categories

Machine Learning Engineer

About the job

LlamaIndex is seeking a Head of Engineering to lead and develop their engineering team in building software that makes enterprise data useful for AI applications. This is a hands-on role where you will own engineering execution, technical direction, hiring, and team development. You will collaborate closely with company leadership and product teams to define product strategy and delivery. The ideal candidate has shipped and operated B2B API products, with experience in both model serving and model training, from experimentation and evaluation to deployment and production operations.

Responsibilities

  • Lead and develop the engineering team, establishing clear priorities, ownership, and accountability.
  • Partner with company leadership to translate customer needs into a roadmap and deliver software predictably.
  • Be hands-on with architecture, code reviews, production debugging, and implementation of critical changes.
  • Set technical direction for application services, APIs, data pipelines, and ML infrastructure, balancing immediate needs with long-term maintainability.
  • Collaborate with ML engineers and researchers on model training, fine-tuning, evaluation, and deployment.
  • Guide model serving decisions regarding latency, throughput, GPU utilization, capacity, reliability, and inference cost.
  • Improve engineering practices for testing, observability, security, incident response, and releases.
  • Hire and coach engineers and technical leaders, fostering a culture of direct communication, customer focus, and accountability.

Requirements

  • Experience leading engineering teams in B2B SaaS, delivering and operating customer-facing products.
  • Experience leading multiple engineering teams or technical domains comparable to a 20-30 person organization.
  • Strong, current hands-on engineering skills, including writing production code, reviewing architecture, and diagnosing technical problems.
  • Practical familiarity with model serving and model training workflows, including evaluation, deployment, and production operations.
  • Understanding of tradeoffs between model quality, latency, throughput, GPU resources, reliability, and cost.
  • Strong background in backend systems, APIs, distributed systems, and cloud infrastructure.
  • Track record of hiring and developing engineers, setting clear expectations, and providing constructive feedback.
  • Good product judgment, balancing speed with correctness, reliability, and security.
  • Clear written and verbal communication skills with technical teams, customers, and leadership.

About llamainndex ai

Get new AI jobs in your inbox

A weekly digest of the newest AI engineering roles.

© 2026 AI Job Board. All rights reserved.