Senior Software Engineer, AI Platform
$193k - $290k • Remote • New York • FullTime
Posted 14d ago
Job Location
New York
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Harvey is transforming legal and professional services by combining frontier agentic AI with an enterprise-grade platform and deep domain expertise. This is a unique opportunity to help build a generational company at a critical inflection point, with strong product-market fit and investor support, we are scaling rapidly and defining a new category. The AI Platform team builds the foundational model layer and agent infrastructure that underpins all of Harvey's products and agents, ensuring high-quality AI output in complex domains like legal services where accuracy is paramount. This team is early and has significant opportunities to shape the future of AI at Harvey, focusing on areas like model routing, agent architecture, context management, and evaluation.
Responsibilities
- Design and build platform-level systems and abstractions to enhance all of Harvey's agentic products.
- Develop and maintain infrastructure for model integration, routing, and evaluation to optimize foundation model deployment.
- Create evaluation frameworks and tooling to enable teams to systematically improve AI quality.
- Collaborate with product engineering, PMs, and design to launch advanced AI products.
- Evaluate, prototype, and integrate emerging advancements in AI and agentic systems.
Requirements
- 5+ years of backend systems development experience, with at least 1 year in AI/ML engineering.
- Experience building and deploying multi-model or multi-provider AI systems in production.
- Familiarity with context management, session state, or memory systems in AI or distributed systems.
- Proven track record of building adopted internal platforms, SDKs, or shared infrastructure.
- Strong understanding of developer experience and its importance.
- Proficiency in designing abstractions and a pragmatic approach to incremental shipping.
- Enthusiasm for agentic AI and the challenges of building reliable autonomous systems.
- Demonstrated ability to take ownership and navigate ambiguity.
- Bonus: Experience with evaluation frameworks, agent/function-calling architectures, legal domains, or early-stage startups.