Software Engineer, AI Systems (United States)
United States • Full time
Posted 19d ago
Job Location
United States
Tech Stack
Remote Work Policy
On-site
Employment Type
Full time
Categories
LLM Engineer
About the job
Haven Safety AI is building an enterprise learning intelligence layer for safety, designed to help high-risk organizations learn faster from incidents and prevent future occurrences. The platform integrates with existing EHS systems to enhance incident investigation, assessment, and learning processes. It leverages current incident evidence, company knowledge, historical cases, and an industry knowledge graph, with specialized AI agents analyzing various data points to produce traceable assessments. This role is a build-and-operate position focused on the AI and LLM engineering layer, connecting the evidence base and knowledge graph to the product experiences used by investigators and safety leaders. The work is technically demanding and operationally critical, serving enterprise customers in regulated, safety-critical industries where precision, traceability, evaluation, and human oversight are paramount.
Responsibilities
- Build and operate multi-step LLM pipelines for production reasoning systems, coordinating model calls, tool calls, graph queries, retrieval, quality gates, and specialist-agent handoffs.
- Extend Haven’s coordinated agent team and the orchestration layer for incident analysis and enterprise learning.
- Design the context layer for grounding and retrieval, encompassing Neo4j graph traversal, vector search, and hybrid retrieval.
- Build datasets, scoring, regression suites, model comparisons, human-label loops, and per-stage quality attribution for extraction and reasoning tasks.
- Implement tracing, tool-call audits, cost and latency monitoring, failure handling, and quality dashboards for production AI operations.
- Select models across providers (OpenAI, Anthropic, Google) and shape the AI roadmap in partnership with product and knowledge engineering.
Requirements
- Experience in AI or ML engineering, including shipping LLM systems for real users.
- Hands-on experience building and debugging multi-step, tool-calling workflows with frameworks like LangGraph or LangChain.
- A repeatable approach to LLM evaluation, including datasets, regression testing, LLM-as-judge techniques, or human review loops.
- Experience assembling context for LLMs and a clear point of view on retrieval strategies.
- A track record of owning systems from deployment through monitoring and incident response.
- Comfort working across model providers and explaining tradeoffs in quality, latency, cost, context, and operational risk.
- Familiarity with Neo4j and Cypher, or a comparable graph store, with the ability to quickly learn graph data modeling.
- Strong Python skills with production habits in FastAPI, asynchronous services, testing, observability, and maintainable interfaces.
Benefits
- Meaningful reasoning problems in a critical domain.
- An evaluation-first culture.
- Visible customer impact with direct feedback.
- Small team environment with high ownership and consequential technical decisions.
- Opportunity to see work reach customers quickly.