AI Engineer II – Trust & Explainability (AI Platform)
$104k - $178k • Remote • US Remote • FullTime
Posted 19d ago
Job Location
US Remote
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
LLM Engineer
About the job
The AI Platform team is building the shared runtime for all AI agents at MeridianLink, encompassing model gateway, orchestration, memory, and observability. This role focuses on the trust and explainability layer of this runtime, developing tracing, evaluation, and explanation capabilities. These features will enable engineers to understand multi-agent workflows and allow product teams to provide customers with appropriate explanations to build trust in agent outputs. This is a hands-on engineering role where you will write code for tracing agent actions, adopt and extend open-source frameworks for explainability and observability, and build primitives for surfacing explanations to end-users. You will work closely with a Staff engineer team lead and architects who will support your growth in this area.
Responsibilities
- Build tracing across the platform's gateway, orchestration, memory, and tool layers.
- Build the correlation that links actions across agents in a single multi-agent workflow, including handoffs, parallel branches, and retries.
- Build the developer-facing trace view that makes a full agent workflow readable.
- Instrument agents to trace conversations from the first user message through every model call, tool call, retrieval, agent handoff, and decision to the final response.
- Turn raw trace data into human-readable explanations of agent actions, dependencies, and decision paths.
- Build customer-facing explanations tailored for different user types (engineers, borrowers, loan officers).
- Integrate and extend open-source frameworks for LLM observability, tracing, and evaluation.
- Build primitives for product teams to surface agent disclosures (actions, data used, confidence, verification needs).
- Understand and implement measures for non-deterministic output quality (golden datasets, rubric/LLM-as-judge scoring, regression baselines).
- Build evaluation checks that run in CI for model swaps, prompt changes, or tool changes.
- Implement tests for common LLM application attack patterns (prompt injection, jailbreaks, tool misuse, data exfiltration).
- Build tests to ensure tenant isolation and prevent data exfiltration between customers.
- Contribute to the shared guardrail layer for agent safety.
Requirements
- Real hands-on experience with LLM-based systems.
- Strong interest in problems of trust, tracing, and evaluation.
- Ability to handle a broad range of work independently, from bug fixes to feature development.
- Ability to write code that other engineers trust.
- Ability to ask good questions and help surrounding engineers.
- Active, daily user of AI-assisted development tools.
- Ability to design components within a well-defined scope and escalate complex system design questions.
- Ability to participate in code review and provide constructive feedback.
- Ability to surface blockers proactively.
- Ability to write tests that cover shipped functionality.
- Ability to monitor and respond to issues with own work.
- Ability to document decisions and implementation details.
- Familiarity with the open-source landscape for LLM observability, tracing, and evaluation.
- Understanding of tenant isolation as a platform guarantee.
- Understanding of common attack patterns against LLM applications.