Principal AI Engineer - Context - Agents and Context
€80k - €127k • Spain • Full-time
Posted 1mo ago
About the job
The Context Engine team builds the knowledge layer that AI agents use to work with enterprise data in Elasticsearch. This layer extracts knowledge from any data sources into a structured AI Index, serves it to agents through public APIs and framework integrations, and closes the loop with agent traces to improve knowledge from real usage. As a Principal AI Engineer, you will own the end-to-end improvement loop of this product, focusing on how agents, automations, and skills behave in production, how they are observed and evaluated, and how changes are shipped safely. This is a hybrid role at the intersection of engineering, data science, and product, where you will write production code, design evaluation and telemetry, and set the technical bar for agentic behavior iteration. You will collaborate with data scientists, backend engineers, product, and UX, and your work will directly impact what customers build on top of Elastic.
Responsibilities
- Own the production improvement loop for Context Engine, understanding extraction automations, retrieval tools, and memory behavior through evaluations and telemetry.
- Define safe iteration processes for agents and skills, including versioning, rollout, regression coverage, staged evaluation, and guardrails.
- Design telemetry for AI systems to enable data-informed engineering and product decisions, capturing agent traces, tool calls, and knowledge retrieval.
- Partner with the data science team on evaluation strategy, including golden datasets and evaluators for quality, latency, and cost.
- Raise the technical bar for the team by reviewing designs and PRs, mentoring engineers, and writing technical proposals.
Requirements
- 10+ years of software engineering experience, with recent years focused on shipping and operating AI-driven products in production.
- Track record of eval-driven product improvement, diagnosing agent or LLM behavior and implementing fixes.
- Direct experience building agents with state and memory, and iterating on prompts, skills, and tool behavior safely in production.
- Familiarity with MCP, including exposing public MCP servers and tools.
- Experience designing and using telemetry for AI systems to make engineering and product decisions.
- Experience running product experiments end-to-end, including instrumentation and result interpretation.
- Experience building products with public APIs and evolving data models while maintaining external contracts.
- Strong backend engineering skills in Python or TypeScript, including APIs, stateful workflows, data pipelines, and production services.
- Comfort collaborating with data scientists, engineers, and product managers, translating between measurement and shipping, and communicating trade-offs.
- Experience with agent frameworks such as LangGraph, CrewAI, Claude Agent SDK, or similar (bonus).
- Agentic retrieval experience, including knowledge representation (bonus).
- Practical Elasticsearch experience (bonus).