Applied AI Engineer - Agent
$250k - $300k • Remote • New York • FullTime
Posted 1y ago
Job Location
New York
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
We are seeking an Applied AI Engineer to advance the capabilities of our Cofounder agent. This role involves owning core backend systems and applied LLM work, focusing on enhancing agent reliability and autonomy. You will be responsible for building evaluation pipelines and implementing techniques that demonstrably improve agent performance. This is a hands-on position with significant ownership, covering the entire research-to-production lifecycle, including prototyping, instrumentation, evaluation, and deployment of user-facing improvements.
Responsibilities
- Design and implement agent improvements, including prompting strategies, tool selection, action planning, memory usage, safety features, and recovery mechanisms.
- Build robust evaluation pipelines for the agent, encompassing offline evaluations (golden tasks, regression suites, behavior tests) and online metrics (latency, success rate, fallout, cost efficiency), along with experimentation frameworks.
- Productionize applied LLM techniques such as function/tool-calling orchestration, self-reflection, retrieval/RAG, multi-agent handoffs, caching/embedding strategies, and hallucination reduction.
- Improve core backend systems, focusing on reliable job orchestration, retries, idempotency, auditability, scalable memory and context routing, data pipelines, and observability for agent actions.
- Collaborate with product and infrastructure teams to define success metrics and execute rapid, safe iterations.
- Write clean, well-tested code and document design decisions and operational procedures.
Requirements
- 4+ years of backend engineering experience, with a preference for Python.
- Hands-on experience with LLMs, including prompt engineering, function-calling, retrieval, embeddings, and evaluation design.
- Proven experience shipping LLM features to production.
- Track record of building evaluation harnesses and using them to drive improvements.
- Solid understanding of distributed systems fundamentals, including concurrency, reliability, performance, data modeling, and lifecycle management.
- Experience with pragmatic experimentation, from hypothesis to measured improvement and rollout.
- Excellent debugging and instrumentation skills.
Benefits
- Competitive salary
- Meaningful equity
- Comprehensive benefits
- Flexible work setup