Applied AI Engineer
Remote • Remote, Americas • FullTime
Posted 17d ago
About the job
Monte Carlo is seeking an Applied AI Engineer to build the products that ensure the trustworthiness of enterprise AI agents. This role involves taking ambiguous problems from research and prototyping through to production, integrating them into the platform alongside engineering and data science teams. The position is critical due to the rapid growth of agent observability, moving from a roadmap item to a revenue driver.
Responsibilities
- Take open problems end-to-end, from research and prototyping through production.
- Design and ship agent-powered features like root-cause analysis and incident triage.
- Build evaluation infrastructure for feature safety, including golden datasets and scoring.
- Own retrieval and context pipelines over customer metadata and lineage.
- Instrument agent behavior in production to monitor traces, failure taxonomies, and budgets.
- Partner with data science on detection quality and experiment design.
- Partner with product management on defining agent capabilities.
- Set the technical standard for building with LLMs, including patterns and guardrails.
Requirements
- Proven experience building autonomous agents in production, not just integrating frameworks or using tools.
- Experience running agent evaluations and monitoring agents post-launch, including owning an eval framework.
- Proficiency in Python as a daily language, with a solid backend understanding.
- Ability to reason about model behavior and understand models deeply.
- Experience working from ambiguous problem statements to design experiments and take solutions to production.
- Daily use of AI tools like Claude for coding and research.
- Preference for shipping functional solutions quickly over perfecting them.
- Experience with statistics and hypothesis testing is a plus.
- Experience in the data and cloud space (Snowflake, Databricks, dbt, Airflow) is a plus.
Benefits
- Competitive compensation
- Equity
- Remote-first environment