Applied AI Engineer - India
Salary not listed • Remote • Remote - India • Full-time
Posted 2mo ago
This job has been open for more than 60 days. It may no longer be actively hiring, so check with the company before putting a lot into an application.
About the job
Dscout is seeking an Applied AI Engineer with 2-5 years of experience to build and ship AI systems for enterprise use. This role focuses on the unique challenges of AI-native products, where reliability and performance across real-world scenarios are paramount. You will be instrumental in improving agent behavior, understanding user feedback, and ensuring AI systems deliver measurable business impact. The ideal candidate is comfortable with modern LLM-based systems and agentic workflows, possesses strong product judgment, and can effectively balance LLM approaches with traditional ML techniques.
Responsibilities
- Own the production improvement loop for agent behavior, feedback, evaluation, and business outcomes.
- Instrument agent workflows to monitor model interactions, tool use, decisions, and failures.
- Define quality standards, evaluation datasets, regression coverage, and production monitoring.
- Investigate and resolve agent performance issues across various aspects like context, knowledge, instructions, tools, and workflow design.
- Design and implement improvements to prompting, context construction, decision logic, and tool use.
- Build backend services, APIs, data models, and feedback pipelines for agent behavior.
- Run controlled experiments and staged rollouts to measure the impact of changes on quality and business results.
- Collaborate with Product, Data Science, and Sales to prioritize problems and define success metrics.
- Implement safeguards for privacy, security, reliability, human oversight, and safe operational rollout.
Requirements
- 2-5 years of software engineering experience, with hands-on experience building or operating LLM-powered features or agents in production.
- Fluency with prompting and context engineering as an engineering discipline.
- Experience building or maintaining evaluation harnesses for AI systems.
- Comfort reasoning about agent behavior across a distribution of production traffic.
- Experience running experiments (A/B, staged rollouts, production replay) to validate changes.
- A track record of shipping features used by real users and owning post-launch performance.
- High-agency mindset, comfortable investigating ambiguous performance issues.
- Comfort using AI coding tools as part of your workflow.
Benefits
- Strong and competitive compensation package with a built-in bonus and equity program.
- Incredible and progressive benefits package for you and your dependents.
- Flexible PTO.
- 15 company holidays.
- 12 weeks of paid parental leave.
- 401k match.
- Education stipend.