Research Engineer, LangSmith Engine
New York, NY • FullTime
Posted 1mo ago
Job Location
New York, NY
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Research Engineer
About the job
LangChain is seeking an experienced Research Engineer to join the LangSmith Engine team. This role focuses on enhancing the capabilities and efficiency of a proactive agent engineer that analyzes production traces, identifies failures, and implements fixes to prevent recurrence. You will study agent failures, build benchmarks, run experiments to improve performance, and translate successful ideas into production. This involves optimizing prompting, agent harnesses, model selection, fine-tuning, and post-training custom models, with a strong emphasis on measurable improvements to the overall agent. The role also requires an understanding of production engineering and system-level trade-offs, including cost, latency, reliability, and scalability, working closely with production engineers to ensure reliable real-world performance.
Responsibilities
- Build and maintain benchmarks and evaluations to measure the quality and efficiency of Engine agents on real-world tasks.
- Design and run experiments to improve agent performance across models, prompting, context, tools, orchestration, and agent strategies.
- Explore and implement post-training and fine-tuning techniques to improve agent capabilities, quality, or cost.
- Translate successful experiments into production improvements, measuring impact and preventing regressions.
- Help define the ML roadmap and technical direction for improving Engine agents.
- Mentor other engineers through strong technical leadership.
Requirements
- 4+ years of experience in ML/AI research or a closely related field.
- Master's or Ph.D. in a relevant scientific field.
- Hands-on experience working with LLMs and AI agents, including analyzing model behavior and improving real-world performance.
- Strong experience designing benchmarks, evaluations, and experiments for AI/ML systems.
- Strong software engineering skills with a track record of taking ideas from research prototype to measurable production impact.
- Ability to identify high-impact problems, work through ambiguity, move quickly, and communicate findings clearly.
Benefits
- Competitive compensation including base salary and variable compensation.
- Meaningful equity.
- Medical, dental, and vision coverage.
- Flexible vacation.
- 401(k) plan.
- Life insurance.
- Meals on in-office days in the US.