Technical Legal Engineer - Evals
Stockholm HQ • FullTime
Posted 26d ago
Job Location
Stockholm HQ
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Legora is seeking a tech-savvy lawyer with knowledge of, or a strong interest in tech products, AI, and LLM evaluations. In this role, you will apply your legal training and practice experience to evaluate, stress-test, and improve AI systems used in legal work. You will not be expected to code, but will apply legal judgment at scale by reviewing and helping to improve AI-generated outputs, similar to how an experienced lawyer would review junior work, precedents, or research memos. This position is ideal for a lawyer who enjoys issue-spotting, quality control, and analytical rigor, and is eager to shape how legal work is performed using modern technology. You will collaborate closely with senior legal engineers, product managers, and engineers to ensure AI outputs meet professional standards for legal accuracy, reasoning, and usability.
Responsibilities
- Review AI-generated legal outputs for accuracy, completeness, legal reasoning, and jurisdictional correctness across research, drafting, and analysis use cases.
- Assess AI outputs for real-world usability and identify failure modes, edge cases, and legal ambiguities that affect product reliability.
- Design and build high-quality evaluation datasets, including benchmark cases and gold-standard legal answers.
- Define clear scoring rubrics and evaluation guidelines for consistent, objective assessment.
- Continuously refine benchmarks as products, models, and legal requirements evolve.
- Identify recurring legal errors, edge cases, and failure patterns.
- Distinguish minor drafting issues from substantive legal risks.
- Flag AI behavior that could mislead users or create downstream legal risk.
- Work closely with senior legal engineers, product managers, and engineers.
- Translate legal feedback into clear, actionable guidance for non-lawyers.
Requirements
- Tech-savvy lawyer with knowledge of, or strong interest in, tech products, AI, and LLM evaluations.
- Legal training and practice experience.
- Enjoy issue-spotting, quality control, and analytical rigor.
- Eager to shape how legal work is performed using modern technology.