Machine Learning Engineer, Frontier Data Products
San Francisco • FullTime
Posted 3mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Mercor is seeking a Machine Learning Engineer to build and enhance the ML systems powering their Frontier Data Products. This role is crucial for scoring, validating, and improving complex work products where correctness is nuanced and labels are imperfect. You will design evaluation frameworks for ambiguous tasks, build feedback loops to improve models, and own production ML behavior end-to-end, considering tradeoffs in precision, recall, latency, and cost. This is an applied ML product engineering role with real production constraints, focusing on shipping ML systems that directly impact product and business metrics.
Responsibilities
- Build ML systems to score, validate, and improve complex work products with imperfect labels.
- Design evaluation frameworks for ambiguous tasks with partial or disputed ground truth.
- Develop feedback loops to translate review, disagreement, and adjudication into model improvements.
- Own end-to-end production ML behavior, including precision/recall tradeoffs, regression detection, drift, latency, cost, and explainability.
- Improve model quality using various techniques like prompting, fine-tuning, retrieval, active learning, heuristics, and error analysis.
- Collaborate with backend engineers to integrate inference into production workflows while maintaining debuggability and human oversight.
Requirements
- Proven track record of shipping ML systems that improved a real product, workflow, or business metric.
- Strong understanding of model quality, evaluation design, error analysis, and production failure modes.
- Comfort operating in ambiguous problem spaces with imperfect labels and evolving correctness.
- Sound judgment in selecting appropriate ML techniques (prompting, fine-tuning, heuristics, retrieval, human review, etc.).
- Solid engineering fundamentals across the full ML stack.
- Familiarity with LLM applications, model-assisted workflows, evaluation frameworks, or human-in-the-loop ML is a strong plus.
- Ability to default to simple, inspectable ML systems that improve quickly and fail understandably.
- Comfort with ambiguity and making reasonable bets with incomplete information.
- Focus on real-world system output over benchmarks.
Benefits
- Bi-annual performance bonus structure.
- Generous equity grant vested over 4 years.
- Up to $15k Relocation bonus.
- $10K housing bonus (if living within 0.5 miles of office).
- $1.5K monthly stipend for meals.
- Free Equinox membership.
- $200 monthly laundry reimbursement.
- $200 monthly personal wellness reimbursement.
- Health, Dental, Vision insurance.