Machine Learning Engineer, Core Experimentation
Remote • Seattle • FullTime
Posted 15h ago
About the job
We are seeking a Machine Learning Engineer to spearhead the technical direction for ML-powered experimentation and insights. This role involves building production systems that leverage privacy-protected data to generate evidence-backed insights and inform decision-making, helping teams identify promising ideas for live testing. This is an end-to-end, 0-to-1 position requiring work across ML modeling, retrieval and LLM systems, statistical methods, simulation, data pipelines, backend services, and user-facing product experiences. The core challenge lies in ensuring that insights and predictions are traceable, calibrated, useful, and safe enough to influence critical product decisions, with a focus on making uncertainty explicit and learning from outcomes.
Responsibilities
- Set and execute the technical roadmap for Generative Insights and Predictive Experimentation.
- Build cross-experiment learning systems to synthesize historical experiments and detect recurring effects.
- Develop predictive models and simulation workflows to estimate impact and uncertainty before live experiments.
- Create high-quality datasets and feature/retrieval pipelines with strong lineage and quality controls.
- Establish rigorous evaluation through offline benchmarks, backtests, and monitoring.
- Develop durable product, API, and agent workflows for insight-to-action processes.
- Partner with data science and product teams on experiment design and causal inference.
- Build reliable services and intuitive workflows for sophisticated ML capabilities.
- Provide technical leadership and raise the bar for production ML quality.
Requirements
- Experience leading ambiguous 0-to-1 production ML products where success is measured by real-world decisions.
- Strong hands-on experience across the ML lifecycle: dataset design, training, evaluation, deployment, monitoring, and iteration.
- Depth in LLM and retrieval systems, ranking or recommendation, forecasting or anomaly detection, causal ML or experiment analysis, or simulation.
- Strong software engineering fundamentals and ability to build production systems in Python.
- Comfort working across data, backend, and platform boundaries.
- Strong grounding in machine learning, statistics, computer science, or a related field.
- Understanding of experimentation and statistical reasoning.