A/B experimentation Jobs

Glean is building a world-class Data Organization composed of product data science, applied science, data engineering, and business intelligence groups. This is an applied science role based in our San Francisco office. You will collaborate with product data science and engineering teams to identify techniques, tooling, and process improvements in online A/B experimentation to assist rigorous decision-making across all relevant product domains. You will develop and maintain our A/B experimentation platform based on stakeholder feedback, and write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management, and design teams. You will also conduct end-to-end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results. You will break down end-to-end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design. You will design, develop, and own best practices, tools, and processes across various evaluation problems, e.g., query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, and industry benchmarking.

$180k - $330k

Glean is building a world-class Data Organization composed of product data science, applied science, data engineering, and business intelligence groups. This is an applied science role based in our San Francisco office. You will collaborate with product data science and engineering teams to identify techniques, tooling, and process improvements in online A/B experimentation to assist rigorous decision-making across all relevant product domains. You will develop and maintain our A/B experimentation platform based on stakeholder feedback, and write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management, and design teams. You will also conduct end-to-end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results. You will break down end-to-end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design. You will design, develop, and own best practices, tools, and processes across various evaluation problems, e.g., query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, and industry benchmarking.

$180k - $330k