Data Science Tech Lead Manager, Model Experience
$380k - $460k • New York City, NY; San Francisco, CA; Seattle, WA
Posted 1d ago
Job Location
New York City, NY; San Francisco, CA; Seattle, WA
Tech Stack
Remote Work Policy
On-site
Categories
LLM Engineer
About the job
As a senior data science leader for Model Experience, you will play an instrumental role in shaping data-informed decision-making across the organization. You will be a strategic thought partner to product, engineering, and research leaders, influencing model roadmap and launch decisions by providing insights into user experience and model performance beyond benchmark scores. In this tech lead manager role, you will also be responsible for hiring, managing, and developing a team of data scientists, setting their agenda, leading high-stakes analyses, and elevating the impact of data on model and product decisions.
Responsibilities
- Act as a strategic thought partner to product, engineering, and research leaders, helping shape model roadmap priorities, launch decisions, and major team bets.
- Set the data science agenda for Model Experience, determining areas of maximum impact on model development and launch decisions, and identifying areas for deprioritization.
- Hire, manage, and develop a team of data scientists, coaching them to transition from answering questions to shaping decisions.
- Frame critical and ambiguous questions about model quality and user experience, quantify opportunities, and provide clear recommendations, even when they challenge current directions.
- Define success metrics for Model Experience, establishing core metrics and measurement frameworks for external products and internal testing to align teams and leadership.
- Set the standard for experimentation and causal inference within Model Experience, addressing situations without clean A/B tests and personally leading high-stakes analyses.
- Translate complex analyses of model and user behavior into clear narratives and recommendations that drive decisions with senior leadership and cross-functional partners.
- Establish foundational data practices and shape the analytics infrastructure needed for rapid iteration and decision-making as models and products evolve.
Requirements
- Extensive experience in data science, including leading analytics for a product or research area at significant scale.
- 3+ years of experience managing data science teams, including setting team direction, hiring, and developing senior data scientists.
- Proven experience as a trusted partner to senior product, research, and business leaders, with examples of influencing roadmaps or high-stakes decisions.
- Strong product sense and business judgment, with the ability to frame the right questions and take positions with incomplete information.
- Deep expertise in experimental design, causal inference, and statistical modeling at high scale.
- Fluency in Python and SQL sufficient to remain hands-on and guide others' technical work.
- Clear written and verbal communication skills, including presenting to executive audiences.
- A bias for action and ability to thrive in ambiguous, fast-moving environments, driving clarity and progress.
- Experience with AI/ML products, large language models, or developer tools in the AI/ML ecosystem.
- Experience as a tech lead manager or player-coach, balancing hands-on technical work with people management.
- Experience building or scaling a data science team in a fast-growing organization.
- Experience working closely with Product or Research teams, especially on LLM or AI products.
- Comfort working with unstructured data.