human evaluation Jobs

Innodata is seeking a Research Scientist to focus on the challenging area of video and multimodal AI models. This role involves partnering directly with customers and frontier labs to advance the science behind video understanding, video-language, and video-generation models. You will be instrumental in defining how data is designed, structured, and evaluated for these cutting-edge AI systems, ensuring that the data collected and the evaluations performed drive meaningful progress. Your work will directly influence the direction of research and development in this rapidly evolving field.

$160k - $185k

Figma is seeking a Director, Research - AI Evals to lead the measurement of quality for Figma's AI-powered experiences. This role is crucial as Figma integrates more AI capabilities, requiring a rigorous approach to defining and assessing "good" AI features. You will establish frameworks, quality standards, and trusted signals that guide product teams in their AI feature development and release decisions. The ideal candidate possesses deep, hands-on experience in evaluating AI/LLM products, combining human and automated methods, with strong product intuition and communication skills to ensure evaluation is impactful. You will collaborate with Product, Design, Engineering, and Data Science teams, influencing AI shipping decisions that affect millions of users.

$258k - $348k