Senior/Staff FDE - CUA

About the job

Responsibilities

• Design and build task environments, datasets, and evaluation workflows for computer-using agents across various software interfaces.
• Translate customer objectives and agent failure modes into representative, multi-step tasks with clear success criteria.
• Develop data generation, validation, and quality assurance pipelines for multimodal and agentic training and evaluation data.
• Build automated evaluators, checks, and measurement frameworks to assess task completion, correctness, robustness, efficiency, and adherence to requirements.
• Diagnose agent failures in planning, tool use, perception, state management, and UI interaction, and use findings to improve tasks, data, and evaluations.
• Design and execute experiments to measure the impact of data, task design, and evaluation changes on agent performance.
• Deliver production-grade task suites, datasets, and evaluation assets for training, benchmarking, and improving computer-use agents.
• Lead technical workstreams from solution design to production delivery, managing ambiguity and making sound technical decisions.
• Build, refine, and iterate on solutions based on customer needs and feedback to ensure tangible value.
• Rapidly prototype and productionize solutions involving models, agent frameworks, APIs, browser/desktop environments, and custom applications.
• Communicate technical tradeoffs, experimental results, and recommendations to technical and cross-functional stakeholders.
• Act as a trusted technical partner to customers and internal teams, resolving blockers and driving alignment.
• Identify recurring patterns in customer engagements to create reusable task frameworks, evaluators, tooling, and best practices.
• Define and improve technical standards for agent task design, environment reliability, evaluation, and delivery.
• Collaborate with DaaS Engineering, Research, and Product teams to influence platform and product capabilities based on customer needs.
• Lead technical design reviews, share expertise, and mentor other engineers.
• Stay current with emerging agentic AI, computer-use, evaluation, and data curation techniques and assess their applicability.

Requirements

• 5+ years of experience in machine learning engineering, software engineering, applied AI, forward deployed engineering, solutions engineering, or a similar technical role.
• Strong Python skills and experience building reliable production software, data, or ML systems.
• Hands-on experience building, evaluating, or deploying LLM-based or agentic systems, including computer-use agents (CUA).
• Strong understanding of experimentation and evaluation, including LLM-as-a-judge, defining metrics, and using empirical results to guide technical decisions.
• Experience designing task environments, datasets, and verifiers for agents, including reward & verifier design (RL with verifiable rewards).
• Experience building or working with APIs, automation, web applications, browser-based systems, desktop applications, or developer tools.
• Experience manipulating, analyzing, and validating large or complex datasets using Python and the modern GenAI/LLM stack.
• Demonstrated ability to take ambiguous technical problems from problem definition through implementation and delivery.
• Strong technical communication skills and experience working directly with customers or cross-functional stakeholders.
• Demonstrated experience setting technical direction, creating reusable approaches across projects, and influencing broader engineering or product decisions.
• Experience developing agent benchmarks, task suites, or simulators.
• Experience with multimodal models, visual grounding, or evaluating agents that interact with visual interfaces.