Staff AI Scientist
$220k - $260k • Remote • Palo Alto • FullTime
Posted 1mo ago
Job Location
Palo Alto
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Fiddler is building trust into AI, especially with the rise of Generative AI and Agents. Our platform helps organizations deploy trustworthy and transparent AI solutions by monitoring, evaluating, securing, analyzing, and improving AI applications. We partner with AI-first organizations to establish responsible AI practices, enabling engineering teams and business stakeholders to understand AI outcomes. Joining Fiddler means making an impact by ensuring AI applications at production scale have operational transparency and security. This is an opportunity to be a trailblazer in the rapidly innovating AI and ML industry, contributing to AI Observability.
Responsibilities
- Lead applied research and development for models and datasets powering Fiddler's Trust Service and guardrail classifiers for LLM and agentic applications.
- Partner with engineering, Product, and Customer Success teams to support customer AI observability journeys and ensure value realization.
- Design, train, and deploy production classifiers for safety, security, and quality detection (e.g., prompt injection, jailbreaks, PII, hallucination, faithfulness) under strict latency and cost constraints.
- Lead the development of synthetic and adversarial dataset pipelines for generating, filtering, and validating data to expose model failure modes.
- Drive the technical direction of generative insights, the LLM- and agent-powered analysis layer for diagnosing AI application issues.
- Contribute to evaluation and experimentation infrastructure for measuring model quality, regression, and drift.
- Explore reinforcement learning and preference-based methods for model improvement.
- Collaborate with Backend and Platform engineers to transition research prototypes to hardened, scaled, observable services.
- Translate enterprise customer needs into research problems and research results into product features.
- Mentor AI Scientists, raise the technical bar through code and design reviews, and represent Fiddler externally through publications or talks.
Requirements
- 7+ years of applied AI experience with a track record of taking models from research to production.
- Experience in LLM or Agentic Evals and Guardrailing.
- Deep expertise in training and fine-tuning classifier models, including modern encoder architectures and LLM-as-classifier approaches.
- Hands-on experience with dataset development, including sourcing, labeling, synthetic generation, adversarial augmentation, and quality control.
- Strong applied experience with LLMs and agentic systems, including prompting, fine-tuning, and evaluation.
- Proficiency in Python and the modern ML stack (PyTorch, Hugging Face, common training/serving frameworks).
- Comfort working in production environments and partnering with backend teams.