ML/Research Engineer, Safeguards
San Francisco, CA | New York City, NY
Posted 17d ago
Job Location
San Francisco, CA | New York City, NY
Tech Stack
Remote Work Policy
On-site
Categories
AI Research Engineer
About the job
Anthropic is seeking ML Engineers and Research Engineers to join the Safeguards ML team. The primary focus of this role is to develop systems that detect and mitigate misuse of AI systems, ranging from individual policy violations to sophisticated coordinated attacks. You will build defenses to ensure product safety as AI capabilities advance, protect user well-being, and guarantee appropriate model behavior across various contexts. This work is crucial for Anthropic's Responsible Scaling Policy commitments.
Responsibilities
- Develop classifiers to detect misuse and anomalous behavior at scale, including synthetic data pipelines and automated evaluation methods.
- Build systems to monitor for multi-exchange harms like coordinated cyber attacks and influence operations, and develop methods for aggregating and analyzing signals.
- Evaluate and improve the safety of agentic products by developing threat models, testing environments for agentic risks, and deploying mitigations for prompt injection attacks.
- Conduct research on automated red-teaming, adversarial robustness, and other areas to identify and prevent misuse.
Requirements
- 4+ years of experience in ML engineering, research engineering, or applied research (academia or industry).
- Proficiency in Python and experience building ML systems.
- Comfort working across the research-to-deployment pipeline, from exploratory experiments to production systems.
- Concern about AI misuse risks and a desire to mitigate them.
- Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders.
Benefits
- Annual compensation range: $350,000 - $500,000 USD
- Visa sponsorship available (with efforts to secure for offers)
- Hybrid work policy (expecting at least 25% in-office time)