mechanistic interpretability Jobs

Anthropic

Anthropic's mission is to create reliable, interpretable, and steerable AI systems that are safe and beneficial for users and society. The Interpretability team focuses on mechanistic interpretability, aiming to discover how neural network parameters map to meaningful algorithms. This involves reverse-engineering how trained models work, treating neural networks like biological systems or computer programs to be reverse-engineered. The team's goal is to create a solid scientific foundation for understanding neural networks and making them safe, with a focus on issues like "superposition" and decomposing models into interpretable components. This role supports a team of expert researchers and engineers working on these ambitious safety research goals.

$350k - $500k

Anthropic

The Interpretability team at Anthropic is dedicated to reverse-engineering how trained models work, believing that a mechanistic understanding is crucial for making advanced AI systems safe and steerable. This role focuses on mechanistic interpretability, aiming to discover how neural network parameters map to meaningful algorithms, akin to performing "biology" or "neuroscience" on neural networks. The team has made significant progress in understanding issues like "superposition" and decomposing models into interpretable components, with recent work identifying millions of features in production models and developing methods to understand multi-hop reasoning and planning. This research is foundational for building reliable, interpretable, and steerable AI systems and often involves collaboration with other teams at Anthropic to enhance AI safety.

$350k - $850k

OpenAI

OpenAI is seeking a researcher passionate about understanding deep networks, with a strong background in engineering, quantitative reasoning, and the research process. You will develop and carry out a research plan in mechanistic interpretability, in close collaboration with a highly motivated team. You will play a critical role in helping OpenAI ensure future models remain safe even as they grow in capability. This will make a significant impact on our goal of building and deploying safe AGI. The Interpretability team studies internal representations of deep learning models, aiming to understand model behavior and engineer models with more understandable representations, with a particular focus on ensuring the alignment of powerful AI systems.

$295k - $500k