Circuit Tracer - Anthropic open source tool for visualizing the inner workings of models

What's Circuit Tracer?

Circuit Tracer. Anthropic An open source tool for studying the internal workings of large language models. Circuit Tracer is based on the generation of attribution graphs that reveal the steps a model goes through internally when generating a particular output. Attribution graphs help researchers trace the model's decision-making process, visualize the relationships between features, and test different hypotheses.Circuit Tracer supports a variety of popular open-source models, such as Gemma and Llama, and provides an interactive visual interface based on Neuronpedia that allows users to easily explore and analyze the behavior of the model.Circuit Tracer supports Circuit Tracer supports model intervention, which allows users to modify feature values to observe changes in model outputs and verify model behavior and assumptions.

Main functions of Circuit Tracer

• Generating attribution maps: Reveal the internal decision paths of the model as it generates specific outputs, showing the direct influence relationships between features and nodes.
• Visualization and Interaction: Based on the interactive interface provided by Neuronpedia, view and manipulate attribution maps intuitively for easy understanding and sharing.
• model intervention: Modify the eigenvalues in the attribution graph and observe changes in the model output to validate the model behavior and assumptions.
• Support for multiple models: Compatible with a variety of open-source models, such as Gemma and Llama, to facilitate comparative studies.
• Graph pruning and optimization: Automatically removes less influential nodes and edges, simplifying the attribution graph and improving readability.

Circuit Tracer's official website address

• Project website::https://www.anthropic.com/research/open-source-circuit-tracing
• GitHub repository::https://github.com/safety-research/circuit-tracer

How to use Circuit Tracer

• Python scripts or Jupyter notebooks use the::
  • Cloning GitHub Repositories::

git clone https://github.com/safety-research/circuit-tracer.git

• • Installation of dependencies::

cd circuit-tracer
pip install .

• • Run the tutorial notebook demos/circuit_tracing_tutorial.ipynb or create your own script.
• Command Line Interface (CLI) Usage::
  • After installing the dependencies, run the CLI command::

circuit-tracer attribute --prompt "Your prompt here" --transcoder_set gemma --slug demo --graph_file_dir ./graph_files --server

• • Visit the local server (e.g. localhost:8041) to view the attribution graph.

Circuit Tracer's Core Benefits

• Enhancing Model Interpretability: Visualize the model decision-making process based on attribution graphs to help understand the model logic.
• Support for model interventions: Modify the eigenvalues to observe changes in the model output and verify the model behavior.
• Compatible with multiple models: Support Gemma, Llama and other open-source models, easy to compare and research.
• easy-to-use: Web interface, Python scripts, Jupyter notebooks, and command line.
• Open Source and Community Support: Open source code for easy modification and expansion to promote community exchange.
• Graph pruning and optimization: Automatically simplify attribution charts to improve readability.

Who Circuit Tracer is for

• AI researchers: Researchers who need a deeper understanding of the inner workings of the model, e.g., scholars studying complex behaviors such as multilingual models and multi-step reasoning.
• Machine Learning Engineer: Engineers working on model development and optimization, debugging and improving model performance based on Circuit Tracer.
• data scientist: Data scientists who need to explain the decision-making process of a model, especially in domains such as finance and healthcare that require high levels of model interpretability.
• Academic researchers: Scholars conducting AI-related research at universities or research organizations, conducting experiments and publishing research results.
• technology enthusiast: Developers or tech enthusiasts interested in the internal mechanisms of AI models and wanting tools to explore and learn about model behavior.