Thanks to the interpretability repository provided by Meta.

Our repository retains the functionalities of the original and introduces a new convenient plotting feature using Matplotlib. Additionally, we have modified the code to support direct execution of local checkpoints.

The guide of installation and running are as follows:

Local Installation

# download
git clone git@github.com:facebookresearch/llm-transparency-tool.git
cd llm-transparency-tool

# install the necessary packages
conda env create --name llmtt -f env.yaml
# install the `llm_transparency_tool` package
pip install -e .

# now, we need to build the frontend
# don't worry, even `yarn` comes preinstalled by `env.yaml`
cd llm_transparency_tool/components/frontend
yarn install
yarn build

Collect Knowledge Circuits

To collect Knowledge Circuits, execute the run.py script with the following command:

python run.py --model_path "/path/to/model_ckpt/" --model_name "model-offical-name" --output_dir "/path/to/save.json"

Alternatively, you can use the provided shell script:

bash run.sh

Plot Knowledge Circuits Figures

To generate and plot the Knowledge Circuits figures, run the following command:

bash plt.sh

This will produce visual representations of the Knowledge Circuits based on the collected data.

The effect is as follows:

The demo of original github

streamlit run llm_transparency_tool/server/app.py -- config/local.json

Adding support for your LLM

Initially, the tool allows you to select from just a handful of models. Here are the options you can try for using your model in the tool, from least to most effort.

The model is already supported by TransformerLens

Full list of models is here. In this case, the model can be added to the configuration json file.

Tuned version of a model supported by TransformerLens

Add the official name of the model to the config along with the location to read the weights from.

The model is not supported by TransformerLens

In this case the UI wouldn't know how to create proper hooks for the model. You'd need to implement your version of TransparentLlm class and alter the Streamlit app to use your implementation.

Citation

If you use the LLM Transparency Tool for your research, please consider citing:

@article{tufanov2024lm,
      title={LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models}, 
      author={Igor Tufanov and Karen Hambardzumyan and Javier Ferrando and Elena Voita},
      year={2024},
      journal={Arxiv},
      url={https://arxiv.org/abs/2404.07004}
}

@article{ferrando2024information,
    title={Information Flow Routes: Automatically Interpreting Language Models at Scale}, 
    author={Javier Ferrando and Elena Voita},
    year={2024},
    journal={Arxiv},
    url={https://arxiv.org/abs/2403.00824}
}

License

This code is made available under a CC BY-NC 4.0 license, as found in the LICENSE file. However you may have other legal obligations that govern your use of other content, such as the terms of service for third-party models.