AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2023
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916553947086848 |
|---|---|
| author | Deiseroth, Björn Deb, Mayukh Weinbach, Samuel Brack, Manuel Schramowski, Patrick Kersting, Kristian |
| author_facet | Deiseroth, Björn Deb, Mayukh Weinbach, Samuel Brack, Manuel Schramowski, Patrick Kersting, Kristian |
| contents | Generative transformer models have become increasingly complex, with large numbers of parameters and the ability to process multiple input modalities. Current methods for explaining their predictions are resource-intensive. Most crucially, they require prohibitively large amounts of extra memory, since they rely on backpropagation which allocates almost twice as much GPU memory as the forward pass. This makes it difficult, if not impossible, to use them in production. We present AtMan that provides explanations of generative transformer models at almost no extra cost. Specifically, AtMan is a modality-agnostic perturbation method that manipulates the attention mechanisms of transformers to produce relevance maps for the input with respect to the output prediction. Instead of using backpropagation, AtMan applies a parallelizable token-based search method based on cosine similarity neighborhood in the embedding space. Our exhaustive experiments on text and image-text benchmarks demonstrate that AtMan outperforms current state-of-the-art gradient-based methods on several metrics while being computationally efficient. As such, AtMan is suitable for use in large model inference deployments. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_08110 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation Deiseroth, Björn Deb, Mayukh Weinbach, Samuel Brack, Manuel Schramowski, Patrick Kersting, Kristian Machine Learning Artificial Intelligence Generative transformer models have become increasingly complex, with large numbers of parameters and the ability to process multiple input modalities. Current methods for explaining their predictions are resource-intensive. Most crucially, they require prohibitively large amounts of extra memory, since they rely on backpropagation which allocates almost twice as much GPU memory as the forward pass. This makes it difficult, if not impossible, to use them in production. We present AtMan that provides explanations of generative transformer models at almost no extra cost. Specifically, AtMan is a modality-agnostic perturbation method that manipulates the attention mechanisms of transformers to produce relevance maps for the input with respect to the output prediction. Instead of using backpropagation, AtMan applies a parallelizable token-based search method based on cosine similarity neighborhood in the embedding space. Our exhaustive experiments on text and image-text benchmarks demonstrate that AtMan outperforms current state-of-the-art gradient-based methods on several metrics while being computationally efficient. As such, AtMan is suitable for use in large model inference deployments. |
| title | AtMan: Understanding Transformer Predictions Through Memory Efficient Attention Manipulation |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2301.08110 |