Unveiling Transformer Perception by Exploring Input Manifolds
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908606074454016 |
|---|---|
| author | Benfenati, Alessandro Ferrara, Alfio Marta, Alessio Riva, Davide Rocchetti, Elisabetta |
| author_facet | Benfenati, Alessandro Ferrara, Alfio Marta, Alessio Riva, Davide Rocchetti, Elisabetta |
| contents | This paper introduces a general method for the exploration of equivalence classes in the input space of Transformer models. The proposed approach is based on sound mathematical theory which describes the internal layers of a Transformer architecture as sequential deformations of the input manifold. Using eigendecomposition of the pullback of the distance metric defined on the output space through the Jacobian of the model, we are able to reconstruct equivalence classes in the input space and navigate across them. Our method enables two complementary exploration procedures: the first retrieves input instances that produce the same class probability distribution as the original instance-thus identifying elements within the same equivalence class-while the second discovers instances that yield a different class probability distribution, effectively navigating toward distinct equivalence classes. Finally, we demonstrate how the retrieved instances can be meaningfully interpreted by projecting their embeddings back into a human-readable format. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_06019 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Unveiling Transformer Perception by Exploring Input Manifolds Benfenati, Alessandro Ferrara, Alfio Marta, Alessio Riva, Davide Rocchetti, Elisabetta Machine Learning Artificial Intelligence Computation and Language I.2.7; I.6.4 This paper introduces a general method for the exploration of equivalence classes in the input space of Transformer models. The proposed approach is based on sound mathematical theory which describes the internal layers of a Transformer architecture as sequential deformations of the input manifold. Using eigendecomposition of the pullback of the distance metric defined on the output space through the Jacobian of the model, we are able to reconstruct equivalence classes in the input space and navigate across them. Our method enables two complementary exploration procedures: the first retrieves input instances that produce the same class probability distribution as the original instance-thus identifying elements within the same equivalence class-while the second discovers instances that yield a different class probability distribution, effectively navigating toward distinct equivalence classes. Finally, we demonstrate how the retrieved instances can be meaningfully interpreted by projecting their embeddings back into a human-readable format. |
| title | Unveiling Transformer Perception by Exploring Input Manifolds |
| topic | Machine Learning Artificial Intelligence Computation and Language I.2.7; I.6.4 |
| url | https://arxiv.org/abs/2410.06019 |