Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2023
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909134424637440 |
|---|---|
| author | Jelenić, Fran Jukić, Josip Tutek, Martin Puljiz, Mate Šnajder, Jan |
| author_facet | Jelenić, Fran Jukić, Josip Tutek, Martin Puljiz, Mate Šnajder, Jan |
| contents | Effective out-of-distribution (OOD) detection is crucial for reliable machine learning models, yet most current methods are limited in practical use due to requirements like access to training data or intervention in training. We present a novel method for detecting OOD data in Transformers based on transformation smoothness between intermediate layers of a network (BLOOD), which is applicable to pre-trained models without access to training data. BLOOD utilizes the tendency of between-layer representation transformations of in-distribution (ID) data to be smoother than the corresponding transformations of OOD data, a property that we also demonstrate empirically. We evaluate BLOOD on several text classification tasks with Transformer networks and demonstrate that it outperforms methods with comparable resource requirements. Our analysis also suggests that when learning simpler tasks, OOD data transformations maintain their original sharpness, whereas sharpness increases with more complex tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2310_02832 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness Jelenić, Fran Jukić, Josip Tutek, Martin Puljiz, Mate Šnajder, Jan Machine Learning Computation and Language Effective out-of-distribution (OOD) detection is crucial for reliable machine learning models, yet most current methods are limited in practical use due to requirements like access to training data or intervention in training. We present a novel method for detecting OOD data in Transformers based on transformation smoothness between intermediate layers of a network (BLOOD), which is applicable to pre-trained models without access to training data. BLOOD utilizes the tendency of between-layer representation transformations of in-distribution (ID) data to be smoother than the corresponding transformations of OOD data, a property that we also demonstrate empirically. We evaluate BLOOD on several text classification tasks with Transformer networks and demonstrate that it outperforms methods with comparable resource requirements. Our analysis also suggests that when learning simpler tasks, OOD data transformations maintain their original sharpness, whereas sharpness increases with more complex tasks. |
| title | Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness |
| topic | Machine Learning Computation and Language |
| url | https://arxiv.org/abs/2310.02832 |