Unsupervised Layer-wise Score Aggregation for Textual OOD Detection
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914687113756672 |
|---|---|
| author | Darrin, Maxime Staerman, Guillaume Gomes, Eduardo Dadalto Câmara Cheung, Jackie CK Piantanida, Pablo Colombo, Pierre |
| author_facet | Darrin, Maxime Staerman, Guillaume Gomes, Eduardo Dadalto Câmara Cheung, Jackie CK Piantanida, Pablo Colombo, Pierre |
| contents | Out-of-distribution (OOD) detection is a rapidly growing field due to new robustness and security requirements driven by an increased number of AI-based systems. Existing OOD textual detectors often rely on an anomaly score (e.g., Mahalanobis distance) computed on the embedding output of the last layer of the encoder. In this work, we observe that OOD detection performance varies greatly depending on the task and layer output. More importantly, we show that the usual choice (the last layer) is rarely the best one for OOD detection and that far better results could be achieved if the best layer were picked. To leverage this observation, we propose a data-driven, unsupervised method to combine layer-wise anomaly scores. In addition, we extend classical textual OOD benchmarks by including classification tasks with a greater number of classes (up to 77), which reflects more realistic settings. On this augmented benchmark, we show that the proposed post-aggregation methods achieve robust and consistent results while removing manual feature selection altogether. Their performance achieves near oracle's best layer performance. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2302_09852 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Unsupervised Layer-wise Score Aggregation for Textual OOD Detection Darrin, Maxime Staerman, Guillaume Gomes, Eduardo Dadalto Câmara Cheung, Jackie CK Piantanida, Pablo Colombo, Pierre Computation and Language Artificial Intelligence Out-of-distribution (OOD) detection is a rapidly growing field due to new robustness and security requirements driven by an increased number of AI-based systems. Existing OOD textual detectors often rely on an anomaly score (e.g., Mahalanobis distance) computed on the embedding output of the last layer of the encoder. In this work, we observe that OOD detection performance varies greatly depending on the task and layer output. More importantly, we show that the usual choice (the last layer) is rarely the best one for OOD detection and that far better results could be achieved if the best layer were picked. To leverage this observation, we propose a data-driven, unsupervised method to combine layer-wise anomaly scores. In addition, we extend classical textual OOD benchmarks by including classification tasks with a greater number of classes (up to 77), which reflects more realistic settings. On this augmented benchmark, we show that the proposed post-aggregation methods achieve robust and consistent results while removing manual feature selection altogether. Their performance achieves near oracle's best layer performance. |
| title | Unsupervised Layer-wise Score Aggregation for Textual OOD Detection |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2302.09852 |