Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2407.13141 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910533109678080 |
|---|---|
| author | Gulati, Aryan Dong, Xingjian Hurtado, Carlos Shekkizhar, Sarath Swayamdipta, Swabha Ortega, Antonio |
| author_facet | Gulati, Aryan Dong, Xingjian Hurtado, Carlos Shekkizhar, Sarath Swayamdipta, Swabha Ortega, Antonio |
| contents | As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to 11x improvement in inference time and 87% reduction in storage requirements) and outperforms existing approaches by up to 4 AUROC points on four different benchmarks. We also introduce an entropy-constrained version of our algorithm, which leads to further reductions in storage requirements (up to 97% lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13141 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression Gulati, Aryan Dong, Xingjian Hurtado, Carlos Shekkizhar, Sarath Swayamdipta, Swabha Ortega, Antonio Machine Learning As language models become more general purpose, increased attention needs to be paid to detecting out-of-distribution (OOD) instances, i.e., those not belonging to any of the distributions seen during training. Existing methods for detecting OOD data are computationally complex and storage-intensive. We propose a novel soft clustering approach for OOD detection based on non-negative kernel regression. Our approach greatly reduces computational and space complexities (up to 11x improvement in inference time and 87% reduction in storage requirements) and outperforms existing approaches by up to 4 AUROC points on four different benchmarks. We also introduce an entropy-constrained version of our algorithm, which leads to further reductions in storage requirements (up to 97% lower than comparable approaches) while retaining competitive performance. Our soft clustering approach for OOD detection highlights its potential for detecting tail-end phenomena in extreme-scale data settings. |
| title | Out-of-Distribution Detection through Soft Clustering with Non-Negative Kernel Regression |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2407.13141 |