Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2511.18925 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915783393673216 |
|---|---|
| author | Mali, Yash Shelhamer, Evan |
| author_facet | Mali, Yash Shelhamer, Evan |
| contents | Test-time adaptation (TTA) updates models during inference to reduce error on distribution shifts. While entropy minimization over the output distribution has proven effective as a TTA loss, we study using the intermediate distributions computed by transformers in the attention mechanism. We propose LookSharp, which minimizes the entropy of CLS-to-patch attention in the final layer as a novel TTA objective, encouraging the model to maintain focused attention on shifted data. We demonstrate that attention entropy minimization improves robustness on ImageNet-C. We also show that it is complementary to output entropy minimization and maintains performance on clean data. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_18925 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | LookSharp: Attention Entropy Minimization for Test-Time Adaptation Mali, Yash Shelhamer, Evan Computer Vision and Pattern Recognition Test-time adaptation (TTA) updates models during inference to reduce error on distribution shifts. While entropy minimization over the output distribution has proven effective as a TTA loss, we study using the intermediate distributions computed by transformers in the attention mechanism. We propose LookSharp, which minimizes the entropy of CLS-to-patch attention in the final layer as a novel TTA objective, encouraging the model to maintain focused attention on shifted data. We demonstrate that attention entropy minimization improves robustness on ImageNet-C. We also show that it is complementary to output entropy minimization and maintains performance on clean data. |
| title | LookSharp: Attention Entropy Minimization for Test-Time Adaptation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.18925 |