Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916900592680960 |
|---|---|
| author | Wu, You Li, Yongxin Liu, Mengyuan Wang, Xucheng Yang, Xiangyang Ye, Hengzhou Zeng, Dan Zhao, Qijun Li, Shuiwang |
| author_facet | Wu, You Li, Yongxin Liu, Mengyuan Wang, Xucheng Yang, Xiangyang Ye, Hengzhou Zeng, Dan Zhao, Qijun Li, Shuiwang |
| contents | Transformer-based models have improved visual tracking, but most still cannot run in real time on resource-limited devices, especially for unmanned aerial vehicle (UAV) tracking. To achieve a better balance between performance and efficiency, we propose AVTrack, an adaptive computation tracking framework that adaptively activates transformer blocks through an Activation Module (AM), which dynamically optimizes the ViT architecture by selectively engaging relevant components. To address extreme viewpoint variations, we propose to learn view-invariant representations via mutual information (MI) maximization. In addition, we propose AVTrack-MD, an enhanced tracker incorporating a novel MI maximization-based multi-teacher knowledge distillation framework. Leveraging multiple off-the-shelf AVTrack models as teachers, we maximize the MI between their aggregated softened features and the corresponding softened feature of the student model, improving the generalization and performance of the student, especially under noisy conditions. Extensive experiments show that AVTrack-MD achieves performance comparable to AVTrack's performance while reducing model complexity and boosting average tracking speed by over 17\%. Codes is available at: https://github.com/wuyou3474/AVTrack. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_20002 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking Wu, You Li, Yongxin Liu, Mengyuan Wang, Xucheng Yang, Xiangyang Ye, Hengzhou Zeng, Dan Zhao, Qijun Li, Shuiwang Computer Vision and Pattern Recognition Transformer-based models have improved visual tracking, but most still cannot run in real time on resource-limited devices, especially for unmanned aerial vehicle (UAV) tracking. To achieve a better balance between performance and efficiency, we propose AVTrack, an adaptive computation tracking framework that adaptively activates transformer blocks through an Activation Module (AM), which dynamically optimizes the ViT architecture by selectively engaging relevant components. To address extreme viewpoint variations, we propose to learn view-invariant representations via mutual information (MI) maximization. In addition, we propose AVTrack-MD, an enhanced tracker incorporating a novel MI maximization-based multi-teacher knowledge distillation framework. Leveraging multiple off-the-shelf AVTrack models as teachers, we maximize the MI between their aggregated softened features and the corresponding softened feature of the student model, improving the generalization and performance of the student, especially under noisy conditions. Extensive experiments show that AVTrack-MD achieves performance comparable to AVTrack's performance while reducing model complexity and boosting average tracking speed by over 17\%. Codes is available at: https://github.com/wuyou3474/AVTrack. |
| title | Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2412.20002 |