Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.22373 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916865778909184 |
|---|---|
| author | Zhang, Yufei Xu, Yicheng Wei, Hongxin Lin, Zhiping Zou, Xiaofeng Chen, Cen Zhuang, Huiping |
| author_facet | Zhang, Yufei Xu, Yicheng Wei, Hongxin Lin, Zhiping Zou, Xiaofeng Chen, Cen Zhuang, Huiping |
| contents | Test-Time Adaptation (TTA) enables pre-trained models to bridge the gap between source and target datasets using unlabeled test data, addressing domain shifts caused by corruptions like weather changes, noise, or sensor malfunctions in test time. Multi-Modal Continual Test-Time Adaptation (MM-CTTA), as an extension of standard TTA, further allows models to handle multi-modal inputs and adapt to continuously evolving target domains. However, MM-CTTA faces critical challenges such as catastrophic forgetting and reliability bias, which are rarely addressed effectively under multi-modal corruption scenarios. In this paper, we propose a novel approach, Multi-modality Dynamic Analytic Adapter (MDAA), to tackle MM-CTTA tasks. MDAA introduces analytic learning,a closed-form training technique,through Analytic Classifiers (ACs) to mitigate catastrophic forgetting. Furthermore, we design the Dynamic Late Fusion Mechanism (DLFM) to dynamically select and integrate reliable information from different modalities. Extensive experiments show that MDAA achieves state-of-the-art performance across the proposed tasks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_22373 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Analytic Continual Test-Time Adaptation for Multi-Modality Corruption Zhang, Yufei Xu, Yicheng Wei, Hongxin Lin, Zhiping Zou, Xiaofeng Chen, Cen Zhuang, Huiping Machine Learning Artificial Intelligence Test-Time Adaptation (TTA) enables pre-trained models to bridge the gap between source and target datasets using unlabeled test data, addressing domain shifts caused by corruptions like weather changes, noise, or sensor malfunctions in test time. Multi-Modal Continual Test-Time Adaptation (MM-CTTA), as an extension of standard TTA, further allows models to handle multi-modal inputs and adapt to continuously evolving target domains. However, MM-CTTA faces critical challenges such as catastrophic forgetting and reliability bias, which are rarely addressed effectively under multi-modal corruption scenarios. In this paper, we propose a novel approach, Multi-modality Dynamic Analytic Adapter (MDAA), to tackle MM-CTTA tasks. MDAA introduces analytic learning,a closed-form training technique,through Analytic Classifiers (ACs) to mitigate catastrophic forgetting. Furthermore, we design the Dynamic Late Fusion Mechanism (DLFM) to dynamically select and integrate reliable information from different modalities. Extensive experiments show that MDAA achieves state-of-the-art performance across the proposed tasks. |
| title | Analytic Continual Test-Time Adaptation for Multi-Modality Corruption |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.22373 |