Cross-Domain Few-Shot Learning via Adaptive Transformer Networks
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866910307792715776 |
|---|---|
| author | Paeedeh, Naeem Pratama, Mahardhika Ma'sum, Muhammad Anwar Mayer, Wolfgang Cao, Zehong Kowlczyk, Ryszard |
| author_facet | Paeedeh, Naeem Pratama, Mahardhika Ma'sum, Muhammad Anwar Mayer, Wolfgang Cao, Zehong Kowlczyk, Ryszard |
| contents | Most few-shot learning works rely on the same domain assumption between the base and the target tasks, hindering their practical applications. This paper proposes an adaptive transformer network (ADAPTER), a simple but effective solution for cross-domain few-shot learning where there exist large domain shifts between the base task and the target task. ADAPTER is built upon the idea of bidirectional cross-attention to learn transferable features between the two domains. The proposed architecture is trained with DINO to produce diverse, and less biased features to avoid the supervision collapse problem. Furthermore, the label smoothing approach is proposed to improve the consistency and reliability of the predictions by also considering the predicted labels of the close samples in the embedding space. The performance of ADAPTER is rigorously evaluated in the BSCD-FSL benchmarks in which it outperforms prior arts with significant margins. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2401_13987 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cross-Domain Few-Shot Learning via Adaptive Transformer Networks Paeedeh, Naeem Pratama, Mahardhika Ma'sum, Muhammad Anwar Mayer, Wolfgang Cao, Zehong Kowlczyk, Ryszard Machine Learning Artificial Intelligence Most few-shot learning works rely on the same domain assumption between the base and the target tasks, hindering their practical applications. This paper proposes an adaptive transformer network (ADAPTER), a simple but effective solution for cross-domain few-shot learning where there exist large domain shifts between the base task and the target task. ADAPTER is built upon the idea of bidirectional cross-attention to learn transferable features between the two domains. The proposed architecture is trained with DINO to produce diverse, and less biased features to avoid the supervision collapse problem. Furthermore, the label smoothing approach is proposed to improve the consistency and reliability of the predictions by also considering the predicted labels of the close samples in the embedding space. The performance of ADAPTER is rigorously evaluated in the BSCD-FSL benchmarks in which it outperforms prior arts with significant margins. |
| title | Cross-Domain Few-Shot Learning via Adaptive Transformer Networks |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2401.13987 |