Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908384520830976 |
|---|---|
| author | Geng, Yuxia Zhu, Runkai Chen, Jiaoyan Chen, Jintai Chen, Xiang Chen, Zhuo Qiao, Shuofei Wang, Yuxiang Xu, Xiaoliang Huang, Sheng-Jun |
| author_facet | Geng, Yuxia Zhu, Runkai Chen, Jiaoyan Chen, Jintai Chen, Xiang Chen, Zhuo Qiao, Shuofei Wang, Yuxiang Xu, Xiaoliang Huang, Sheng-Jun |
| contents | Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL). However, due to the feature divergence of an attribute (resp. object) when combined with different objects (resp. attributes), it is challenging to learn disentangled primitive features that are general across different compositions. To this end, we propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions. More specifically, we leverage a compositional graph to define the overall primitive-sharing relationships between compositions, and build a task-specific architecture upon the recently successful large pre-trained vision-language model (VLM) CLIP, with dual cross-composition disentangling adapters (called L-Adapter and V-Adapter) inserted into CLIP's frozen text and image encoders, respectively. Evaluation on three popular CZSL benchmarks shows that our proposed solution significantly improves the performance of CZSL, and its components have been verified by solid ablation studies. Our code and data are available at:https://github.com/zhurunkai/DCDA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_09786 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning Geng, Yuxia Zhu, Runkai Chen, Jiaoyan Chen, Jintai Chen, Xiang Chen, Zhuo Qiao, Shuofei Wang, Yuxiang Xu, Xiaoliang Huang, Sheng-Jun Computer Vision and Pattern Recognition Disentanglement of visual features of primitives (i.e., attributes and objects) has shown exceptional results in Compositional Zero-shot Learning (CZSL). However, due to the feature divergence of an attribute (resp. object) when combined with different objects (resp. attributes), it is challenging to learn disentangled primitive features that are general across different compositions. To this end, we propose the solution of cross-composition feature disentanglement, which takes multiple primitive-sharing compositions as inputs and constrains the disentangled primitive features to be general across these compositions. More specifically, we leverage a compositional graph to define the overall primitive-sharing relationships between compositions, and build a task-specific architecture upon the recently successful large pre-trained vision-language model (VLM) CLIP, with dual cross-composition disentangling adapters (called L-Adapter and V-Adapter) inserted into CLIP's frozen text and image encoders, respectively. Evaluation on three popular CZSL benchmarks shows that our proposed solution significantly improves the performance of CZSL, and its components have been verified by solid ablation studies. Our code and data are available at:https://github.com/zhurunkai/DCDA. |
| title | Graph-guided Cross-composition Feature Disentanglement for Compositional Zero-shot Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.09786 |