Interpreting Low-level Vision Models with Causal Effect Maps
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916667131428864 |
|---|---|
| author | Hu, Jinfan Gu, Jinjin Yu, Shiyao Yu, Fanghua Li, Zheyuan You, Zhiyuan Lu, Chaochao Dong, Chao |
| author_facet | Hu, Jinfan Gu, Jinjin Yu, Shiyao Yu, Fanghua Li, Zheyuan You, Zhiyuan Lu, Chaochao Dong, Chao |
| contents | Deep neural networks have significantly improved the performance of low-level vision tasks but also increased the difficulty of interpretability. A deep understanding of deep models is beneficial for both network design and practical reliability. To take up this challenge, we introduce causality theory to interpret low-level vision models and propose a model-/task-agnostic method called Causal Effect Map (CEM). With CEM, we can visualize and quantify the input-output relationships on either positive or negative effects. After analyzing various low-level vision tasks with CEM, we have reached several interesting insights, such as: (1) Using more information of input images (e.g., larger receptive field) does NOT always yield positive outcomes. (2) Attempting to incorporate mechanisms with a global receptive field (e.g., channel attention) into image denoising may prove futile. (3) Integrating multiple tasks to train a general model could encourage the network to prioritize local information over global context. Based on the causal effect theory, the proposed diagnostic tool can refresh our common knowledge and bring a deeper understanding of low-level vision models. Codes are available at https://github.com/J-FHu/CEM. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_19789 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Interpreting Low-level Vision Models with Causal Effect Maps Hu, Jinfan Gu, Jinjin Yu, Shiyao Yu, Fanghua Li, Zheyuan You, Zhiyuan Lu, Chaochao Dong, Chao Computer Vision and Pattern Recognition Deep neural networks have significantly improved the performance of low-level vision tasks but also increased the difficulty of interpretability. A deep understanding of deep models is beneficial for both network design and practical reliability. To take up this challenge, we introduce causality theory to interpret low-level vision models and propose a model-/task-agnostic method called Causal Effect Map (CEM). With CEM, we can visualize and quantify the input-output relationships on either positive or negative effects. After analyzing various low-level vision tasks with CEM, we have reached several interesting insights, such as: (1) Using more information of input images (e.g., larger receptive field) does NOT always yield positive outcomes. (2) Attempting to incorporate mechanisms with a global receptive field (e.g., channel attention) into image denoising may prove futile. (3) Integrating multiple tasks to train a general model could encourage the network to prioritize local information over global context. Based on the causal effect theory, the proposed diagnostic tool can refresh our common knowledge and bring a deeper understanding of low-level vision models. Codes are available at https://github.com/J-FHu/CEM. |
| title | Interpreting Low-level Vision Models with Causal Effect Maps |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2407.19789 |