Explaining Representation Learning with Perceptual Components
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917690109591552 |
|---|---|
| author | Yarici, Yavuz Kokilepersaud, Kiran Prabhushankar, Mohit AlRegib, Ghassan |
| author_facet | Yarici, Yavuz Kokilepersaud, Kiran Prabhushankar, Mohit AlRegib, Ghassan |
| contents | Self-supervised models create representation spaces that lack clear semantic meaning. This interpretability problem of representations makes traditional explainability methods ineffective in this context. In this paper, we introduce a novel method to analyze representation spaces using three key perceptual components: color, shape, and texture. We employ selective masking of these components to observe changes in representations, resulting in distinct importance maps for each. In scenarios, where labels are absent, these importance maps provide more intuitive explanations as they are integral to the human visual system. Our approach enhances the interpretability of the representation space, offering explanations that resonate with human visual perception. We analyze how different training objectives create distinct representation spaces using perceptual components. Additionally, we examine the representation of images across diverse image domains, providing insights into the role of these components in different contexts. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_06930 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Explaining Representation Learning with Perceptual Components Yarici, Yavuz Kokilepersaud, Kiran Prabhushankar, Mohit AlRegib, Ghassan Computer Vision and Pattern Recognition Self-supervised models create representation spaces that lack clear semantic meaning. This interpretability problem of representations makes traditional explainability methods ineffective in this context. In this paper, we introduce a novel method to analyze representation spaces using three key perceptual components: color, shape, and texture. We employ selective masking of these components to observe changes in representations, resulting in distinct importance maps for each. In scenarios, where labels are absent, these importance maps provide more intuitive explanations as they are integral to the human visual system. Our approach enhances the interpretability of the representation space, offering explanations that resonate with human visual perception. We analyze how different training objectives create distinct representation spaces using perceptual components. Additionally, we examine the representation of images across diverse image domains, providing insights into the role of these components in different contexts. |
| title | Explaining Representation Learning with Perceptual Components |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.06930 |