TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913494543106048 |
|---|---|
| author | Aydemir, Bahar Hoffstetter, Ludo Zhang, Tong Salzmann, Mathieu Süsstrunk, Sabine |
| author_facet | Aydemir, Bahar Hoffstetter, Ludo Zhang, Tong Salzmann, Mathieu Süsstrunk, Sabine |
| contents | Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. However, none of these models consider the temporal nature of gaze shifts during image observation. We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals by exploiting human temporal attention patterns. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our experiments show that our method outperforms the state-of-the-art models, including a multi-duration saliency model, on the SALICON benchmark. Our code will be publicly available on GitHub. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2301_02315 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction Aydemir, Bahar Hoffstetter, Ludo Zhang, Tong Salzmann, Mathieu Süsstrunk, Sabine Computer Vision and Pattern Recognition Deep saliency prediction algorithms complement the object recognition features, they typically rely on additional information, such as scene context, semantic relationships, gaze direction, and object dissimilarity. However, none of these models consider the temporal nature of gaze shifts during image observation. We introduce a novel saliency prediction model that learns to output saliency maps in sequential time intervals by exploiting human temporal attention patterns. Our approach locally modulates the saliency predictions by combining the learned temporal maps. Our experiments show that our method outperforms the state-of-the-art models, including a multi-duration saliency model, on the SALICON benchmark. Our code will be publicly available on GitHub. |
| title | TempSAL -- Uncovering Temporal Information for Deep Saliency Prediction |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2301.02315 |