Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909471972786176 |
|---|---|
| author | Girmaji, Rohit Jain, Siddharth Beri, Bhav Bansal, Sarthak Gandhi, Vineet |
| author_facet | Girmaji, Rohit Jain, Siddharth Beri, Bhav Bansal, Sarthak Gandhi, Vineet |
| contents | This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameters without compromising performance. Additionally, ViNet-A (148MB) incorporates spatio-temporal action localization (STAL) features, differing from traditional video saliency models that use action classification backbones. Our studies show that an ensemble of ViNet-S and ViNet-A, by averaging predicted saliency maps, achieves state-of-the-art performance on three visual-only and six audio-visual saliency datasets, outperforming transformer-based models in both parameter efficiency and real-time performance, with ViNet-S reaching over 1000fps. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_00397 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues Girmaji, Rohit Jain, Siddharth Beri, Bhav Bansal, Sarthak Gandhi, Vineet Computer Vision and Pattern Recognition This paper introduces ViNet-S, a 36MB model based on the ViNet architecture with a U-Net design, featuring a lightweight decoder that significantly reduces model size and parameters without compromising performance. Additionally, ViNet-A (148MB) incorporates spatio-temporal action localization (STAL) features, differing from traditional video saliency models that use action classification backbones. Our studies show that an ensemble of ViNet-S and ViNet-A, by averaging predicted saliency maps, achieves state-of-the-art performance on three visual-only and six audio-visual saliency datasets, outperforming transformer-based models in both parameter efficiency and real-time performance, with ViNet-S reaching over 1000fps. |
| title | Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2502.00397 |