VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability
Fuente:
arXiv
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Preprint |
| Published: |
2018
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913244803760128 |
|---|---|
| author | Cohendet, Romain Demarty, Claire-Hélène Duong, Ngoc Q. K. Engilberge, Martin |
| author_facet | Cohendet, Romain Demarty, Claire-Hélène Duong, Ngoc Q. K. Engilberge, Martin |
| contents | Humans share a strong tendency to memorize/forget some of the visual information they encounter. This paper focuses on providing computational models for the prediction of the intrinsic memorability of visual content. To address this new challenge, we introduce a large scale dataset (VideoMem) composed of 10,000 videos annotated with memorability scores. In contrast to previous work on image memorability -- where memorability was measured a few minutes after memorization -- memory performance is measured twice: a few minutes after memorization and again 24-72 hours later. Hence, the dataset comes with short-term and long-term memorability annotations. After an in-depth analysis of the dataset, we investigate several deep neural network based models for the prediction of video memorability. Our best model using a ranking loss achieves a Spearman's rank correlation of 0.494 for short-term memorability prediction, while our proposed model with attention mechanism provides insights of what makes a content memorable. The VideoMem dataset with pre-extracted features is publicly available. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_1812_01973 |
| institution | arXiv |
| publishDate | 2018 |
| record_format | arxiv |
| spellingShingle | VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability Cohendet, Romain Demarty, Claire-Hélène Duong, Ngoc Q. K. Engilberge, Martin Computer Vision and Pattern Recognition Multimedia Humans share a strong tendency to memorize/forget some of the visual information they encounter. This paper focuses on providing computational models for the prediction of the intrinsic memorability of visual content. To address this new challenge, we introduce a large scale dataset (VideoMem) composed of 10,000 videos annotated with memorability scores. In contrast to previous work on image memorability -- where memorability was measured a few minutes after memorization -- memory performance is measured twice: a few minutes after memorization and again 24-72 hours later. Hence, the dataset comes with short-term and long-term memorability annotations. After an in-depth analysis of the dataset, we investigate several deep neural network based models for the prediction of video memorability. Our best model using a ranking loss achieves a Spearman's rank correlation of 0.494 for short-term memorability prediction, while our proposed model with attention mechanism provides insights of what makes a content memorable. The VideoMem dataset with pre-extracted features is publicly available. |
| title | VideoMem: Constructing, Analyzing, Predicting Short-term and Long-term Video Memorability |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/1812.01973 |