Location-Free Scene Graph Generation
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2023
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866915111000604672 |
|---|---|
| author | Özsoy, Ege Holm, Felix Saleh, Mahdi Czempiel, Tobias Pellegrini, Chantal Navab, Nassir Busam, Benjamin |
| author_facet | Özsoy, Ege Holm, Felix Saleh, Mahdi Czempiel, Tobias Pellegrini, Chantal Navab, Nassir Busam, Benjamin |
| contents | Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other. Existing works rely on location labels in form of bounding boxes or segmentation masks, increasing annotation costs and limiting dataset expansion. Recognizing that many applications do not require location data, we break this dependency and introduce location-free scene graph generation (LF-SGG). This new task aims at predicting instances of entities, as well as their relationships, without the explicit calculation of their spatial localization. To objectively evaluate the task, the predicted and ground truth scene graphs need to be compared. We solve this NP-hard problem through an efficient branching algorithm. Additionally, we design the first LF-SGG method, Pix2SG, using autoregressive sequence modeling. We demonstrate the effectiveness of our method on three scene graph generation datasets as well as two downstream tasks, image retrieval and visual question answering, and show that our approach is competitive to existing methods while not relying on location cues. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2303_10944 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Location-Free Scene Graph Generation Özsoy, Ege Holm, Felix Saleh, Mahdi Czempiel, Tobias Pellegrini, Chantal Navab, Nassir Busam, Benjamin Computer Vision and Pattern Recognition Scene Graph Generation (SGG) is a visual understanding task, aiming to describe a scene as a graph of entities and their relationships with each other. Existing works rely on location labels in form of bounding boxes or segmentation masks, increasing annotation costs and limiting dataset expansion. Recognizing that many applications do not require location data, we break this dependency and introduce location-free scene graph generation (LF-SGG). This new task aims at predicting instances of entities, as well as their relationships, without the explicit calculation of their spatial localization. To objectively evaluate the task, the predicted and ground truth scene graphs need to be compared. We solve this NP-hard problem through an efficient branching algorithm. Additionally, we design the first LF-SGG method, Pix2SG, using autoregressive sequence modeling. We demonstrate the effectiveness of our method on three scene graph generation datasets as well as two downstream tasks, image retrieval and visual question answering, and show that our approach is competitive to existing methods while not relying on location cues. |
| title | Location-Free Scene Graph Generation |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2303.10944 |