Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866915612125560832 |
|---|---|
| author | Mahmood, Syed Ahmed Ali, Ali Shah Ahmed, Umer Fateh, Fawad Javed Zia, M. Zeeshan Tran, Quoc-Huy |
| author_facet | Mahmood, Syed Ahmed Ali, Ali Shah Ahmed, Umer Fateh, Fawad Javed Zia, M. Zeeshan Tran, Quoc-Huy |
| contents | We study self-supervised procedure learning, which discovers key steps and their order from a set of unlabeled videos. Previous methods typically learn frame-to-frame correspondences between videos before determining key steps and their order. However, their performance often suffers from order variations, background/redundant frames, and repeated actions. To overcome these challenges, we propose a self-supervised framework, which utilizes a fused Gromov-Wasserstein optimal transport with a structural prior for frame-to-frame mapping. However, optimizing only for the above temporal alignment may lead to degenerate solutions, where all frames are mapped to a small cluster in the embedding space and thus every video is assigned to just one key step. To address that issue, we integrate a contrastive regularization, which maps different frames to various points, avoiding trivial solutions. Finally, extensive experiments on egocentric and third-person benchmarks demonstrate our superior performance over prior works, including OPEL which relies on a classical Kantorovich optimal transport with an optimality prior. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_15540 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport Mahmood, Syed Ahmed Ali, Ali Shah Ahmed, Umer Fateh, Fawad Javed Zia, M. Zeeshan Tran, Quoc-Huy Computer Vision and Pattern Recognition We study self-supervised procedure learning, which discovers key steps and their order from a set of unlabeled videos. Previous methods typically learn frame-to-frame correspondences between videos before determining key steps and their order. However, their performance often suffers from order variations, background/redundant frames, and repeated actions. To overcome these challenges, we propose a self-supervised framework, which utilizes a fused Gromov-Wasserstein optimal transport with a structural prior for frame-to-frame mapping. However, optimizing only for the above temporal alignment may lead to degenerate solutions, where all frames are mapped to a small cluster in the embedding space and thus every video is assigned to just one key step. To address that issue, we integrate a contrastive regularization, which maps different frames to various points, avoiding trivial solutions. Finally, extensive experiments on egocentric and third-person benchmarks demonstrate our superior performance over prior works, including OPEL which relies on a classical Kantorovich optimal transport with an optimality prior. |
| title | Procedure Learning via Regularized Gromov-Wasserstein Optimal Transport |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2507.15540 |