Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.02072 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866914001574690816 |
|---|---|
| author | Chen, Yi-Hsin Yao, Yi-Chen Ho, Kuan-Wei Wu, Chun-Hung Phung, Huu-Tai Benjak, Martin Ostermann, Jörn Peng, Wen-Hsiao |
| author_facet | Chen, Yi-Hsin Yao, Yi-Chen Ho, Kuan-Wei Wu, Chun-Hung Phung, Huu-Tai Benjak, Martin Ostermann, Jörn Peng, Wen-Hsiao |
| contents | Most frame-based learned video codecs can be interpreted as recurrent neural networks (RNNs) propagating reference information along the temporal dimension. This work revisits the limitations of the current approaches from an RNN perspective. The output-recurrence methods, which propagate decoded frames, are intuitive but impose dual constraints on the output decoded frames, leading to suboptimal rate-distortion performance. In contrast, the hidden-to-hidden connection approaches, which propagate latent features within the RNN, offer greater flexibility but require large buffer sizes. To address these issues, we propose HyTIP, a learned video coding framework that combines both mechanisms. Our hybrid buffering strategy uses explicit decoded frames and a small number of implicit latent features to achieve competitive coding performance. Experimental results show that our HyTIP outperforms the sole use of either output-recurrence or hidden-to-hidden approaches. Furthermore, it achieves comparable performance to state-of-the-art methods but with a much smaller buffer size, and outperforms VTM 17.0 (Low-delay B) in terms of PSNR-RGB and MS-SSIM-RGB. The source code of HyTIP is available at https://github.com/NYCU-MAPL/HyTIP. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_02072 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | HyTIP: Hybrid Temporal Information Propagation for Masked Conditional Residual Video Coding Chen, Yi-Hsin Yao, Yi-Chen Ho, Kuan-Wei Wu, Chun-Hung Phung, Huu-Tai Benjak, Martin Ostermann, Jörn Peng, Wen-Hsiao Image and Video Processing Most frame-based learned video codecs can be interpreted as recurrent neural networks (RNNs) propagating reference information along the temporal dimension. This work revisits the limitations of the current approaches from an RNN perspective. The output-recurrence methods, which propagate decoded frames, are intuitive but impose dual constraints on the output decoded frames, leading to suboptimal rate-distortion performance. In contrast, the hidden-to-hidden connection approaches, which propagate latent features within the RNN, offer greater flexibility but require large buffer sizes. To address these issues, we propose HyTIP, a learned video coding framework that combines both mechanisms. Our hybrid buffering strategy uses explicit decoded frames and a small number of implicit latent features to achieve competitive coding performance. Experimental results show that our HyTIP outperforms the sole use of either output-recurrence or hidden-to-hidden approaches. Furthermore, it achieves comparable performance to state-of-the-art methods but with a much smaller buffer size, and outperforms VTM 17.0 (Low-delay B) in terms of PSNR-RGB and MS-SSIM-RGB. The source code of HyTIP is available at https://github.com/NYCU-MAPL/HyTIP. |
| title | HyTIP: Hybrid Temporal Information Propagation for Masked Conditional Residual Video Coding |
| topic | Image and Video Processing |
| url | https://arxiv.org/abs/2508.02072 |