Reparo: Loss-Resilient Generative Codec for Video Conferencing
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2023
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866913532541403136 |
|---|---|
| author | Li, Tianhong Sivaraman, Vibhaalakshmi Karimi, Pantea Fan, Lijie Alizadeh, Mohammad Katabi, Dina |
| author_facet | Li, Tianhong Sivaraman, Vibhaalakshmi Karimi, Pantea Fan, Lijie Alizadeh, Mohammad Katabi, Dina |
| contents | Packet loss during video conferencing often results in poor quality and video freezing. Retransmitting lost packets is often impractical due to the need for real-time playback, and using Forward Error Correction (FEC) for packet recovery is challenging due to the unpredictable and bursty nature of Internet losses. Excessive redundancy leads to inefficiency and wasted bandwidth, while insufficient redundancy results in undecodable frames, causing video freezes and quality degradation in subsequent frames.
We introduce Reparo -- a loss-resilient video conferencing framework based on generative deep learning models to address these issues. Our approach generates missing information when a frame or part of a frame is lost. This generation is conditioned on the data received thus far, considering the model's understanding of how people and objects appear and interact within the visual realm. Experimental results, using publicly available video conferencing datasets, demonstrate that Reparo outperforms state-of-the-art FEC-based video conferencing solutions in terms of both video quality (measured through PSNR, SSIM, and LPIPS) and the occurrence of video freezes. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2305_14135 |
| institution | arXiv |
| publishDate | 2023 |
| record_format | arxiv |
| spellingShingle | Reparo: Loss-Resilient Generative Codec for Video Conferencing Li, Tianhong Sivaraman, Vibhaalakshmi Karimi, Pantea Fan, Lijie Alizadeh, Mohammad Katabi, Dina Networking and Internet Architecture Computer Vision and Pattern Recognition Packet loss during video conferencing often results in poor quality and video freezing. Retransmitting lost packets is often impractical due to the need for real-time playback, and using Forward Error Correction (FEC) for packet recovery is challenging due to the unpredictable and bursty nature of Internet losses. Excessive redundancy leads to inefficiency and wasted bandwidth, while insufficient redundancy results in undecodable frames, causing video freezes and quality degradation in subsequent frames. We introduce Reparo -- a loss-resilient video conferencing framework based on generative deep learning models to address these issues. Our approach generates missing information when a frame or part of a frame is lost. This generation is conditioned on the data received thus far, considering the model's understanding of how people and objects appear and interact within the visual realm. Experimental results, using publicly available video conferencing datasets, demonstrate that Reparo outperforms state-of-the-art FEC-based video conferencing solutions in terms of both video quality (measured through PSNR, SSIM, and LPIPS) and the occurrence of video freezes. |
| title | Reparo: Loss-Resilient Generative Codec for Video Conferencing |
| topic | Networking and Internet Architecture Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2305.14135 |