Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910385685135360 |
|---|---|
| author | Shan, Ziyu Zhang, Yujie Yang, Qi Yang, Haichen Xu, Yiling Hwang, Jenq-Neng Xu, Xiaozhong Liu, Shan |
| author_facet | Shan, Ziyu Zhang, Yujie Yang, Qi Yang, Haichen Xu, Yiling Hwang, Jenq-Neng Xu, Xiaozhong Liu, Shan |
| contents | No-reference point cloud quality assessment (NR-PCQA) aims to automatically evaluate the perceptual quality of distorted point clouds without available reference, which have achieved tremendous improvements due to the utilization of deep neural networks. However, learning-based NR-PCQA methods suffer from the scarcity of labeled data and usually perform suboptimally in terms of generalization. To solve the problem, we propose a novel contrastive pre-training framework tailored for PCQA (CoPA), which enables the pre-trained model to learn quality-aware representations from unlabeled data. To obtain anchors in the representation space, we project point clouds with different distortions into images and randomly mix their local patches to form mixed images with multiple distortions. Utilizing the generated anchors, we constrain the pre-training process via a quality-aware contrastive loss following the philosophy that perceptual quality is closely related to both content and distortion. Furthermore, in the model fine-tuning stage, we propose a semantic-guided multi-view fusion module to effectively integrate the features of projected images from multiple perspectives. Extensive experiments show that our method outperforms the state-of-the-art PCQA methods on popular benchmarks. Further investigations demonstrate that CoPA can also benefit existing learning-based PCQA models. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_10066 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment Shan, Ziyu Zhang, Yujie Yang, Qi Yang, Haichen Xu, Yiling Hwang, Jenq-Neng Xu, Xiaozhong Liu, Shan Computer Vision and Pattern Recognition Multimedia No-reference point cloud quality assessment (NR-PCQA) aims to automatically evaluate the perceptual quality of distorted point clouds without available reference, which have achieved tremendous improvements due to the utilization of deep neural networks. However, learning-based NR-PCQA methods suffer from the scarcity of labeled data and usually perform suboptimally in terms of generalization. To solve the problem, we propose a novel contrastive pre-training framework tailored for PCQA (CoPA), which enables the pre-trained model to learn quality-aware representations from unlabeled data. To obtain anchors in the representation space, we project point clouds with different distortions into images and randomly mix their local patches to form mixed images with multiple distortions. Utilizing the generated anchors, we constrain the pre-training process via a quality-aware contrastive loss following the philosophy that perceptual quality is closely related to both content and distortion. Furthermore, in the model fine-tuning stage, we propose a semantic-guided multi-view fusion module to effectively integrate the features of projected images from multiple perspectives. Extensive experiments show that our method outperforms the state-of-the-art PCQA methods on popular benchmarks. Further investigations demonstrate that CoPA can also benefit existing learning-based PCQA models. |
| title | Contrastive Pre-Training with Multi-View Fusion for No-Reference Point Cloud Quality Assessment |
| topic | Computer Vision and Pattern Recognition Multimedia |
| url | https://arxiv.org/abs/2403.10066 |