ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910633804431360 |
|---|---|
| author | Kwon, Daekyu Kim, Dongyoung Ki, Sehwan Jo, Younghyun Lee, Hyong-Euk Kim, Seon Joo |
| author_facet | Kwon, Daekyu Kim, Dongyoung Ki, Sehwan Jo, Younghyun Lee, Hyong-Euk Kim, Seon Joo |
| contents | In no-reference image quality assessment (NR-IQA), the challenge of limited dataset sizes hampers the development of robust and generalizable models. Conventional methods address this issue by utilizing large datasets to extract rich representations for IQA. Also, some approaches propose vision language models (VLM) based IQA, but the domain gap between generic VLM and IQA constrains their scalability. In this work, we propose a novel pretraining framework that constructs a generalizable representation for IQA by selectively extracting quality-related knowledge from VLM and leveraging the scalability of large datasets. Specifically, we select optimal text prompts for five representative image quality attributes and use VLM to generate pseudo-labels. Numerous attribute-aware pseudo-labels can be generated with large image datasets, allowing our IQA model to learn rich representations about image quality. Our approach achieves state-of-the-art performance on multiple IQA datasets and exhibits remarkable generalization capabilities. Leveraging these strengths, we propose several applications, such as evaluating image generation models and training image enhancement models, demonstrating our model's real-world applicability. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2406_01020 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining Kwon, Daekyu Kim, Dongyoung Ki, Sehwan Jo, Younghyun Lee, Hyong-Euk Kim, Seon Joo Computer Vision and Pattern Recognition In no-reference image quality assessment (NR-IQA), the challenge of limited dataset sizes hampers the development of robust and generalizable models. Conventional methods address this issue by utilizing large datasets to extract rich representations for IQA. Also, some approaches propose vision language models (VLM) based IQA, but the domain gap between generic VLM and IQA constrains their scalability. In this work, we propose a novel pretraining framework that constructs a generalizable representation for IQA by selectively extracting quality-related knowledge from VLM and leveraging the scalability of large datasets. Specifically, we select optimal text prompts for five representative image quality attributes and use VLM to generate pseudo-labels. Numerous attribute-aware pseudo-labels can be generated with large image datasets, allowing our IQA model to learn rich representations about image quality. Our approach achieves state-of-the-art performance on multiple IQA datasets and exhibits remarkable generalization capabilities. Leveraging these strengths, we propose several applications, such as evaluating image generation models and training image enhancement models, demonstrating our model's real-world applicability. |
| title | ATTIQA: Generalizable Image Quality Feature Extractor using Attribute-aware Pretraining |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2406.01020 |