Cropper: Vision-Language Model for Image Cropping through In-Context Learning
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910899370983424 |
|---|---|
| author | Lee, Seung Hyun Jiang, Jijun Xu, Yiran Li, Zhuofang Ke, Junjie Li, Yinxiao He, Junfeng Hickson, Steven Datsenko, Katie Kim, Sangpil Yang, Ming-Hsuan Essa, Irfan Yang, Feng |
| author_facet | Lee, Seung Hyun Jiang, Jijun Xu, Yiran Li, Zhuofang Ke, Junjie Li, Yinxiao He, Junfeng Hickson, Steven Datsenko, Katie Kim, Sangpil Yang, Ming-Hsuan Essa, Irfan Yang, Feng |
| contents | The goal of image cropping is to identify visually appealing crops in an image. Conventional methods are trained on specific datasets and fail to adapt to new requirements. Recent breakthroughs in large vision-language models (VLMs) enable visual in-context learning without explicit training. However, downstream tasks with VLMs remain under explored. In this paper, we propose an effective approach to leverage VLMs for image cropping. First, we propose an efficient prompt retrieval mechanism for image cropping to automate the selection of in-context examples. Second, we introduce an iterative refinement strategy to iteratively enhance the predicted crops. The proposed framework, we refer to as Cropper, is applicable to a wide range of cropping tasks, including free-form cropping, subject-aware cropping, and aspect ratio-aware cropping. Extensive experiments demonstrate that Cropper significantly outperforms state-of-the-art methods across several benchmarks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2408_07790 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cropper: Vision-Language Model for Image Cropping through In-Context Learning Lee, Seung Hyun Jiang, Jijun Xu, Yiran Li, Zhuofang Ke, Junjie Li, Yinxiao He, Junfeng Hickson, Steven Datsenko, Katie Kim, Sangpil Yang, Ming-Hsuan Essa, Irfan Yang, Feng Computer Vision and Pattern Recognition The goal of image cropping is to identify visually appealing crops in an image. Conventional methods are trained on specific datasets and fail to adapt to new requirements. Recent breakthroughs in large vision-language models (VLMs) enable visual in-context learning without explicit training. However, downstream tasks with VLMs remain under explored. In this paper, we propose an effective approach to leverage VLMs for image cropping. First, we propose an efficient prompt retrieval mechanism for image cropping to automate the selection of in-context examples. Second, we introduce an iterative refinement strategy to iteratively enhance the predicted crops. The proposed framework, we refer to as Cropper, is applicable to a wide range of cropping tasks, including free-form cropping, subject-aware cropping, and aspect ratio-aware cropping. Extensive experiments demonstrate that Cropper significantly outperforms state-of-the-art methods across several benchmarks. |
| title | Cropper: Vision-Language Model for Image Cropping through In-Context Learning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2408.07790 |