OmniCaptioner: One Captioner to Rule Them All
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866910979443392512 |
|---|---|
| author | Lu, Yiting Yuan, Jiakang Li, Zhen Zhao, Shitian Qin, Qi Li, Xinyue Zhuo, Le Wen, Licheng Liu, Dongyang Cao, Yuewen Yan, Xiangchao Li, Xin Peng, Tianshuo Zhang, Shufei Shi, Botian Chen, Tao Chen, Zhibo Bai, Lei Gao, Peng Zhang, Bo |
| author_facet | Lu, Yiting Yuan, Jiakang Li, Zhen Zhao, Shitian Qin, Qi Li, Xinyue Zhuo, Le Wen, Licheng Liu, Dongyang Cao, Yuewen Yan, Xiangchao Li, Xin Peng, Tianshuo Zhang, Shufei Shi, Botian Chen, Tao Chen, Zhibo Bai, Lei Gao, Peng Zhang, Bo |
| contents | We propose OmniCaptioner, a versatile visual captioning framework for generating fine-grained textual descriptions across a wide variety of visual domains. Unlike prior methods limited to specific image types (e.g., natural images or geometric visuals), our framework provides a unified solution for captioning natural images, visual text (e.g., posters, UIs, textbooks), and structured visuals (e.g., documents, tables, charts). By converting low-level pixel information into semantically rich textual representations, our framework bridges the gap between visual and textual modalities. Our results highlight three key advantages: (i) Enhanced Visual Reasoning with LLMs, where long-context captions of visual modalities empower LLMs, particularly the DeepSeek-R1 series, to reason effectively in multimodal scenarios; (ii) Improved Image Generation, where detailed captions improve tasks like text-to-image generation and image transformation; and (iii) Efficient Supervised Fine-Tuning (SFT), which enables faster convergence with less data. We believe the versatility and adaptability of OmniCaptioner can offer a new perspective for bridging the gap between language and visual modalities. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_07089 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | OmniCaptioner: One Captioner to Rule Them All Lu, Yiting Yuan, Jiakang Li, Zhen Zhao, Shitian Qin, Qi Li, Xinyue Zhuo, Le Wen, Licheng Liu, Dongyang Cao, Yuewen Yan, Xiangchao Li, Xin Peng, Tianshuo Zhang, Shufei Shi, Botian Chen, Tao Chen, Zhibo Bai, Lei Gao, Peng Zhang, Bo Computer Vision and Pattern Recognition Computation and Language We propose OmniCaptioner, a versatile visual captioning framework for generating fine-grained textual descriptions across a wide variety of visual domains. Unlike prior methods limited to specific image types (e.g., natural images or geometric visuals), our framework provides a unified solution for captioning natural images, visual text (e.g., posters, UIs, textbooks), and structured visuals (e.g., documents, tables, charts). By converting low-level pixel information into semantically rich textual representations, our framework bridges the gap between visual and textual modalities. Our results highlight three key advantages: (i) Enhanced Visual Reasoning with LLMs, where long-context captions of visual modalities empower LLMs, particularly the DeepSeek-R1 series, to reason effectively in multimodal scenarios; (ii) Improved Image Generation, where detailed captions improve tasks like text-to-image generation and image transformation; and (iii) Efficient Supervised Fine-Tuning (SFT), which enables faster convergence with less data. We believe the versatility and adaptability of OmniCaptioner can offer a new perspective for bridging the gap between language and visual modalities. |
| title | OmniCaptioner: One Captioner to Rule Them All |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2504.07089 |