Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910037820047360 |
|---|---|
| author | Yan, Yibo Ou, Mingdong Cao, Yi Zou, Xin Liu, Shuliang Huo, Jiahao Huang, Yu Kwok, James Hu, Xuming |
| author_facet | Yan, Yibo Ou, Mingdong Cao, Yi Zou, Xin Liu, Shuliang Huo, Jiahao Huang, Yu Kwok, James Hu, Xuming |
| contents | Harnessing the full potential of visually-rich documents requires retrieval systems that understand not just text, but intricate layouts, a core challenge in Visual Document Retrieval (VDR). The prevailing multi-vector architectures, while powerful, face a crucial storage bottleneck that current optimization strategies, such as embedding merging, pruning, or using abstract tokens, fail to resolve without compromising performance or ignoring vital layout cues. To address this, we introduce ColParse, a novel paradigm that leverages a document parsing model to generate a small set of layout-informed sub-image embeddings, which are then fused with a global page-level vector to create a compact and structurally-aware multi-vector representation. Extensive experiments demonstrate that our method reduces storage requirements by over 95% while simultaneously yielding significant performance gains across numerous benchmarks and base models. ColParse thus bridges the critical gap between the fine-grained accuracy of multi-vector retrieval and the practical demands of large-scale deployment, offering a new path towards efficient and interpretable multimodal information systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2603_01666 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations Yan, Yibo Ou, Mingdong Cao, Yi Zou, Xin Liu, Shuliang Huo, Jiahao Huang, Yu Kwok, James Hu, Xuming Computation and Language Information Retrieval Harnessing the full potential of visually-rich documents requires retrieval systems that understand not just text, but intricate layouts, a core challenge in Visual Document Retrieval (VDR). The prevailing multi-vector architectures, while powerful, face a crucial storage bottleneck that current optimization strategies, such as embedding merging, pruning, or using abstract tokens, fail to resolve without compromising performance or ignoring vital layout cues. To address this, we introduce ColParse, a novel paradigm that leverages a document parsing model to generate a small set of layout-informed sub-image embeddings, which are then fused with a global page-level vector to create a compact and structurally-aware multi-vector representation. Extensive experiments demonstrate that our method reduces storage requirements by over 95% while simultaneously yielding significant performance gains across numerous benchmarks and base models. ColParse thus bridges the critical gap between the fine-grained accuracy of multi-vector retrieval and the practical demands of large-scale deployment, offering a new path towards efficient and interpretable multimodal information systems. |
| title | Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations |
| topic | Computation and Language Information Retrieval |
| url | https://arxiv.org/abs/2603.01666 |