DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2026
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866915954559025152 |
|---|---|
| author | Choi, Joonmyung Lee, Sanghyeok Kim, Jongha Kim, Sehyung Ko, Dohwan Kil, Jihyung Kim, Hyunwoo J. |
| author_facet | Choi, Joonmyung Lee, Sanghyeok Kim, Jongha Kim, Sehyung Ko, Dohwan Kil, Jihyung Kim, Hyunwoo J. |
| contents | Recent advances in vision-language models have demonstrated remarkable performance across diverse multi-modal tasks, including document question answering that leverages structured visual cues from text, tables, and figures. However, unlike natural images, document images contain large backgrounds and only sparse supporting evidence, leading to the inefficient consumption of substantial computational resources, especially for long documents. We observe that existing token-reduction methods for natural images and videos fall short in utilizing the structural sparsity unique to documents. To address this, we propose DocPrune, a training-free and progressive document token pruning framework designed for efficient long-document understanding. The proposed method preserves only the essential tokens for the task while removing unnecessary ones, such as background or question-irrelevant tokens. Moreover, it automatically selects the appropriate layers to initiate token pruning based on the model's level of comprehension. Our experiments on the M3DocRAG show that DocPrune improves throughput by 3.0x and 3.3x in the encoder and decoder, respectively, while boosting the F1 score by +1.0, achieving both higher accuracy and efficiency without any additional training. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_22281 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning Choi, Joonmyung Lee, Sanghyeok Kim, Jongha Kim, Sehyung Ko, Dohwan Kil, Jihyung Kim, Hyunwoo J. Computer Vision and Pattern Recognition Recent advances in vision-language models have demonstrated remarkable performance across diverse multi-modal tasks, including document question answering that leverages structured visual cues from text, tables, and figures. However, unlike natural images, document images contain large backgrounds and only sparse supporting evidence, leading to the inefficient consumption of substantial computational resources, especially for long documents. We observe that existing token-reduction methods for natural images and videos fall short in utilizing the structural sparsity unique to documents. To address this, we propose DocPrune, a training-free and progressive document token pruning framework designed for efficient long-document understanding. The proposed method preserves only the essential tokens for the task while removing unnecessary ones, such as background or question-irrelevant tokens. Moreover, it automatically selects the appropriate layers to initiate token pruning based on the model's level of comprehension. Our experiments on the M3DocRAG show that DocPrune improves throughput by 3.0x and 3.3x in the encoder and decoder, respectively, while boosting the F1 score by +1.0, achieving both higher accuracy and efficiency without any additional training. |
| title | DocPrune:Efficient Document Question Answering via Background, Question, and Comprehension-aware Token Pruning |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2604.22281 |