DocSLM: A Small Vision-Language Model for Long Multimodal Document Understanding
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866908668850601984 |
|---|---|
| author | Hannan, Tanveer Mallios, Dimitrios Pathak, Parth Sardari, Faegheh Seidl, Thomas Bertasius, Gedas Fayyaz, Mohsen Sengupta, Sunando |
| author_facet | Hannan, Tanveer Mallios, Dimitrios Pathak, Parth Sardari, Faegheh Seidl, Thomas Bertasius, Gedas Fayyaz, Mohsen Sengupta, Sunando |
| contents | Large Vision-Language Models (LVLMs) have demonstrated strong multimodal reasoning capabilities on long and complex documents. However, their high memory footprint makes them impractical for deployment on resource-constrained edge devices. We present DocSLM, an efficient Small Vision-Language Model designed for long-document understanding under constrained memory resources. DocSLM incorporates a Hierarchical Multimodal Compressor that jointly encodes visual, textual, and layout information from each page into a fixed-length sequence, greatly reducing memory consumption while preserving both local and global semantics. To enable scalable processing over arbitrarily long inputs, we introduce a Streaming Abstention mechanism that operates on document segments sequentially and filters low-confidence responses using an entropy-based uncertainty calibrator. Across multiple long multimodal document benchmarks, DocSLM matches or surpasses state-of-the-art methods while using 82\% fewer visual tokens, 75\% fewer parameters, and 71\% lower latency, delivering reliable multimodal document understanding on lightweight edge devices. Code and Model are available in https://github.com/Tanveer81/DocSLM.git. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_11313 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DocSLM: A Small Vision-Language Model for Long Multimodal Document Understanding Hannan, Tanveer Mallios, Dimitrios Pathak, Parth Sardari, Faegheh Seidl, Thomas Bertasius, Gedas Fayyaz, Mohsen Sengupta, Sunando Computer Vision and Pattern Recognition Large Vision-Language Models (LVLMs) have demonstrated strong multimodal reasoning capabilities on long and complex documents. However, their high memory footprint makes them impractical for deployment on resource-constrained edge devices. We present DocSLM, an efficient Small Vision-Language Model designed for long-document understanding under constrained memory resources. DocSLM incorporates a Hierarchical Multimodal Compressor that jointly encodes visual, textual, and layout information from each page into a fixed-length sequence, greatly reducing memory consumption while preserving both local and global semantics. To enable scalable processing over arbitrarily long inputs, we introduce a Streaming Abstention mechanism that operates on document segments sequentially and filters low-confidence responses using an entropy-based uncertainty calibrator. Across multiple long multimodal document benchmarks, DocSLM matches or surpasses state-of-the-art methods while using 82\% fewer visual tokens, 75\% fewer parameters, and 71\% lower latency, delivering reliable multimodal document understanding on lightweight edge devices. Code and Model are available in https://github.com/Tanveer81/DocSLM.git. |
| title | DocSLM: A Small Vision-Language Model for Long Multimodal Document Understanding |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2511.11313 |