SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866929760024657920 |
|---|---|
| author | Nassar, Ahmed Marafioti, Andres Omenetti, Matteo Lysak, Maksym Livathinos, Nikolaos Auer, Christoph Morin, Lucas de Lima, Rafael Teixeira Kim, Yusik Gurbuz, A. Said Dolfi, Michele Farré, Miquel Staar, Peter W. J. |
| author_facet | Nassar, Ahmed Marafioti, Andres Omenetti, Matteo Lysak, Maksym Livathinos, Nikolaos Auer, Christoph Morin, Lucas de Lima, Rafael Teixeira Kim, Yusik Gurbuz, A. Said Dolfi, Michele Farré, Miquel Staar, Peter W. J. |
| contents | We introduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms -- significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_11576 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion Nassar, Ahmed Marafioti, Andres Omenetti, Matteo Lysak, Maksym Livathinos, Nikolaos Auer, Christoph Morin, Lucas de Lima, Rafael Teixeira Kim, Yusik Gurbuz, A. Said Dolfi, Michele Farré, Miquel Staar, Peter W. J. Computer Vision and Pattern Recognition We introduce SmolDocling, an ultra-compact vision-language model targeting end-to-end document conversion. Our model comprehensively processes entire pages by generating DocTags, a new universal markup format that captures all page elements in their full context with location. Unlike existing approaches that rely on large foundational models, or ensemble solutions that rely on handcrafted pipelines of multiple specialized models, SmolDocling offers an end-to-end conversion for accurately capturing content, structure and spatial location of document elements in a 256M parameters vision-language model. SmolDocling exhibits robust performance in correctly reproducing document features such as code listings, tables, equations, charts, lists, and more across a diverse range of document types including business documents, academic papers, technical reports, patents, and forms -- significantly extending beyond the commonly observed focus on scientific papers. Additionally, we contribute novel publicly sourced datasets for charts, tables, equations, and code recognition. Experimental results demonstrate that SmolDocling competes with other Vision Language Models that are up to 27 times larger in size, while reducing computational requirements substantially. The model is currently available, datasets will be publicly available soon. |
| title | SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2503.11576 |