Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2025
|
| Soggetti: | |
| Accesso online: | https://arxiv.org/abs/2504.05299 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866916676592730112 |
|---|---|
| author | Marafioti, Andrés Zohar, Orr Farré, Miquel Noyan, Merve Bakouch, Elie Cuenca, Pedro Zakka, Cyril Allal, Loubna Ben Lozhkov, Anton Tazi, Nouamane Srivastav, Vaibhav Lochner, Joshua Larcher, Hugo Morlon, Mathieu Tunstall, Lewis von Werra, Leandro Wolf, Thomas |
| author_facet | Marafioti, Andrés Zohar, Orr Farré, Miquel Noyan, Merve Bakouch, Elie Cuenca, Pedro Zakka, Cyril Allal, Loubna Ben Lozhkov, Anton Tazi, Nouamane Srivastav, Vaibhav Lochner, Joshua Larcher, Hugo Morlon, Mathieu Tunstall, Lewis von Werra, Leandro Wolf, Thomas |
| contents | Large Vision-Language Models (VLMs) deliver exceptional performance but require significant computational resources, limiting their deployment on mobile and edge devices. Smaller VLMs typically mirror design choices of larger models, such as extensive image tokenization, leading to inefficient GPU memory usage and constrained practicality for on-device applications.
We introduce SmolVLM, a series of compact multimodal models specifically engineered for resource-efficient inference. We systematically explore architectural configurations, tokenization strategies, and data curation optimized for low computational overhead. Through this, we identify key design choices that yield substantial performance gains on image and video tasks with minimal memory footprints.
Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during inference and outperforms the 300-times larger Idefics-80B model, despite an 18-month development gap. Our largest model, at 2.2B parameters, rivals state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend beyond static images, demonstrating robust video comprehension capabilities.
Our results emphasize that strategic architectural optimizations, aggressive yet efficient tokenization, and carefully curated training data significantly enhance multimodal performance, facilitating practical, energy-efficient deployments at significantly smaller scales. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_05299 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | SmolVLM: Redefining small and efficient multimodal models Marafioti, Andrés Zohar, Orr Farré, Miquel Noyan, Merve Bakouch, Elie Cuenca, Pedro Zakka, Cyril Allal, Loubna Ben Lozhkov, Anton Tazi, Nouamane Srivastav, Vaibhav Lochner, Joshua Larcher, Hugo Morlon, Mathieu Tunstall, Lewis von Werra, Leandro Wolf, Thomas Artificial Intelligence Computer Vision and Pattern Recognition Large Vision-Language Models (VLMs) deliver exceptional performance but require significant computational resources, limiting their deployment on mobile and edge devices. Smaller VLMs typically mirror design choices of larger models, such as extensive image tokenization, leading to inefficient GPU memory usage and constrained practicality for on-device applications. We introduce SmolVLM, a series of compact multimodal models specifically engineered for resource-efficient inference. We systematically explore architectural configurations, tokenization strategies, and data curation optimized for low computational overhead. Through this, we identify key design choices that yield substantial performance gains on image and video tasks with minimal memory footprints. Our smallest model, SmolVLM-256M, uses less than 1GB GPU memory during inference and outperforms the 300-times larger Idefics-80B model, despite an 18-month development gap. Our largest model, at 2.2B parameters, rivals state-of-the-art VLMs consuming twice the GPU memory. SmolVLM models extend beyond static images, demonstrating robust video comprehension capabilities. Our results emphasize that strategic architectural optimizations, aggressive yet efficient tokenization, and carefully curated training data significantly enhance multimodal performance, facilitating practical, energy-efficient deployments at significantly smaller scales. |
| title | SmolVLM: Redefining small and efficient multimodal models |
| topic | Artificial Intelligence Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2504.05299 |