Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2410.07073 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912067992158208 |
|---|---|
| author | Agrawal, Pravesh Antoniak, Szymon Hanna, Emma Bou Bout, Baptiste Chaplot, Devendra Chudnovsky, Jessica Costa, Diogo De Monicault, Baudouin Garg, Saurabh Gervet, Theophile Ghosh, Soham Héliou, Amélie Jacob, Paul Jiang, Albert Q. Khandelwal, Kartik Lacroix, Timothée Lample, Guillaume Casas, Diego Las Lavril, Thibaut Scao, Teven Le Lo, Andy Marshall, William Martin, Louis Mensch, Arthur Muddireddy, Pavankumar Nemychnikova, Valera Pellat, Marie Von Platen, Patrick Raghuraman, Nikhil Rozière, Baptiste Sablayrolles, Alexandre Saulnier, Lucile Sauvestre, Romain Shang, Wendy Soletskyi, Roman Stewart, Lawrence Stock, Pierre Studnia, Joachim Subramanian, Sandeep Vaze, Sagar Wang, Thomas Yang, Sophia |
| author_facet | Agrawal, Pravesh Antoniak, Szymon Hanna, Emma Bou Bout, Baptiste Chaplot, Devendra Chudnovsky, Jessica Costa, Diogo De Monicault, Baudouin Garg, Saurabh Gervet, Theophile Ghosh, Soham Héliou, Amélie Jacob, Paul Jiang, Albert Q. Khandelwal, Kartik Lacroix, Timothée Lample, Guillaume Casas, Diego Las Lavril, Thibaut Scao, Teven Le Lo, Andy Marshall, William Martin, Louis Mensch, Arthur Muddireddy, Pavankumar Nemychnikova, Valera Pellat, Marie Von Platen, Patrick Raghuraman, Nikhil Rozière, Baptiste Sablayrolles, Alexandre Saulnier, Lucile Sauvestre, Romain Shang, Wendy Soletskyi, Roman Stewart, Lawrence Stock, Pierre Studnia, Joachim Subramanian, Sandeep Vaze, Sagar Wang, Thomas Yang, Sophia |
| contents | We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpassing a number of larger models. Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks. Pixtral uses a new vision encoder trained from scratch, which allows it to ingest images at their natural resolution and aspect ratio. This gives users flexibility on the number of tokens used to process an image. Pixtral is also able to process any number of images in its long context window of 128K tokens. Pixtral 12B substanially outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B). It also outperforms much larger open models like Llama-3.2 90B while being 7x smaller. We further contribute an open-source benchmark, MM-MT-Bench, for evaluating vision-language models in practical scenarios, and provide detailed analysis and code for standardized evaluation protocols for multimodal LLMs. Pixtral-12B is released under Apache 2.0 license. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_07073 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Pixtral 12B Agrawal, Pravesh Antoniak, Szymon Hanna, Emma Bou Bout, Baptiste Chaplot, Devendra Chudnovsky, Jessica Costa, Diogo De Monicault, Baudouin Garg, Saurabh Gervet, Theophile Ghosh, Soham Héliou, Amélie Jacob, Paul Jiang, Albert Q. Khandelwal, Kartik Lacroix, Timothée Lample, Guillaume Casas, Diego Las Lavril, Thibaut Scao, Teven Le Lo, Andy Marshall, William Martin, Louis Mensch, Arthur Muddireddy, Pavankumar Nemychnikova, Valera Pellat, Marie Von Platen, Patrick Raghuraman, Nikhil Rozière, Baptiste Sablayrolles, Alexandre Saulnier, Lucile Sauvestre, Romain Shang, Wendy Soletskyi, Roman Stewart, Lawrence Stock, Pierre Studnia, Joachim Subramanian, Sandeep Vaze, Sagar Wang, Thomas Yang, Sophia Computer Vision and Pattern Recognition Computation and Language We introduce Pixtral-12B, a 12--billion-parameter multimodal language model. Pixtral-12B is trained to understand both natural images and documents, achieving leading performance on various multimodal benchmarks, surpassing a number of larger models. Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks. Pixtral uses a new vision encoder trained from scratch, which allows it to ingest images at their natural resolution and aspect ratio. This gives users flexibility on the number of tokens used to process an image. Pixtral is also able to process any number of images in its long context window of 128K tokens. Pixtral 12B substanially outperforms other open models of similar sizes (Llama-3.2 11B \& Qwen-2-VL 7B). It also outperforms much larger open models like Llama-3.2 90B while being 7x smaller. We further contribute an open-source benchmark, MM-MT-Bench, for evaluating vision-language models in practical scenarios, and provide detailed analysis and code for standardized evaluation protocols for multimodal LLMs. Pixtral-12B is released under Apache 2.0 license. |
| title | Pixtral 12B |
| topic | Computer Vision and Pattern Recognition Computation and Language |
| url | https://arxiv.org/abs/2410.07073 |