Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2509.14233 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866914176787546112 |
|---|---|
| author | Apertus, Project Hernández-Cano, Alejandro Hägele, Alexander Huang, Allen Hao Romanou, Angelika Solergibert, Antoni-Joan Pasztor, Barna Messmer, Bettina Garbaya, Dhia Ďurech, Eduard Frank Hakimi, Ido Giraldo, Juan García Ismayilzada, Mete Foroutan, Negar Moalla, Skander Chen, Tiancheng Sabolčec, Vinko Xu, Yixuan Aerni, Michael AlKhamissi, Badr Mariñas, Inés Altemir Amani, Mohammad Hossein Ansaripour, Matin Badanin, Ilia Benoit, Harold Boros, Emanuela Browning, Nicholas Bösch, Fabian Böther, Maximilian Canova, Niklas Challier, Camille Charmillot, Clement Coles, Jonathan Deriu, Jan Devos, Arnout Drescher, Lukas Dzenhaliou, Daniil Ehrmann, Maud Fan, Dongyang Fan, Simin Gao, Silin Gila, Miguel Grandury, María Hashemi, Diba Hoyle, Alexander Jiang, Jiaming Klein, Mark Kucharavy, Andrei Kucherenko, Anastasiia Lübeck, Frederike Machacek, Roman Manitaras, Theofilos Marfurt, Andreas Matoba, Kyle Matrenok, Simon Mendonça, Henrique Mohamed, Fawzi Roberto Montariol, Syrielle Mouchel, Luca Najem-Meyer, Sven Ni, Jingwei Oliva, Gennaro Pagliardini, Matteo Palme, Elia Panferov, Andrei Paoletti, Léo Passerini, Marco Pavlov, Ivan Poiroux, Auguste Ponkshe, Kaustubh Ranchin, Nathan Rando, Javi Sauser, Mathieu Saydaliev, Jakhongir Sayfiddinov, Muhammad Ali Schneider, Marian Schuppli, Stefano Scialanga, Marco Semenov, Andrei Shridhar, Kumar Singhal, Raghav Sotnikova, Anna Sternfeld, Alexander Tarun, Ayush Kumar Teiletche, Paul Vamvas, Jannis Yao, Xiaozhe Zhao, Hao Ilic, Alexander Klimovic, Ana Krause, Andreas Gulcehre, Caglar Rosenthal, David Ash, Elliott Tramèr, Florian VandeVondele, Joost Veraldi, Livio Rajman, Martin Schulthess, Thomas Hoefler, Torsten Bosselut, Antoine Jaggi, Martin Schlag, Imanol |
| author_facet | Apertus, Project Hernández-Cano, Alejandro Hägele, Alexander Huang, Allen Hao Romanou, Angelika Solergibert, Antoni-Joan Pasztor, Barna Messmer, Bettina Garbaya, Dhia Ďurech, Eduard Frank Hakimi, Ido Giraldo, Juan García Ismayilzada, Mete Foroutan, Negar Moalla, Skander Chen, Tiancheng Sabolčec, Vinko Xu, Yixuan Aerni, Michael AlKhamissi, Badr Mariñas, Inés Altemir Amani, Mohammad Hossein Ansaripour, Matin Badanin, Ilia Benoit, Harold Boros, Emanuela Browning, Nicholas Bösch, Fabian Böther, Maximilian Canova, Niklas Challier, Camille Charmillot, Clement Coles, Jonathan Deriu, Jan Devos, Arnout Drescher, Lukas Dzenhaliou, Daniil Ehrmann, Maud Fan, Dongyang Fan, Simin Gao, Silin Gila, Miguel Grandury, María Hashemi, Diba Hoyle, Alexander Jiang, Jiaming Klein, Mark Kucharavy, Andrei Kucherenko, Anastasiia Lübeck, Frederike Machacek, Roman Manitaras, Theofilos Marfurt, Andreas Matoba, Kyle Matrenok, Simon Mendonça, Henrique Mohamed, Fawzi Roberto Montariol, Syrielle Mouchel, Luca Najem-Meyer, Sven Ni, Jingwei Oliva, Gennaro Pagliardini, Matteo Palme, Elia Panferov, Andrei Paoletti, Léo Passerini, Marco Pavlov, Ivan Poiroux, Auguste Ponkshe, Kaustubh Ranchin, Nathan Rando, Javi Sauser, Mathieu Saydaliev, Jakhongir Sayfiddinov, Muhammad Ali Schneider, Marian Schuppli, Stefano Scialanga, Marco Semenov, Andrei Shridhar, Kumar Singhal, Raghav Sotnikova, Anna Sternfeld, Alexander Tarun, Ayush Kumar Teiletche, Paul Vamvas, Jannis Yao, Xiaozhe Zhao, Hao Ilic, Alexander Klimovic, Ana Krause, Andreas Gulcehre, Caglar Rosenthal, David Ash, Elliott Tramèr, Florian VandeVondele, Joost Veraldi, Livio Rajman, Martin Schulthess, Thomas Hoefler, Torsten Bosselut, Antoine Jaggi, Martin Schlag, Imanol |
| contents | We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingual representation. Unlike many prior models that release weights without reproducible data pipelines or regard for content-owner rights, Apertus models are pretrained exclusively on openly available data, retroactively respecting `robots.txt` exclusions and filtering for non-permissive, toxic, and personally identifiable content. To mitigate risks of memorization, we adopt the Goldfish objective during pretraining, strongly suppressing verbatim recall of data while retaining downstream task performance. The Apertus models also expand multilingual coverage, training on 15T tokens from over 1800 languages, with ~40% of pretraining data allocated to non-English content. Released at 8B and 70B scales, Apertus approaches state-of-the-art results among fully open models on multilingual benchmarks, rivalling or surpassing open-weight counterparts. Beyond model weights, we release all scientific artifacts from our development cycle with a permissive license, including data preparation scripts, checkpoints, evaluation suites, and training code, enabling transparent audit and extension. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_14233 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Apertus: Democratizing Open and Compliant LLMs for Global Language Environments Apertus, Project Hernández-Cano, Alejandro Hägele, Alexander Huang, Allen Hao Romanou, Angelika Solergibert, Antoni-Joan Pasztor, Barna Messmer, Bettina Garbaya, Dhia Ďurech, Eduard Frank Hakimi, Ido Giraldo, Juan García Ismayilzada, Mete Foroutan, Negar Moalla, Skander Chen, Tiancheng Sabolčec, Vinko Xu, Yixuan Aerni, Michael AlKhamissi, Badr Mariñas, Inés Altemir Amani, Mohammad Hossein Ansaripour, Matin Badanin, Ilia Benoit, Harold Boros, Emanuela Browning, Nicholas Bösch, Fabian Böther, Maximilian Canova, Niklas Challier, Camille Charmillot, Clement Coles, Jonathan Deriu, Jan Devos, Arnout Drescher, Lukas Dzenhaliou, Daniil Ehrmann, Maud Fan, Dongyang Fan, Simin Gao, Silin Gila, Miguel Grandury, María Hashemi, Diba Hoyle, Alexander Jiang, Jiaming Klein, Mark Kucharavy, Andrei Kucherenko, Anastasiia Lübeck, Frederike Machacek, Roman Manitaras, Theofilos Marfurt, Andreas Matoba, Kyle Matrenok, Simon Mendonça, Henrique Mohamed, Fawzi Roberto Montariol, Syrielle Mouchel, Luca Najem-Meyer, Sven Ni, Jingwei Oliva, Gennaro Pagliardini, Matteo Palme, Elia Panferov, Andrei Paoletti, Léo Passerini, Marco Pavlov, Ivan Poiroux, Auguste Ponkshe, Kaustubh Ranchin, Nathan Rando, Javi Sauser, Mathieu Saydaliev, Jakhongir Sayfiddinov, Muhammad Ali Schneider, Marian Schuppli, Stefano Scialanga, Marco Semenov, Andrei Shridhar, Kumar Singhal, Raghav Sotnikova, Anna Sternfeld, Alexander Tarun, Ayush Kumar Teiletche, Paul Vamvas, Jannis Yao, Xiaozhe Zhao, Hao Ilic, Alexander Klimovic, Ana Krause, Andreas Gulcehre, Caglar Rosenthal, David Ash, Elliott Tramèr, Florian VandeVondele, Joost Veraldi, Livio Rajman, Martin Schulthess, Thomas Hoefler, Torsten Bosselut, Antoine Jaggi, Martin Schlag, Imanol Computation and Language Artificial Intelligence Machine Learning We present Apertus, a fully open suite of large language models (LLMs) designed to address two systemic shortcomings in today's open model ecosystem: data compliance and multilingual representation. Unlike many prior models that release weights without reproducible data pipelines or regard for content-owner rights, Apertus models are pretrained exclusively on openly available data, retroactively respecting `robots.txt` exclusions and filtering for non-permissive, toxic, and personally identifiable content. To mitigate risks of memorization, we adopt the Goldfish objective during pretraining, strongly suppressing verbatim recall of data while retaining downstream task performance. The Apertus models also expand multilingual coverage, training on 15T tokens from over 1800 languages, with ~40% of pretraining data allocated to non-English content. Released at 8B and 70B scales, Apertus approaches state-of-the-art results among fully open models on multilingual benchmarks, rivalling or surpassing open-weight counterparts. Beyond model weights, we release all scientific artifacts from our development cycle with a permissive license, including data preparation scripts, checkpoints, evaluation suites, and training code, enabling transparent audit and extension. |
| title | Apertus: Democratizing Open and Compliant LLMs for Global Language Environments |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2509.14233 |