Instella: Fully Open Language Models with Stellar Performance
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912707501883392 |
|---|---|
| author | Liu, Jiang Wu, Jialian Yu, Xiaodong Su, Yusheng Mishra, Prakamya Ramesh, Gowtham Ranjan, Sudhanshu Manem, Chaitanya Sun, Ximeng Wang, Ze Brahma, Pratik Prabhanjan Liu, Zicheng Barsoum, Emad |
| author_facet | Liu, Jiang Wu, Jialian Yu, Xiaodong Su, Yusheng Mishra, Prakamya Ramesh, Gowtham Ranjan, Sudhanshu Manem, Chaitanya Sun, Ximeng Wang, Ze Brahma, Pratik Prabhanjan Liu, Zicheng Barsoum, Emad |
| contents | Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks, yet the majority of high-performing models remain closed-source or partially open, limiting transparency and reproducibility. In this work, we introduce Instella, a family of fully open three billion parameter language models trained entirely on openly available data and codebase. Powered by AMD Instinct MI300X GPUs, Instella is developed through large-scale pre-training, general-purpose instruction tuning, and alignment with human preferences. Despite using substantially fewer pre-training tokens than many contemporaries, Instella achieves state-of-the-art results among fully open models and is competitive with leading open-weight models of comparable size. We further release two specialized variants: Instella-Long, capable of handling context lengths up to 128K tokens, and Instella-Math, a reasoning-focused model enhanced through supervised fine-tuning and reinforcement learning on mathematical tasks. Together, these contributions establish Instella as a transparent, performant, and versatile alternative for the community, advancing the goal of open and reproducible language modeling research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2511_10628 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Instella: Fully Open Language Models with Stellar Performance Liu, Jiang Wu, Jialian Yu, Xiaodong Su, Yusheng Mishra, Prakamya Ramesh, Gowtham Ranjan, Sudhanshu Manem, Chaitanya Sun, Ximeng Wang, Ze Brahma, Pratik Prabhanjan Liu, Zicheng Barsoum, Emad Computation and Language Artificial Intelligence Machine Learning Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks, yet the majority of high-performing models remain closed-source or partially open, limiting transparency and reproducibility. In this work, we introduce Instella, a family of fully open three billion parameter language models trained entirely on openly available data and codebase. Powered by AMD Instinct MI300X GPUs, Instella is developed through large-scale pre-training, general-purpose instruction tuning, and alignment with human preferences. Despite using substantially fewer pre-training tokens than many contemporaries, Instella achieves state-of-the-art results among fully open models and is competitive with leading open-weight models of comparable size. We further release two specialized variants: Instella-Long, capable of handling context lengths up to 128K tokens, and Instella-Math, a reasoning-focused model enhanced through supervised fine-tuning and reinforcement learning on mathematical tasks. Together, these contributions establish Instella as a transparent, performant, and versatile alternative for the community, advancing the goal of open and reproducible language modeling research. |
| title | Instella: Fully Open Language Models with Stellar Performance |
| topic | Computation and Language Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2511.10628 |