Buchumschlag

OLMo: Accelerating the Science of Language Models

Fuente: arXiv

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Groeneveld, Dirk, Beltagy, Iz, Walsh, Pete, Bhagia, Akshita, Kinney, Rodney, Tafjord, Oyvind, Jha, Ananya Harsh, Ivison, Hamish, Magnusson, Ian, Wang, Yizhong, Arora, Shane, Atkinson, David, Authur, Russell, Chandu, Khyathi Raghavi, Cohan, Arman, Dumas, Jennifer, Elazar, Yanai, Gu, Yuling, Hessel, Jack, Khot, Tushar, Merrill, William, Morrison, Jacob, Muennighoff, Niklas, Naik, Aakanksha, Nam, Crystal, Peters, Matthew E., Pyatkin, Valentina, Ravichander, Abhilasha, Schwenk, Dustin, Shah, Saurabh, Smith, Will, Strubell, Emma, Subramani, Nishant, Wortsman, Mitchell, Dasigi, Pradeep, Lambert, Nathan, Richardson, Kyle, Zettlemoyer, Luke, Dodge, Jesse, Lo, Kyle, Soldaini, Luca, Smith, Noah A., Hajishirzi, Hannaneh
Format:	Preprint
Veröffentlicht:	2024
Schlagworte:	Computation and Language
Online-Zugang:	Acceder al recurso
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

_version_	1866911910366019584
author	Groeneveld, Dirk Beltagy, Iz Walsh, Pete Bhagia, Akshita Kinney, Rodney Tafjord, Oyvind Jha, Ananya Harsh Ivison, Hamish Magnusson, Ian Wang, Yizhong Arora, Shane Atkinson, David Authur, Russell Chandu, Khyathi Raghavi Cohan, Arman Dumas, Jennifer Elazar, Yanai Gu, Yuling Hessel, Jack Khot, Tushar Merrill, William Morrison, Jacob Muennighoff, Niklas Naik, Aakanksha Nam, Crystal Peters, Matthew E. Pyatkin, Valentina Ravichander, Abhilasha Schwenk, Dustin Shah, Saurabh Smith, Will Strubell, Emma Subramani, Nishant Wortsman, Mitchell Dasigi, Pradeep Lambert, Nathan Richardson, Kyle Zettlemoyer, Luke Dodge, Jesse Lo, Kyle Soldaini, Luca Smith, Noah A. Hajishirzi, Hannaneh
author_facet	Groeneveld, Dirk Beltagy, Iz Walsh, Pete Bhagia, Akshita Kinney, Rodney Tafjord, Oyvind Jha, Ananya Harsh Ivison, Hamish Magnusson, Ian Wang, Yizhong Arora, Shane Atkinson, David Authur, Russell Chandu, Khyathi Raghavi Cohan, Arman Dumas, Jennifer Elazar, Yanai Gu, Yuling Hessel, Jack Khot, Tushar Merrill, William Morrison, Jacob Muennighoff, Niklas Naik, Aakanksha Nam, Crystal Peters, Matthew E. Pyatkin, Valentina Ravichander, Abhilasha Schwenk, Dustin Shah, Saurabh Smith, Will Strubell, Emma Subramani, Nishant Wortsman, Mitchell Dasigi, Pradeep Lambert, Nathan Richardson, Kyle Zettlemoyer, Luke Dodge, Jesse Lo, Kyle Soldaini, Luca Smith, Noah A. Hajishirzi, Hannaneh
contents	Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, we have built OLMo, a competitive, truly Open Language Model, to enable the scientific study of language models. Unlike most prior efforts that have only released model weights and inference code, we release OLMo alongside open training data and training and evaluation code. We hope this release will empower the open research community and inspire a new wave of innovation.
format	Preprint
id	arxiv_https___arxiv_org_abs_2402_00838
institution	arXiv
publishDate	2024
record_format	arxiv
spellingShingle	OLMo: Accelerating the Science of Language Models Groeneveld, Dirk Beltagy, Iz Walsh, Pete Bhagia, Akshita Kinney, Rodney Tafjord, Oyvind Jha, Ananya Harsh Ivison, Hamish Magnusson, Ian Wang, Yizhong Arora, Shane Atkinson, David Authur, Russell Chandu, Khyathi Raghavi Cohan, Arman Dumas, Jennifer Elazar, Yanai Gu, Yuling Hessel, Jack Khot, Tushar Merrill, William Morrison, Jacob Muennighoff, Niklas Naik, Aakanksha Nam, Crystal Peters, Matthew E. Pyatkin, Valentina Ravichander, Abhilasha Schwenk, Dustin Shah, Saurabh Smith, Will Strubell, Emma Subramani, Nishant Wortsman, Mitchell Dasigi, Pradeep Lambert, Nathan Richardson, Kyle Zettlemoyer, Luke Dodge, Jesse Lo, Kyle Soldaini, Luca Smith, Noah A. Hajishirzi, Hannaneh Computation and Language Language models (LMs) have become ubiquitous in both NLP research and in commercial product offerings. As their commercial importance has surged, the most powerful models have become closed off, gated behind proprietary interfaces, with important details of their training data, architectures, and development undisclosed. Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs. To this end, we have built OLMo, a competitive, truly Open Language Model, to enable the scientific study of language models. Unlike most prior efforts that have only released model weights and inference code, we release OLMo alongside open training data and training and evaluation code. We hope this release will empower the open research community and inspire a new wave of innovation.
title	OLMo: Accelerating the Science of Language Models
topic	Computation and Language
url	https://arxiv.org/abs/2402.00838