Towards Best Practices for Open Datasets for LLM Training
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866917892883218432 |
|---|---|
| author | Baack, Stefan Biderman, Stella Odrozek, Kasia Skowron, Aviya Bdeir, Ayah Bommarito, Jillian Ding, Jennifer Gahntz, Maximilian Keller, Paul Langlais, Pierre-Carl Lindahl, Greg Majstorovic, Sebastian Marda, Nik Penedo, Guilherme Van Segbroeck, Maarten Wang, Jennifer von Werra, Leandro Baker, Mitchell Belião, Julie Chmielinski, Kasia Fadaee, Marzieh Gutermuth, Lisa Kydlíček, Hynek Leppert, Greg Lewis-Jong, EM Larsen, Solana Longpre, Shayne Lungati, Angela Oduor Miller, Cullen Miller, Victor Ryabinin, Max Siminyu, Kathleen Strait, Andrew Surman, Mark Tumadóttir, Anna Weber, Maurice Weiss, Rebecca White, Lee Wolf, Thomas |
| author_facet | Baack, Stefan Biderman, Stella Odrozek, Kasia Skowron, Aviya Bdeir, Ayah Bommarito, Jillian Ding, Jennifer Gahntz, Maximilian Keller, Paul Langlais, Pierre-Carl Lindahl, Greg Majstorovic, Sebastian Marda, Nik Penedo, Guilherme Van Segbroeck, Maarten Wang, Jennifer von Werra, Leandro Baker, Mitchell Belião, Julie Chmielinski, Kasia Fadaee, Marzieh Gutermuth, Lisa Kydlíček, Hynek Leppert, Greg Lewis-Jong, EM Larsen, Solana Longpre, Shayne Lungati, Angela Oduor Miller, Cullen Miller, Victor Ryabinin, Max Siminyu, Kathleen Strait, Andrew Surman, Mark Tumadóttir, Anna Weber, Maurice Weiss, Rebecca White, Lee Wolf, Thomas |
| contents | Many AI companies are training their large language models (LLMs) on data without the permission of the copyright owners. The permissibility of doing so varies by jurisdiction: in countries like the EU and Japan, this is allowed under certain restrictions, while in the United States, the legal landscape is more ambiguous. Regardless of the legal status, concerns from creative producers have led to several high-profile copyright lawsuits, and the threat of litigation is commonly cited as a reason for the recent trend towards minimizing the information shared about training datasets by both corporate and public interest actors. This trend in limiting data information causes harm by hindering transparency, accountability, and innovation in the broader ecosystem by denying researchers, auditors, and impacted individuals access to the information needed to understand AI models.
While this could be mitigated by training language models on open access and public domain data, at the time of writing, there are no such models (trained at a meaningful scale) due to the substantial technical and sociological challenges in assembling the necessary corpus. These challenges include incomplete and unreliable metadata, the cost and complexity of digitizing physical records, and the diverse set of legal and technical skills required to ensure relevance and responsibility in a quickly changing landscape. Building towards a future where AI systems can be trained on openly licensed data that is responsibly curated and governed requires collaboration across legal, technical, and policy domains, along with investments in metadata standards, digitization, and fostering a culture of openness. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2501_08365 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Towards Best Practices for Open Datasets for LLM Training Baack, Stefan Biderman, Stella Odrozek, Kasia Skowron, Aviya Bdeir, Ayah Bommarito, Jillian Ding, Jennifer Gahntz, Maximilian Keller, Paul Langlais, Pierre-Carl Lindahl, Greg Majstorovic, Sebastian Marda, Nik Penedo, Guilherme Van Segbroeck, Maarten Wang, Jennifer von Werra, Leandro Baker, Mitchell Belião, Julie Chmielinski, Kasia Fadaee, Marzieh Gutermuth, Lisa Kydlíček, Hynek Leppert, Greg Lewis-Jong, EM Larsen, Solana Longpre, Shayne Lungati, Angela Oduor Miller, Cullen Miller, Victor Ryabinin, Max Siminyu, Kathleen Strait, Andrew Surman, Mark Tumadóttir, Anna Weber, Maurice Weiss, Rebecca White, Lee Wolf, Thomas Computers and Society Artificial Intelligence Computation and Language Machine Learning Many AI companies are training their large language models (LLMs) on data without the permission of the copyright owners. The permissibility of doing so varies by jurisdiction: in countries like the EU and Japan, this is allowed under certain restrictions, while in the United States, the legal landscape is more ambiguous. Regardless of the legal status, concerns from creative producers have led to several high-profile copyright lawsuits, and the threat of litigation is commonly cited as a reason for the recent trend towards minimizing the information shared about training datasets by both corporate and public interest actors. This trend in limiting data information causes harm by hindering transparency, accountability, and innovation in the broader ecosystem by denying researchers, auditors, and impacted individuals access to the information needed to understand AI models. While this could be mitigated by training language models on open access and public domain data, at the time of writing, there are no such models (trained at a meaningful scale) due to the substantial technical and sociological challenges in assembling the necessary corpus. These challenges include incomplete and unreliable metadata, the cost and complexity of digitizing physical records, and the diverse set of legal and technical skills required to ensure relevance and responsibility in a quickly changing landscape. Building towards a future where AI systems can be trained on openly licensed data that is responsibly curated and governed requires collaboration across legal, technical, and policy domains, along with investments in metadata standards, digitization, and fostering a culture of openness. |
| title | Towards Best Practices for Open Datasets for LLM Training |
| topic | Computers and Society Artificial Intelligence Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2501.08365 |