Bridging the Data Provenance Gap Across Text, Speech and Video
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2024
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866912237345570816 |
|---|---|
| author | Longpre, Shayne Singh, Nikhil Cherep, Manuel Tiwary, Kushagra Materzynska, Joanna Brannon, William Mahari, Robert Obeng-Marnu, Naana Dey, Manan Hamdy, Mohammed Saxena, Nayan Anis, Ahmad Mustafa Alghamdi, Emad A. Chien, Vu Minh Yin, Da Qian, Kun Li, Yizhi Liang, Minnie Dinh, An Mohanty, Shrestha Mataciunas, Deividas South, Tobin Zhang, Jianguo Lee, Ariel N. Lund, Campbell S. Klamm, Christopher Sileo, Damien Misra, Diganta Shippole, Enrico Klyman, Kevin Miranda, Lester JV Muennighoff, Niklas Ye, Seonghyeon Kim, Seungone Gupta, Vipul Sharma, Vivek Zhou, Xuhui Xiong, Caiming Villa, Luis Biderman, Stella Pentland, Alex Hooker, Sara Kabbara, Jad |
| author_facet | Longpre, Shayne Singh, Nikhil Cherep, Manuel Tiwary, Kushagra Materzynska, Joanna Brannon, William Mahari, Robert Obeng-Marnu, Naana Dey, Manan Hamdy, Mohammed Saxena, Nayan Anis, Ahmad Mustafa Alghamdi, Emad A. Chien, Vu Minh Yin, Da Qian, Kun Li, Yizhi Liang, Minnie Dinh, An Mohanty, Shrestha Mataciunas, Deividas South, Tobin Zhang, Jianguo Lee, Ariel N. Lund, Campbell S. Klamm, Christopher Sileo, Damien Misra, Diganta Shippole, Enrico Klyman, Kevin Miranda, Lester JV Muennighoff, Niklas Ye, Seonghyeon Kim, Seungone Gupta, Vipul Sharma, Vivek Zhou, Xuhui Xiong, Caiming Villa, Luis Biderman, Stella Pentland, Alex Hooker, Sara Kabbara, Jad |
| contents | Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established datasets beyond text. In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities--popular text, speech, and video datasets--from their detailed sourcing trends and use restrictions to their geographical and linguistic representation. Our manual analysis covers nearly 4000 public datasets between 1990-2024, spanning 608 languages, 798 sources, 659 organizations, and 67 countries. We find that multimodal machine learning applications have overwhelmingly turned to web-crawled, synthetic, and social media platforms, such as YouTube, for their training sets, eclipsing all other sources since 2019. Secondly, tracing the chain of dataset derivations we find that while less than 33% of datasets are restrictively licensed, over 80% of the source content in widely-used text, speech, and video datasets, carry non-commercial restrictions. Finally, counter to the rising number of languages and geographies represented in public AI training datasets, our audit demonstrates measures of relative geographical and multilingual representation have failed to significantly improve their coverage since 2013. We believe the breadth of our audit enables us to empirically examine trends in data sourcing, restrictions, and Western-centricity at an ecosystem-level, and that visibility into these questions are essential to progress in responsible AI. As a contribution to ongoing improvements in dataset transparency and responsible use, we release our entire multimodal audit, allowing practitioners to trace data provenance across text, speech, and video. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2412_17847 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Bridging the Data Provenance Gap Across Text, Speech and Video Longpre, Shayne Singh, Nikhil Cherep, Manuel Tiwary, Kushagra Materzynska, Joanna Brannon, William Mahari, Robert Obeng-Marnu, Naana Dey, Manan Hamdy, Mohammed Saxena, Nayan Anis, Ahmad Mustafa Alghamdi, Emad A. Chien, Vu Minh Yin, Da Qian, Kun Li, Yizhi Liang, Minnie Dinh, An Mohanty, Shrestha Mataciunas, Deividas South, Tobin Zhang, Jianguo Lee, Ariel N. Lund, Campbell S. Klamm, Christopher Sileo, Damien Misra, Diganta Shippole, Enrico Klyman, Kevin Miranda, Lester JV Muennighoff, Niklas Ye, Seonghyeon Kim, Seungone Gupta, Vipul Sharma, Vivek Zhou, Xuhui Xiong, Caiming Villa, Luis Biderman, Stella Pentland, Alex Hooker, Sara Kabbara, Jad Artificial Intelligence Computation and Language Computers and Society Machine Learning Multimedia Progress in AI is driven largely by the scale and quality of training data. Despite this, there is a deficit of empirical analysis examining the attributes of well-established datasets beyond text. In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities--popular text, speech, and video datasets--from their detailed sourcing trends and use restrictions to their geographical and linguistic representation. Our manual analysis covers nearly 4000 public datasets between 1990-2024, spanning 608 languages, 798 sources, 659 organizations, and 67 countries. We find that multimodal machine learning applications have overwhelmingly turned to web-crawled, synthetic, and social media platforms, such as YouTube, for their training sets, eclipsing all other sources since 2019. Secondly, tracing the chain of dataset derivations we find that while less than 33% of datasets are restrictively licensed, over 80% of the source content in widely-used text, speech, and video datasets, carry non-commercial restrictions. Finally, counter to the rising number of languages and geographies represented in public AI training datasets, our audit demonstrates measures of relative geographical and multilingual representation have failed to significantly improve their coverage since 2013. We believe the breadth of our audit enables us to empirically examine trends in data sourcing, restrictions, and Western-centricity at an ecosystem-level, and that visibility into these questions are essential to progress in responsible AI. As a contribution to ongoing improvements in dataset transparency and responsible use, we release our entire multimodal audit, allowing practitioners to trace data provenance across text, speech, and video. |
| title | Bridging the Data Provenance Gap Across Text, Speech and Video |
| topic | Artificial Intelligence Computation and Language Computers and Society Machine Learning Multimedia |
| url | https://arxiv.org/abs/2412.17847 |