On the Societal Impact of Open Foundation Models
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866909134969896960 |
|---|---|
| author | Kapoor, Sayash Bommasani, Rishi Klyman, Kevin Longpre, Shayne Ramaswami, Ashwin Cihon, Peter Hopkins, Aspen Bankston, Kevin Biderman, Stella Bogen, Miranda Chowdhury, Rumman Engler, Alex Henderson, Peter Jernite, Yacine Lazar, Seth Maffulli, Stefano Nelson, Alondra Pineau, Joelle Skowron, Aviya Song, Dawn Storchan, Victor Zhang, Daniel Ho, Daniel E. Liang, Percy Narayanan, Arvind |
| author_facet | Kapoor, Sayash Bommasani, Rishi Klyman, Kevin Longpre, Shayne Ramaswami, Ashwin Cihon, Peter Hopkins, Aspen Bankston, Kevin Biderman, Stella Bogen, Miranda Chowdhury, Rumman Engler, Alex Henderson, Peter Jernite, Yacine Lazar, Seth Maffulli, Stefano Nelson, Alondra Pineau, Joelle Skowron, Aviya Song, Dawn Storchan, Victor Zhang, Daniel Ho, Daniel E. Liang, Percy Narayanan, Arvind |
| contents | Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2403_07918 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | On the Societal Impact of Open Foundation Models Kapoor, Sayash Bommasani, Rishi Klyman, Kevin Longpre, Shayne Ramaswami, Ashwin Cihon, Peter Hopkins, Aspen Bankston, Kevin Biderman, Stella Bogen, Miranda Chowdhury, Rumman Engler, Alex Henderson, Peter Jernite, Yacine Lazar, Seth Maffulli, Stefano Nelson, Alondra Pineau, Joelle Skowron, Aviya Song, Dawn Storchan, Victor Zhang, Daniel Ho, Daniel E. Liang, Percy Narayanan, Arvind Computers and Society Artificial Intelligence Machine Learning Foundation models are powerful technologies: how they are released publicly directly shapes their societal impact. In this position paper, we focus on open foundation models, defined here as those with broadly available model weights (e.g. Llama 2, Stable Diffusion XL). We identify five distinctive properties (e.g. greater customizability, poor monitoring) of open foundation models that lead to both their benefits and risks. Open foundation models present significant benefits, with some caveats, that span innovation, competition, the distribution of decision-making power, and transparency. To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk. Across several misuse vectors (e.g. cyberattacks, bioweapons), we find that current research is insufficient to effectively characterize the marginal risk of open foundation models relative to pre-existing technologies. The framework helps explain why the marginal risk is low in some cases, clarifies disagreements about misuse risks by revealing that past work has focused on different subsets of the framework with different assumptions, and articulates a way forward for more constructive debate. Overall, our work helps support a more grounded assessment of the societal impact of open foundation models by outlining what research is needed to empirically validate their theoretical benefits and risks. |
| title | On the Societal Impact of Open Foundation Models |
| topic | Computers and Society Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2403.07918 |