Training Foundation Models as Data Compression: On Information, Model Weights and Copyright Law
Fuente:
arXiv
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910870938845184 |
|---|---|
| author | Franceschelli, Giorgio Cevenini, Claudia Musolesi, Mirco |
| author_facet | Franceschelli, Giorgio Cevenini, Claudia Musolesi, Mirco |
| contents | The training process of foundation models as for other classes of deep learning systems is based on minimizing the reconstruction error over a training set. For this reason, they are susceptible to the memorization and subsequent reproduction of training samples. In this paper, we introduce a training-as-compressing perspective, wherein the model's weights embody a compressed representation of the training data. From a copyright standpoint, this point of view implies that the weights can be considered a reproduction or, more likely, a derivative work of a potentially protected set of works. We investigate the technical and legal challenges that emerge from this framing of the copyright of outputs generated by foundation models, including their implications for practitioners and researchers. We demonstrate that adopting an information-centric approach to the problem presents a promising pathway for tackling these emerging complex legal issues. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2407_13493 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Training Foundation Models as Data Compression: On Information, Model Weights and Copyright Law Franceschelli, Giorgio Cevenini, Claudia Musolesi, Mirco Computers and Society Artificial Intelligence Machine Learning The training process of foundation models as for other classes of deep learning systems is based on minimizing the reconstruction error over a training set. For this reason, they are susceptible to the memorization and subsequent reproduction of training samples. In this paper, we introduce a training-as-compressing perspective, wherein the model's weights embody a compressed representation of the training data. From a copyright standpoint, this point of view implies that the weights can be considered a reproduction or, more likely, a derivative work of a potentially protected set of works. We investigate the technical and legal challenges that emerge from this framing of the copyright of outputs generated by foundation models, including their implications for practitioners and researchers. We demonstrate that adopting an information-centric approach to the problem presents a promising pathway for tackling these emerging complex legal issues. |
| title | Training Foundation Models as Data Compression: On Information, Model Weights and Copyright Law |
| topic | Computers and Society Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2407.13493 |