You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866916604560801792 |
|---|---|
| author | Lehalleur, Simon Pepin Hoogland, Jesse Farrugia-Roberts, Matthew Wei, Susan Oldenziel, Alexander Gietelink Wang, George Carroll, Liam Murfet, Daniel |
| author_facet | Lehalleur, Simon Pepin Hoogland, Jesse Farrugia-Roberts, Matthew Wei, Susan Oldenziel, Alexander Gietelink Wang, George Carroll, Liam Murfet, Daniel |
| contents | In this position paper, we argue that understanding the relation between structure in the data distribution and structure in trained models is central to AI alignment. First, we discuss how two neural networks can have equivalent performance on the training set but compute their outputs in essentially different ways and thus generalise differently. For this reason, standard testing and evaluation are insufficient for obtaining assurances of safety for widely deployed generally intelligent systems. We argue that to progress beyond evaluation to a robust mathematical science of AI alignment, we need to develop statistical foundations for an understanding of the relation between structure in the data distribution, internal structure in models, and how these structures underlie generalisation. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_05475 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisation Lehalleur, Simon Pepin Hoogland, Jesse Farrugia-Roberts, Matthew Wei, Susan Oldenziel, Alexander Gietelink Wang, George Carroll, Liam Murfet, Daniel Machine Learning In this position paper, we argue that understanding the relation between structure in the data distribution and structure in trained models is central to AI alignment. First, we discuss how two neural networks can have equivalent performance on the training set but compute their outputs in essentially different ways and thus generalise differently. For this reason, standard testing and evaluation are insufficient for obtaining assurances of safety for widely deployed generally intelligent systems. We argue that to progress beyond evaluation to a robust mathematical science of AI alignment, we need to develop statistical foundations for an understanding of the relation between structure in the data distribution, internal structure in models, and how these structures underlie generalisation. |
| title | You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisation |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2502.05475 |