Representation Learning on a Random Lattice
Fuente:
arXiv
Guardado en:
| Autor principal: | |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910921579823104 |
|---|---|
| author | Brill, Aryeh |
| author_facet | Brill, Aryeh |
| contents | Decomposing a deep neural network's learned representations into interpretable features could greatly enhance its safety and reliability. To better understand features, we adopt a geometric perspective, viewing them as a learned coordinate system for mapping an embedded data distribution. We motivate a model of a generic data distribution as a random lattice and analyze its properties using percolation theory. Learned features are categorized into context, component, and surface features. The model is qualitatively consistent with recent findings in mechanistic interpretability and suggests directions for future research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2504_20197 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Representation Learning on a Random Lattice Brill, Aryeh Machine Learning Disordered Systems and Neural Networks Artificial Intelligence Decomposing a deep neural network's learned representations into interpretable features could greatly enhance its safety and reliability. To better understand features, we adopt a geometric perspective, viewing them as a learned coordinate system for mapping an embedded data distribution. We motivate a model of a generic data distribution as a random lattice and analyze its properties using percolation theory. Learned features are categorized into context, component, and surface features. The model is qualitatively consistent with recent findings in mechanistic interpretability and suggests directions for future research. |
| title | Representation Learning on a Random Lattice |
| topic | Machine Learning Disordered Systems and Neural Networks Artificial Intelligence |
| url | https://arxiv.org/abs/2504.20197 |