Saved in:
| Main Author: | |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.15830 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909948718350336 |
|---|---|
| author | Di Sipio, Riccardo |
| author_facet | Di Sipio, Riccardo |
| contents | Optimization in large language models (LLMs) unfolds over high-dimensional parameter spaces with non-Euclidean structure. Information geometry frames this landscape using the Fisher information metric, enabling more principled learning via natural gradient descent. Though often impractical, this geometric lens clarifies phenomena such as sharp minima, generalization, and observed scaling laws. We argue that curvature-based approaches deepen our understanding of LLM training. Finally, we speculate on quantum analogies based on the Fubini-Study metric and Quantum Fisher Information, hinting at efficient optimization in quantum-enhanced systems. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_15830 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Rethinking LLM Training through Information Geometry and Quantum Metrics Di Sipio, Riccardo Computation and Language Quantum Physics I.2; I.7 Optimization in large language models (LLMs) unfolds over high-dimensional parameter spaces with non-Euclidean structure. Information geometry frames this landscape using the Fisher information metric, enabling more principled learning via natural gradient descent. Though often impractical, this geometric lens clarifies phenomena such as sharp minima, generalization, and observed scaling laws. We argue that curvature-based approaches deepen our understanding of LLM training. Finally, we speculate on quantum analogies based on the Fubini-Study metric and Quantum Fisher Information, hinting at efficient optimization in quantum-enhanced systems. |
| title | Rethinking LLM Training through Information Geometry and Quantum Metrics |
| topic | Computation and Language Quantum Physics I.2; I.7 |
| url | https://arxiv.org/abs/2506.15830 |