A Multilayer Causal Framework for Geometric, Informational, and Meaning-Level Structure in Large Language Models
Fuente:
Zenodo
Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | English |
| Published: |
Zenodo
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866902004161314816 |
|---|---|
| author | Kawai, Ryoya |
| author_facet | Kawai, Ryoya |
| contents | <p>This work proposes a unified multi-layer causal framework connecting the geometric structure of large language model (LLM) embeddings, the information-flow properties of attention mechanisms, and the emergence of semantic meaning during reasoning.</p> <p>The study introduces three interacting layers—<strong>physical (geometry/topology), information (directed correlations), and meaning (causal constraints)</strong>—and demonstrates that standard transformer embeddings can be analyzed using tools such as persistent homology, local intrinsic-dimension estimation, and density-matrix-like representations.</p> <p>Experimental results using GPT-2 small reveal:</p> <ul> <li> <p>non-trivial H1H1 loops in embedding space,</p> </li> <li> <p>non-uniform low-dimensional structure via TwoNN,</p> </li> <li> <p>attention matrices acting as stochastic operators with meaningful spectral structure.</p> </li> </ul> <p>All code necessary for full reproducibility is included in the appendix.<br>The aim of this work is to encourage further exploration of the deep structure of learned representations and their potential connections to physics, information theory, and semantic emergence.<br><br></p> <p>For readers interested in the broader theoretical context of this work, a companion whitepaper provides an integrated view of how quantum phase structure, geometric manifolds, and topological invariants may relate to semantic stability in AI systems:</p> <p><strong><em>Meaning Unification Framework: A Dual-Tier Whitepaper Connecting Quantum–Geometric Structures and AI Semantic Stability</em></strong><br>URL: <a href="https://zenodo.org/records/17650669" target="_new" rel="noopener">https://zenodo.org/records/17650669</a></p> <p>This whitepaper positions the present paper as part of a larger two-tier research program, linking foundational mathematical structures (Tier I) with applications to alignment, semantic stability, and internal phenomenology (Tier II).</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17609974 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | A Multilayer Causal Framework for Geometric, Informational, and Meaning-Level Structure in Large Language Models Kawai, Ryoya LLM, topology, quantum-inspired models, causal emergence <p>This work proposes a unified multi-layer causal framework connecting the geometric structure of large language model (LLM) embeddings, the information-flow properties of attention mechanisms, and the emergence of semantic meaning during reasoning.</p> <p>The study introduces three interacting layers—<strong>physical (geometry/topology), information (directed correlations), and meaning (causal constraints)</strong>—and demonstrates that standard transformer embeddings can be analyzed using tools such as persistent homology, local intrinsic-dimension estimation, and density-matrix-like representations.</p> <p>Experimental results using GPT-2 small reveal:</p> <ul> <li> <p>non-trivial H1H1 loops in embedding space,</p> </li> <li> <p>non-uniform low-dimensional structure via TwoNN,</p> </li> <li> <p>attention matrices acting as stochastic operators with meaningful spectral structure.</p> </li> </ul> <p>All code necessary for full reproducibility is included in the appendix.<br>The aim of this work is to encourage further exploration of the deep structure of learned representations and their potential connections to physics, information theory, and semantic emergence.<br><br></p> <p>For readers interested in the broader theoretical context of this work, a companion whitepaper provides an integrated view of how quantum phase structure, geometric manifolds, and topological invariants may relate to semantic stability in AI systems:</p> <p><strong><em>Meaning Unification Framework: A Dual-Tier Whitepaper Connecting Quantum–Geometric Structures and AI Semantic Stability</em></strong><br>URL: <a href="https://zenodo.org/records/17650669" target="_new" rel="noopener">https://zenodo.org/records/17650669</a></p> <p>This whitepaper positions the present paper as part of a larger two-tier research program, linking foundational mathematical structures (Tier I) with applications to alignment, semantic stability, and internal phenomenology (Tier II).</p> |
| title | A Multilayer Causal Framework for Geometric, Informational, and Meaning-Level Structure in Large Language Models |
| topic | LLM, topology, quantum-inspired models, causal emergence |
| url | https://doi.org/10.5281/zenodo.17609974 |