Spontaneous Semantic Symmetry Breaking: A Higgs Field Theoretic Interpretation of Transformer Dynamics and Regularization
Fuente:
Zenodo
Gespeichert in:
| 1. Verfasser: | |
|---|---|
| Format: | Recurso digital |
| Sprache: | Englisch |
| Veröffentlicht: |
Zenodo
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866901159159005184 |
|---|---|
| author | Yu, Xueen |
| author_facet | Yu, Xueen |
| contents | <p><strong>Abstract:</strong></p> <p>While Large Language Models (LLMs) demonstrate remarkable semantic emergence, the internal dynamics governing the evolution of their representations remain theoretically opaque. Standard regularization methods, such as L2 weight decay, are based on statistical assumptions that suppress parameter norms towards zero. This approach potentially conflicts with the physical intuition that tokens must acquire significant "mass" (magnitude) to encode rich semantic information.</p> <p>We propose a novel perspective based on <strong>Lagrangian Field Theory</strong>, establishing a mapping between the Transformer architecture and the Standard Model of particle physics. We treat token embeddings as fundamental particles and the Attention Mechanism as a <strong>Higgs Field</strong> that induces interaction. We demonstrate that token initialization corresponds to a massless symmetric state, while the training process acts as a <strong>Spontaneous Symmetry Breaking (SSB)</strong> event. Through coupling with the attention field, tokens acquire "Semantic Mass" (defined as the embedding norm) proportional to their contextual importance.</p> <p>Building on this theory, we introduce <strong>Higgs Regularization</strong>, a novel algorithm that replaces the traditional L2 penalty with a physical <strong>"Mexican Hat" Potential</strong>. This potential guides embedding vectors to converge towards a non-zero Vacuum Expectation Value (VEV), physically simulating the process of mass acquisition. Simulations on a miniature Transformer reveal a clear phase transition: high-frequency functional words and core entities acquire distinct, stable masses, while noise tokens remain massless.</p> <p><strong>Key Contributions:</strong></p> <ul> <li> <p><strong>Physics-AI Bridge:</strong> A first-principles derivation mapping the Transformer attention mechanism to the Higgs field interaction.</p> </li> <li> <p><strong>New Algorithm:</strong> <strong>Higgs Regularization</strong>, a drop-in replacement for L2 Weight Decay that improves semantic stability.</p> </li> <li> <p><strong>Dynamic Analysis:</strong> Visualization of the "Semantic Phase Transition" during model training.</p> </li> </ul> <p><strong>Keywords:</strong> Artificial Intelligence, Transformer Dynamics, Higgs Mechanism, Spontaneous Symmetry Breaking, Regularization, AI Physics.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_18397086 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Spontaneous Semantic Symmetry Breaking: A Higgs Field Theoretic Interpretation of Transformer Dynamics and Regularization Yu, Xueen Artificial Intelligence Deep Learning Transformer Large Language Models Regularization Higgs Mechanism Spontaneous Symmetry Breaking AI Physics Field Theory <p><strong>Abstract:</strong></p> <p>While Large Language Models (LLMs) demonstrate remarkable semantic emergence, the internal dynamics governing the evolution of their representations remain theoretically opaque. Standard regularization methods, such as L2 weight decay, are based on statistical assumptions that suppress parameter norms towards zero. This approach potentially conflicts with the physical intuition that tokens must acquire significant "mass" (magnitude) to encode rich semantic information.</p> <p>We propose a novel perspective based on <strong>Lagrangian Field Theory</strong>, establishing a mapping between the Transformer architecture and the Standard Model of particle physics. We treat token embeddings as fundamental particles and the Attention Mechanism as a <strong>Higgs Field</strong> that induces interaction. We demonstrate that token initialization corresponds to a massless symmetric state, while the training process acts as a <strong>Spontaneous Symmetry Breaking (SSB)</strong> event. Through coupling with the attention field, tokens acquire "Semantic Mass" (defined as the embedding norm) proportional to their contextual importance.</p> <p>Building on this theory, we introduce <strong>Higgs Regularization</strong>, a novel algorithm that replaces the traditional L2 penalty with a physical <strong>"Mexican Hat" Potential</strong>. This potential guides embedding vectors to converge towards a non-zero Vacuum Expectation Value (VEV), physically simulating the process of mass acquisition. Simulations on a miniature Transformer reveal a clear phase transition: high-frequency functional words and core entities acquire distinct, stable masses, while noise tokens remain massless.</p> <p><strong>Key Contributions:</strong></p> <ul> <li> <p><strong>Physics-AI Bridge:</strong> A first-principles derivation mapping the Transformer attention mechanism to the Higgs field interaction.</p> </li> <li> <p><strong>New Algorithm:</strong> <strong>Higgs Regularization</strong>, a drop-in replacement for L2 Weight Decay that improves semantic stability.</p> </li> <li> <p><strong>Dynamic Analysis:</strong> Visualization of the "Semantic Phase Transition" during model training.</p> </li> </ul> <p><strong>Keywords:</strong> Artificial Intelligence, Transformer Dynamics, Higgs Mechanism, Spontaneous Symmetry Breaking, Regularization, AI Physics.</p> |
| title | Spontaneous Semantic Symmetry Breaking: A Higgs Field Theoretic Interpretation of Transformer Dynamics and Regularization |
| topic | Artificial Intelligence Deep Learning Transformer Large Language Models Regularization Higgs Mechanism Spontaneous Symmetry Breaking AI Physics Field Theory |
| url | https://doi.org/10.5281/zenodo.18397086 |