Uloženo v:
| Hlavní autoři: | , |
|---|---|
| Médium: | Recurso digital |
| Jazyk: | |
| Vydáno: |
Zenodo
2025
|
| Témata: | |
| On-line přístup: | https://doi.org/10.5281/zenodo.17680483 |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
Obsah:
- <p><strong>Abstract</strong><br>This position paper argues that standard Transformers suffer from a fundamental structural inefficiency we term the “Semantic Alignment Tax.”<br>This tax represents the prohibitive optimization<br>cost of learning a coherent relational geometry<br>from a chaotic random initialization. To isolate<br>this phenomenon, we employ Iterative Semantic<br>Map Refinement (ISMR) as a diagnostic protocol.<br>By applying this probe to both deep and wide architectures, we uncover a critical invariance: the<br>alignment tax constitutes a fixed geometric barrier<br>that persists regardless of model depth or parameter ratio. Our findings validate the “Sculptor’s<br>Dilemma.” demonstrating that deep reasoning<br>layers cannot effectively refine representations<br>that are initially incoherent without expending<br>a significant portion of the compute budget on<br>geometric alignment. We contend that current<br>scaling strategies merely mask this tax rather than<br>solving it, a strategy that becomes untenable for<br>low-resource tasks. We call for a shift toward<br>“Representation-First” architectures that enforce<br>relational geometry at initialization.</p>