Salvato in:
| Autori principali: | , |
|---|---|
| Natura: | Recurso digital |
| Lingua: | |
| Pubblicazione: |
Zenodo
2025
|
| Soggetti: | |
| Accesso online: | https://doi.org/10.5281/zenodo.17680483 |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
Sommario:
- <p><strong>Abstract</strong><br>This position paper argues that standard Transformers suffer from a fundamental structural inefficiency we term the “Semantic Alignment Tax.”<br>This tax represents the prohibitive optimization<br>cost of learning a coherent relational geometry<br>from a chaotic random initialization. To isolate<br>this phenomenon, we employ Iterative Semantic<br>Map Refinement (ISMR) as a diagnostic protocol.<br>By applying this probe to both deep and wide architectures, we uncover a critical invariance: the<br>alignment tax constitutes a fixed geometric barrier<br>that persists regardless of model depth or parameter ratio. Our findings validate the “Sculptor’s<br>Dilemma.” demonstrating that deep reasoning<br>layers cannot effectively refine representations<br>that are initially incoherent without expending<br>a significant portion of the compute budget on<br>geometric alignment. We contend that current<br>scaling strategies merely mask this tax rather than<br>solving it, a strategy that becomes untenable for<br>low-resource tasks. We call for a shift toward<br>“Representation-First” architectures that enforce<br>relational geometry at initialization.</p>