Uloženo v:
| Hlavní autor: | |
|---|---|
| Médium: | Recurso digital |
| Jazyk: | angličtina |
| Vydáno: |
Zenodo
2026
|
| Témata: | |
| On-line přístup: | https://doi.org/10.5281/zenodo.19840136 |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
Obsah:
- <p>We introduce Motif-Upcycling, a structure-preserving framework for adapting pretrained Transformer models. The key idea is that common feed-forward modules, including SwiGLU FFNs, can be exactly factorized along their intermediate channel axis into motif-aligned components. With neutral routing, the factorized module computes the same function as the original pretrained block at initialization. We further introduce Scale-Aware Residual Control (SARC), an identity-preserving control motif that modulates the magnitude of trainable residual interventions relative to the residual stream. We also propose Emergence as Coupled Budget Thresholds (ECBT), a conditional model showing that apparent capability cliffs can arise from multiplicatively coupled motif effectiveness curves under uniform budget allocation.</p>