שמור ב:
| מחבר ראשי: | |
|---|---|
| פורמט: | Recurso digital |
| שפה: | |
| יצא לאור: |
Zenodo
2025
|
| נושאים: | |
| גישה מקוונת: | https://doi.org/10.5281/zenodo.18060955 |
| תגים: |
הוספת תג
אין תגיות, היה/י הראשונ/ה לתייג את הרשומה!
|
תוכן הענינים:
- <p>We propose Topology-Aware Attention, a novel paradigm that injects topological structural priors into Transformer attention mechanisms. While standard Transformers compute attention weights solely based on semantic similarity between tokens, our approach incorporates topological centrality measures to identify and emphasize structurally important tokens. </p> <p>Key findings:<br>- Topological features provide 99.6% independent information from semantic embeddings<br>- Topology-enhanced attention significantly increases the correlation between structural importance and attention received (from r=0.41 to r=0.81)<br>- We propose four injection schemes: Attention Bias, Gated Fusion, Centrality Weighting, and Dual-Stream Attention</p> <p>This work was conducted independently by an 18-year-old researcher, representing the first application of topological priors to Transformer attention mechanisms.</p>