Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2505.07862 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912372749238272 |
|---|---|
| author | Kiruluta, Andrew Lundy, Eric Burity, Priscilla |
| author_facet | Kiruluta, Andrew Lundy, Eric Burity, Priscilla |
| contents | Existing sequence to sequence models for structured language tasks rely heavily on the dot product self attention mechanism, which incurs quadratic complexity in both computation and memory for input length N. We introduce the Graph Wavelet Transformer (GWT), a novel architecture that replaces this bottleneck with a learnable, multi scale wavelet transform defined over an explicit graph Laplacian derived from syntactic or semantic parses. Our analysis shows that multi scale spectral decomposition offers an interpretable, efficient, and expressive alternative to quadratic self attention for graph structured sequence modeling. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_07862 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition Kiruluta, Andrew Lundy, Eric Burity, Priscilla Computation and Language Existing sequence to sequence models for structured language tasks rely heavily on the dot product self attention mechanism, which incurs quadratic complexity in both computation and memory for input length N. We introduce the Graph Wavelet Transformer (GWT), a novel architecture that replaces this bottleneck with a learnable, multi scale wavelet transform defined over an explicit graph Laplacian derived from syntactic or semantic parses. Our analysis shows that multi scale spectral decomposition offers an interpretable, efficient, and expressive alternative to quadratic self attention for graph structured sequence modeling. |
| title | Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2505.07862 |