Copertina

Beyond Self Attention: A Subquadratic Fourier Wavelet Transformer with Multi Modal Fusion

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Kiruluta, Andrew, Lemos, Andreas, Lundy, Eric
Natura:	Preprint
Pubblicazione:	2021
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attention
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

State Fourier Diffusion Language Model (SFDLM): A Scalable, Novel Iterative Approach to Language Modeling
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Entropic-Time Inference: Self-Organizing Large Language Model Decoding Beyond Attention
di: Kiruluta, Andrew
Pubblicazione: (2026)

Adaptive Two Sided Laplace Transforms: A Learnable, Interpretable, and Scalable Replacement for Self-Attention
di: Kiruluta, Andrew
Pubblicazione: (2025)

A Hybrid Wavelet-Fourier Method for Next-Generation Conditional Diffusion Models
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery
di: Kiruluta, Andrew
Pubblicazione: (2026)

Beyond Neural Networks: Symbolic Reasoning over Wavelet Logic Graph Signals
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Spectral Generative Flow Models: A Physics-Inspired Replacement for Vectorized Large Language Models
di: Kiruluta, Andrew
Pubblicazione: (2026)

Hierarchical Attention Diffusion Networks with Object Priors for Video Change Detection
di: Kiruluta, Andrew, et al.
Pubblicazione: (2024)

From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
di: Kiruluta, Andrew
Pubblicazione: (2026)

History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
di: Kiruluta, Andrew
Pubblicazione: (2026)

Fundamental Limitations on Subquadratic Alternatives to Transformers
di: Alman, Josh, et al.
Pubblicazione: (2024)

From Eigenmodes to Proofs: Integrating Graph Spectral Operators with Symbolic Interpretable Reasoning
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

Structured-Sparse Attention for Entity Tracking with Subquadratic Sequence Complexity
di: Zhao, Hangyue, et al.
Pubblicazione: (2026)

Wavelet Logic Machines: Learning and Reasoning in the Spectral Domain Without Neural Networks
di: Kiruluta, Andrew
Pubblicazione: (2025)

Unsupervised Machine Learning Hybrid Approach Integrating Linear Programming in Loss Function: A Robust Optimization Technique
di: Kiruluta, Andrew, et al.
Pubblicazione: (2024)

Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding
di: Zeris, Athanasios
Pubblicazione: (2026)

Reducing Deep Network Complexity via Sparse Hierarchical Fourier Interaction Networks
di: Kiruluta, Andrew, et al.
Pubblicazione: (2017)

Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
di: Zeris, Athanasios
Pubblicazione: (2026)

A Depression Detection Method Based on Multi-Modal Feature Fusion Using Cross-Attention
di: Li, Shengjie, et al.
Pubblicazione: (2024)

Operator-Based Machine Intelligence: A Hilbert Space Framework for Spectral Learning and Symbolic Reasoning
di: Kiruluta, Andrew, et al.
Pubblicazione: (2025)

From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
di: Kiruluta, Andrew
Pubblicazione: (2025)

DWTSumm: Discrete Wavelet Transform for Document Summarization
di: Salama, Rana, et al.
Pubblicazione: (2026)

Faster Transformer Decoding: N-gram Masked Self-Attention
di: Chelba, Ciprian, et al.
Pubblicazione: (2020)

Improving Transformers with Dynamically Composable Multi-Head Attention
di: Xiao, Da, et al.
Pubblicazione: (2024)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
di: Musat, Tiberiu
Pubblicazione: (2024)

MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task
di: Li, Yanjie, et al.
Pubblicazione: (2024)

ViMGuard: A Novel Multi-Modal System for Video Misinformation Guarding
di: Kan, Andrew, et al.
Pubblicazione: (2024)

Exclusive Self Attention
di: Zhai, Shuangfei
Pubblicazione: (2026)

Fusion Matters: Length-Aware Analysis of Positional-Encoding Fusion in Transformers
di: Hallam, Mohamed Amine, et al.
Pubblicazione: (2026)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
di: Dentamaro, Vincenzo
Pubblicazione: (2025)

LASER: Attention with Exponential Transformation
di: Duvvuri, Sai Surya, et al.
Pubblicazione: (2024)

RecurFormer: Not All Transformer Heads Need Self-Attention
di: Yan, Ruiqing, et al.
Pubblicazione: (2024)

Generalized Probabilistic Attention Mechanism in Transformers
di: Heo, DongNyeong, et al.
Pubblicazione: (2024)

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion
di: Chen, Yilong, et al.
Pubblicazione: (2024)

From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers
di: Ildiz, M. Emrullah, et al.
Pubblicazione: (2024)