Cover Image

Learnable Multi-Scale Wavelet Transformer: A Novel Alternative to Self-Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kiruluta, Andrew, Burity, Priscilla, Williams, Samantha
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Graph Laplacian Wavelet Transformer via Learnable Spectral Decomposition
by: Kiruluta, Andrew, et al.
Published: (2025)

Breaking Quadratic Barriers: A Non-Attention LLM for Ultra-Long Context Horizons
by: Kiruluta, Andrew, et al.
Published: (2025)

From Eigenmodes to Proofs: Integrating Graph Spectral Operators with Symbolic Interpretable Reasoning
by: Kiruluta, Andrew, et al.
Published: (2025)

Adaptive Two Sided Laplace Transforms: A Learnable, Interpretable, and Scalable Replacement for Self-Attention
by: Kiruluta, Andrew
Published: (2025)

Beyond Self Attention: A Subquadratic Fourier Wavelet Transformer with Multi Modal Fusion
by: Kiruluta, Andrew, et al.
Published: (2021)

History-Aware Cross-Attention Reinforcement: Self-Supervised Multi Turn and Chain-of-Thought Fine-Tuning with vLLM
by: Kiruluta, Andrew, et al.
Published: (2025)

Beyond Neural Networks: Symbolic Reasoning over Wavelet Logic Graph Signals
by: Kiruluta, Andrew, et al.
Published: (2025)

Entropic-Time Inference: Self-Organizing Large Language Model Decoding Beyond Attention
by: Kiruluta, Andrew
Published: (2026)

Spectral Neuro-Symbolic Reasoning II: Semantic Node Merging, Entailment Filtering, and Knowledge Graph Alignment
by: Kiruluta, Andrew, et al.
Published: (2025)

Operator-Based Machine Intelligence: A Hilbert Space Framework for Spectral Learning and Symbolic Reasoning
by: Kiruluta, Andrew, et al.
Published: (2025)

State Fourier Diffusion Language Model (SFDLM): A Scalable, Novel Iterative Approach to Language Modeling
by: Kiruluta, Andrew, et al.
Published: (2025)

Spectral Generative Flow Models: A Physics-Inspired Replacement for Vectorized Large Language Models
by: Kiruluta, Andrew
Published: (2026)

Data-Driven Variational Basis Learning Beyond Neural Networks: A Non-Neural Framework for Adaptive Basis Discovery
by: Kiruluta, Andrew
Published: (2026)

From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning
by: Kiruluta, Andrew
Published: (2026)

Filtering Beats Fine Tuning: A Bayesian Kalman View of In Context Learning in LLMs
by: Kiruluta, Andrew
Published: (2026)

FourierNAT: A Fourier-Mixing-Based Non-Autoregressive Transformer for Parallel Sequence Generation
by: Kiruluta, Andrew, et al.
Published: (2025)

From Pixels and Words to Waves: A Unified Framework for Spectral Dictionary vLLMs
by: Kiruluta, Andrew, et al.
Published: (2025)

Reducing Deep Network Complexity via Sparse Hierarchical Fourier Interaction Networks
by: Kiruluta, Andrew, et al.
Published: (2017)

CS-VLM: Compressed Sensing Attention for Efficient Vision-Language Representation Learning
by: Kiruluta, Andrew, et al.
Published: (2025)

A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signals
by: Kiruluta, Andrew, et al.
Published: (2025)

Wavelet Logic Machines: Learning and Reasoning in the Spectral Domain Without Neural Networks
by: Kiruluta, Andrew
Published: (2025)

A Hybrid Wavelet-Fourier Method for Next-Generation Conditional Diffusion Models
by: Kiruluta, Andrew, et al.
Published: (2025)

Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention
by: Zeris, Athanasios
Published: (2026)

Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers
by: Bozic, Vukasin, et al.
Published: (2023)

Learnable Permutation for Structured Sparsity on Transformer Models
by: Li, Zekai, et al.
Published: (2026)

Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
by: He, Mutian, et al.
Published: (2025)

From Attention to Atoms: Spectral Dictionary Learning for Fast, Interpretable Language Models
by: Kiruluta, Andrew
Published: (2025)

Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing
by: Piękos, Piotr, et al.
Published: (2025)

Scaling Attention to Very Long Sequences in Linear Time with Wavelet-Enhanced Random Spectral Attention (WERSA)
by: Dentamaro, Vincenzo
Published: (2025)

DWTSumm: Discrete Wavelet Transform for Document Summarization
by: Salama, Rana, et al.
Published: (2026)

A Novel Architecture for Symbolic Reasoning with Decision Trees and LLM Agents
by: Kiruluta, Andrew
Published: (2025)

LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMs
by: Gordon, Ofir, et al.
Published: (2026)

Linear Transformers with Learnable Kernel Functions are Better In-Context Models
by: Aksenov, Yaroslav, et al.
Published: (2024)

Faster Transformer Decoding: N-gram Masked Self-Attention
by: Chelba, Ciprian, et al.
Published: (2020)

LESA: Learnable LLM Layer Scaling-Up
by: Yang, Yifei, et al.
Published: (2025)

Improving Transformers with Dynamically Composable Multi-Head Attention
by: Xiao, Da, et al.
Published: (2024)

GTS: Inference-Time Scaling of Latent Reasoning with a Learnable Gaussian Thought Sampler
by: Wang, Minghan, et al.
Published: (2026)

Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers
by: Musat, Tiberiu
Published: (2024)

Scale-invariant Attention
by: Anson, Ben, et al.
Published: (2025)

Fundamental Limitations on Subquadratic Alternatives to Transformers
by: Alman, Josh, et al.
Published: (2024)