Imagen de Portada

Energy-Gated Attention and Wavelet Positional Encoding: Complementary Inductive Biases for Transformer Attention

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autor principal:	Zeris, Athanasios
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Computation and Language Signal Processing
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Energy-Gated Attention: Spectral Salience as an Inductive Bias for Transformer Attention
por: Zeris, Athanasios
Publicado: (2026)

Beyond Sinusoids: A Morlet Wavelet Framework for Transformer Positional Encoding
por: Zeris, Athanasios
Publicado: (2026)

Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning
por: Comminiello, Danilo, et al.
Publicado: (2024)

LLMs as High-Dimensional Nonlinear Autoregressive Models with Attention: Training, Alignment and Inference
por: Krishnamurthy, Vikram
Publicado: (2026)

Is Attention always needed? A Case Study on Language Identification from Speech
por: Mandal, Atanu, et al.
Publicado: (2021)

Wavelet GPT: Wavelet Inspired Large Language Models
por: Verma, Prateek
Publicado: (2024)

Training-Free Spectral Fingerprints of Voice Processing in Transformers
por: Noël, Valentin
Publicado: (2025)

UWB TDoA Error Correction using Transformers: Patching and Positional Encoding Strategies
por: Coppens, Dieter, et al.
Publicado: (2025)

Discrete Cosine Transform Based Decorrelated Attention for Vision Transformers
por: Pan, Hongyi, et al.
Publicado: (2024)

sEEG-based Encoding for Sentence Retrieval: A Contrastive Learning Approach to Brain-Language Alignment
por: Liu, Yijun
Publicado: (2025)

Large Language Model-informed ECG Dual Attention Network for Heart Failure Risk Prediction
por: Chen, Chen, et al.
Publicado: (2024)

BitStopper: An Efficient Transformer Attention Accelerator via Stage-fusion and Early Termination
por: Wang, Huizheng, et al.
Publicado: (2025)

TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly
por: Koike-Akino, Toshiaki, et al.
Publicado: (2026)

MEIT: Multimodal Electrocardiogram Instruction Tuning on Large Language Models for Report Generation
por: Wan, Zhongwei, et al.
Publicado: (2024)

Dynamic Fault Characteristics Evaluation in Power Grid
por: Pei, Hao, et al.
Publicado: (2023)

A Graph Signal Processing Framework for Hallucination Detection in Large Language Models
por: Noël, Valentin
Publicado: (2025)

Bridging Brain Signals and Language: A Deep Learning Approach to EEG-to-Text Decoding
por: Gedawy, Mostafa El, et al.
Publicado: (2025)

EEG-CLIP : Learning EEG representations from natural language descriptions
por: Ndir, Tidiane Camaret, et al.
Publicado: (2025)

When marine radar target detection meets pretrained large language models
por: Hu, Qiying, et al.
Publicado: (2025)

Can VLM Pseudo-Labels Train a Time-Series QA Model That Outperforms the VLM?
por: Fujimura, Takuya, et al.
Publicado: (2025)

Lightweight Conceptual Dictionary Learning for Text Classification Using Information Compression
por: Wan, Li, et al.
Publicado: (2024)

Text me the data: Generating Ground Pressure Sequence from Textual Descriptions for HAR
por: Ray, Lala Shakti Swarup, et al.
Publicado: (2024)

PaTH Attention: Position Encoding via Accumulating Householder Transformations
por: Yang, Songlin, et al.
Publicado: (2025)

Bridging Auditory Perception and Language Comprehension through MEG-Driven Encoding Models
por: Ciferri, Matteo, et al.
Publicado: (2024)

Attentive Fusion: A Transformer-based Approach to Multimodal Hate Speech Detection
por: Mandal, Atanu, et al.
Publicado: (2024)

Herglotz-NET: Implicit Neural Representation of Spherical Data with Harmonic Positional Encoding
por: Hanon, Théo, et al.
Publicado: (2025)

Range-aware Positional Encoding via High-order Pretraining: Theory and Practice
por: Nguyen, Viet Anh, et al.
Publicado: (2024)

Transformer-Based Indirect Structural Health Monitoring of Rail Infrastructure with Attention-Driven Detection and Localization of Transient Defects
por: Ma, Sizhe, et al.
Publicado: (2025)

A Deep Learning Model for Battery State Prediction towards Intelligent Energy Management
por: Koukosias, Athanasios, et al.
Publicado: (2026)

Explainable Dual-Attention Tabular Transformer for Soil Electrical Resistivity Prediction: A Decision Support Framework for High-Voltage Substation Construction
por: Kongkitkul, Warat, et al.
Publicado: (2025)

Robust Cross-Domain WiFi Fall Detection via Physics-Driven Attention-Enhanced Transformers
por: Wang, Yingzhe, et al.
Publicado: (2026)

TAnet: A New Temporal Attention Network for EEG-based Auditory Spatial Attention Decoding with a Short Decision Window
por: Ding, Yuting, et al.
Publicado: (2024)

WST-X Series: Wavelet Scattering Transform for Interpretable Speech Deepfake Detection
por: Xuan, Xi, et al.
Publicado: (2026)

IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS
por: Sankar, Ashwin, et al.
Publicado: (2024)

Electrocardiogram Classification with Transformers Using Koopman and Wavelet Features
por: Ghosh, Sucheta, et al.
Publicado: (2026)

Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalities
por: Ioannides, Georgios, et al.
Publicado: (2024)

Attention-Refined Unrolling for Sparse Sequential micro-Doppler Reconstruction
por: Mazzieri, Riccardo, et al.
Publicado: (2023)

Transformer-Based Bearing Fault Detection using Temporal Decomposition Attention Mechanism
por: Mirzaeibonehkhater, Marzieh, et al.
Publicado: (2024)

L-FAME: Longitudinal Focused Attention Meditation EEG Dataset and Benchmark
por: Li, Angqi, et al.
Publicado: (2026)

EEG-GMACN: Interpretable EEG Graph Mutual Attention Convolutional Network
por: Ye, Haili, et al.
Publicado: (2024)