Cover Image

Differential Gated Self-Attention

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lygizou, Elpiniki Maria, Farsang, Mónika, Grosu, Radu
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Synaptic Activation and Dual Liquid Dynamics for Interpretable Bio-Inspired Models
by: Farsang, Mónika, et al.
Published: (2026)

Automated Immunophenotyping Assessment for Diagnosing Childhood Acute Leukemia using Set-Transformers
by: Lygizou, Elpiniki Maria, et al.
Published: (2024)

Single molecule localization microscopy challenge: a biologically inspired benchmark for long-sequence modeling
by: Valeh, Fatemeh, et al.
Published: (2026)

Looped SSMs: Depth-Recurrence and Input Reshaping for Time Series Classification
by: Farsang, Mónika, et al.
Published: (2026)

Parallelization of Non-linear State-Space Models: Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling
by: Farsang, Mónika, et al.
Published: (2025)

Learning with Chemical versus Electrical Synapses -- Does it Make a Difference?
by: Farsang, Mónika, et al.
Published: (2023)

Depth Matters: Multimodal RGB-D Perception for Robust Autonomous Agents
by: Clement, Mihaela-Larisa, et al.
Published: (2025)

Adaptive Control in Autonomous Driving via Real-Time Recurrent RL
by: Lemmel, Julian, et al.
Published: (2026)

MMDVS-LF: Multi-Modal Dynamic Vision Sensor and Eye-Tracking Dataset for Line Following
by: Resch, Felix, et al.
Published: (2024)

Towards Safe Learning-Based Non-Linear Model Predictive Control through Recurrent Neural Network Modeling
by: Clement, Mihaela-Larisa, et al.
Published: (2026)

Liquid Resistance Liquid Capacitance Networks
by: Farsang, Mónika, et al.
Published: (2024)

Real-Time Recurrent Reinforcement Learning
by: Lemmel, Julian, et al.
Published: (2023)

Using Deep Q-Learning to Dynamically Toggle between Push/Pull Actions in Computational Trust Mechanisms
by: Lygizou, Zoi, et al.
Published: (2024)

Neuromorphic IoT Architecture for Efficient Water Management: A Smart Village Case Study
by: Bublin, Mugdim, et al.
Published: (2024)

A quantum-classical reinforcement learning model to play Atari games
by: Freinberger, Dominik, et al.
Published: (2024)

Scalable Offline Reinforcement Learning for Mean Field Games
by: Brunnbauer, Axel, et al.
Published: (2024)

Scenario-Based Curriculum Generation for Multi-Agent Autonomous Driving
by: Brunnbauer, Axel, et al.
Published: (2024)

Quadratic Gating Mixture of Experts: Statistical Insights into Self-Attention
by: Akbarian, Pedram, et al.
Published: (2024)

Understanding Differential Transformer Unchains Pretrained Self-Attentions
by: Kong, Chaerin, et al.
Published: (2025)

Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels
by: Babaiee, Zahra, et al.
Published: (2024)

TD-Interpreter: Enhancing the Understanding of Timing Diagrams with Visual-Language Learning
by: He, Jie, et al.
Published: (2025)

GatedFWA: Linear Flash Windowed Attention with Gated Associative Memory
by: Liu, Jiaxu, et al.
Published: (2025)

TubeDAgger: Reducing the Number of Expert Interventions with Stochastic Reach-Tubes
by: Lemmel, Julian, et al.
Published: (2025)

Online Fine-Tuning of Carbon Emission Predictions using Real-Time Recurrent Learning for State Space Models
by: Lemmel, Julian, et al.
Published: (2025)

Gated Graph Attention Networks with Learnable Temperature
by: Ma, Zhongtian, et al.
Published: (2026)

Recurrent Deep Differentiable Logic Gate Networks
by: Bührer, Simon, et al.
Published: (2025)

Light Differentiable Logic Gate Networks
by: Rüttgers, Lukas, et al.
Published: (2025)

Supplementary dataset for paper: "Towards Safe Learning-Based Non-Linear Model Predictive Control through Recurrent Neural Network Modeling"
by: Clement, Mihaela-Larisa, et al.
Published: (2026)

Relative Entropy Pathwise Policy Optimization
by: Voelcker, Claas, et al.
Published: (2025)

On Zero-Initialized Attention: Optimal Prompt and Gating Factor Estimation
by: Diep, Nghiem T., et al.
Published: (2025)

Boosting House Price Estimations with Multi-Head Gated Attention
by: Sellam, Zakaria Abdellah, et al.
Published: (2024)

Gating Enables Curvature: A Geometric Expressivity Gap in Attention
by: Bathula, Satwik, et al.
Published: (2026)

Gated-SwinRMT: Unifying Swin Windowed Attention with Retentive Manhattan Decay via Input-Dependent Gating
by: Maity, Dipan, et al.
Published: (2026)

Differentiable Sparsity via $D$-Gating: Simple and Versatile Structured Penalization
by: Kolb, Chris, et al.
Published: (2025)

Grouped Differential Attention
by: Lim, Junghwan, et al.
Published: (2025)

Geometric Analysis of Token Selection in Multi-Head Attention
by: Mudarisov, Timur, et al.
Published: (2026)

Gated Linear Attention Transformers with Hardware-Efficient Training
by: Yang, Songlin, et al.
Published: (2023)

Convolutional Differentiable Logic Gate Networks
by: Petersen, Felix, et al.
Published: (2024)

Orthogonal Self-Attention
by: Zhang, Leo, et al.
Published: (2026)

Networkwide Traffic State Forecasting Using Exogenous Information: A Multi-Dimensional Graph Attention-Based Approach
by: Islam, Syed, et al.
Published: (2023)