Cover Image

Setting the Record Straight on Transformer Oversmoothing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dovonon, Gbètondji J-S, Bronstein, Michael M., Kusner, Matt J.
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Agentic Uncertainty Reveals Agentic Overconfidence
by: Kaddour, Jean, et al.
Published: (2026)

Attention Is All You Need But You Don't Need All Of It For Inference of Large Language Models
by: Tyukin, Georgy, et al.
Published: (2024)

An Auditing Test To Detect Behavioral Shift in Language Models
by: Richter, Leo, et al.
Published: (2024)

Causal Machine Learning: A Survey and Open Problems
by: Kaddour, Jean, et al.
Published: (2022)

Resolving Oversmoothing with Opinion Dissensus
by: Wang, Keqin, et al.
Published: (2025)

Custom Gradient Estimators are Straight-Through Estimators in Disguise
by: Schoenbauer, Matt, et al.
Published: (2024)

Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs
by: Scholkemper, Michael, et al.
Published: (2024)

Graph Neural Networks Do Not Always Oversmooth
by: Epping, Bastian, et al.
Published: (2024)

Taming Gradient Oversmoothing and Expansion in Graph Neural Networks
by: Park, MoonJeong, et al.
Published: (2024)

Partially Trained Graph Convolutional Networks Resist Oversmoothing
by: Kelesis, Dimitrios, et al.
Published: (2024)

The Oversmoothing Fallacy: A Misguided Narrative in GNN Research
by: Park, MoonJeong, et al.
Published: (2025)

Oversmoothing: A Nightmare for Graph Contrastive Learning?
by: Li, Jintang, et al.
Published: (2023)

A Signed Graph Approach to Understanding and Mitigating Oversmoothing in GNNs
by: Wang, Jiaqi, et al.
Published: (2025)

Overcoming Oversmoothness in Graph Convolutional Networks via Hybrid Scattering Networks
by: Wenkel, Frederik, et al.
Published: (2022)

Are We Measuring Oversmoothing in Graph Neural Networks Correctly?
by: Zhang, Kaicheng, et al.
Published: (2025)

Reducing Oversmoothing through Informed Weight Initialization in Graph Neural Networks
by: Kelesis, Dimitrios, et al.
Published: (2024)

Backward Oversmoothing: why is it hard to train deep Graph Neural Networks?
by: Keriven, Nicolas
Published: (2025)

Deeper with Riemannian Geometry: Overcoming Oversmoothing and Oversquashing for Graph Foundation Models
by: Sun, Li, et al.
Published: (2025)

Oversmoothing as Representation Degeneracy in Neural Sheaf Diffusion
by: Dönmez, Arif, et al.
Published: (2026)

Learning Physical Operators using Neural Operators
by: Gopakumar, Vignesh, et al.
Published: (2026)

Demystifying Oversmoothing in Attention-Based Graph Neural Networks
by: Wu, Xinyi, et al.
Published: (2023)

Training Transformers for KV Cache Compressibility
by: Gelberg, Yoav, et al.
Published: (2026)

Understanding Oversmoothing in Diffusion-Based GNNs From the Perspective of Operator Semigroup Theory
by: Zhao, Weichen, et al.
Published: (2024)

Adaptive Message Passing: A General Framework to Mitigate Oversmoothing, Oversquashing, and Underreaching
by: Errica, Federico, et al.
Published: (2023)

Analyzing the Effect of Embedding Norms and Singular Values to Oversmoothing in Graph Neural Networks
by: Kelesis, Dimitrios, et al.
Published: (2025)

On the Convergence and Straightness of Rectified Flow
by: Bansal, Vansh, et al.
Published: (2024)

Supercharging Graph Transformers with Advective Diffusion
by: Wu, Qitian, et al.
Published: (2023)

Proxy Methods for Domain Adaptation
by: Tsai, Katherine, et al.
Published: (2024)

Valid Error Bars for Neural Weather Models using Conformal Prediction
by: Gopakumar, Vignesh, et al.
Published: (2024)

Solving Oversmoothing in GNNs via Nonlocal Message Passing: Algebraic Smoothing and Depth Scalability
by: Guan, Weiqi, et al.
Published: (2025)

Improving the Straight-Through Estimator with Zeroth-Order Information
by: Yang, Ningfeng, et al.
Published: (2025)

Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs
by: Park, MoonJeong, et al.
Published: (2024)

Oversmoothing Alleviation in Graph Neural Networks: A Survey and Unified View
by: Jin, Yufei, et al.
Published: (2024)

On the Complexity of Optimal Graph Rewiring for Oversmoothing and Oversquashing in Graph Neural Networks
by: Chehreghani, Mostafa Haghir
Published: (2026)

An Eulerian Perspective on Straight-Line Sampling
by: Tsimpos, Panos, et al.
Published: (2025)

When Can Proxies Improve the Sample Complexity of Preference Learning?
by: Zhu, Yuchen, et al.
Published: (2024)

Tackling Oversmoothing in GNN via Graph Sparsification: A Truss-based Approach
by: Hossain, Tanvir, et al.
Published: (2024)

Learning Straight Flows by Learning Curved Interpolants
by: Shankar, Shiv, et al.
Published: (2025)

A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Neural Networks
by: Chakraborty, Biswadeep, et al.
Published: (2024)

Improving Discrete Optimisation Via Decoupled Straight-Through Estimator
by: Shah, Rushi, et al.
Published: (2024)