Cover Image

Residual Alignment: Uncovering the Mechanisms of Residual Networks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Jianing, Papyan, Vardan
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

On the Importance of Gaussianizing Representations
by: Eftekhari, Daniel, et al.
Published: (2025)

Laplacian Heads Improve Transformers by Smoothing Token Representations
by: Zhang, Yuchong, et al.
Published: (2026)

OATS: Outlier-Aware Pruning Through Sparse and Low Rank Decomposition
by: Zhang, Stephen, et al.
Published: (2024)

Sparsest Models Elude Pruning: An Exposé of Pruning's Current Capabilities
by: Zhang, Stephen, et al.
Published: (2024)

Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddings
by: Zhang, Stephen, et al.
Published: (2025)

Pushing Boundaries: Mixup's Influence on Neural Collapse
by: Fisher, Quinn, et al.
Published: (2024)

Linguistic Collapse: Neural Collapse in (Large) Language Models
by: Wu, Robert, et al.
Published: (2024)

Transformer Block Coupling and its Correlation with Generalization in LLMs
by: Aubry, Murdock, et al.
Published: (2024)

Out of the Ordinary: Spectrally Adapting Regression for Covariate Shift
by: Eyre, Benjamin, et al.
Published: (2023)

SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection
by: Joshi, Maithili, et al.
Published: (2025)

Lorentzian Residual Neural Networks
by: He, Neil, et al.
Published: (2024)

Residue-based Label Protection Mechanisms in Vertical Logistic Regression
by: Tan, Juntao, et al.
Published: (2022)

Residual Hyperbolic Graph Convolution Networks
by: Xue, Yangkai, et al.
Published: (2024)

Residual Random Neural Networks
by: Andrecut, M.
Published: (2024)

Drug Synergy Prediction via Residual Graph Isomorphism Networks and Attention Mechanisms
by: Song, Jiyan, et al.
Published: (2026)

A Qualitative Test-Risk Mechanism for Scaling Behavior in Normalized Residual Networks
by: Cheng, Daning, et al.
Published: (2026)

Progressive Approximation in Deep Residual Networks: Theory and Validation
by: Wang, Wei, et al.
Published: (2026)

Residual Matrix Transformers: Scaling the Size of the Residual Stream
by: Mak, Brian, et al.
Published: (2025)

Multi-Token Residual Prediction
by: Xu, Yufeng, et al.
Published: (2026)

Residual Reservoir Memory Networks
by: Pinna, Matteo, et al.
Published: (2025)

ResidualDroppath: Enhancing Feature Reuse over Residual Connections
by: Park, Sejik
Published: (2024)

From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data
by: Rezaei, Mohammad R., et al.
Published: (2026)

Stability and Generalization for Bellman Residuals
by: Kang, Enoch H., et al.
Published: (2025)

ADMM Algorithms for Residual Network Training: Convergence Analysis and Parallel Implementation
by: Xu, Jintao, et al.
Published: (2023)

Residual Reweighted Conformal Prediction for Graph Neural Networks
by: Zhang, Zheng, et al.
Published: (2025)

Gradient Residual Connections
by: Pan, Yangchen, et al.
Published: (2026)

PirateNets: Physics-informed Deep Learning with Residual Adaptive Networks
by: Wang, Sifan, et al.
Published: (2024)

RRaPINNs: Residual Risk-Aware Physics Informed Neural Networks
by: Akazan, Ange-Clément, et al.
Published: (2025)

Robust Neural Pruning with Gradient Sampling Optimization for Residual Neural Networks
by: Yun, Juyoung
Published: (2023)

Zero-Residual Concept Erasure via Progressive Alignment in Text-to-Image Model
by: Chen, Hongxu, et al.
Published: (2025)

Residual Distribution Predictive Systems
by: Allen, Sam, et al.
Published: (2025)

Physics-Informed Neural Networks: Minimizing Residual Loss with Wide Networks and Effective Activations
by: Dashtbayaz, Nima Hosseini, et al.
Published: (2024)

PRISM: Parallel Residual Iterative Sequence Model
by: Jiang, Jie, et al.
Published: (2026)

Semantic Residual Prompts for Continual Learning
by: Menabue, Martin, et al.
Published: (2024)

Residual Quantization with Implicit Neural Codebooks
by: Huijben, Iris A. M., et al.
Published: (2024)

Residual Deep Gaussian Processes on Manifolds
by: Wyrwal, Kacper, et al.
Published: (2024)

The Quantization Benefits of Residual-Free Transformers
by: Ji, Yiping, et al.
Published: (2026)

ResidualTransformer: Residual Low-Rank Learning with Weight-Sharing for Transformer Layers
by: Wang, Yiming, et al.
Published: (2023)

DouRN: Improving DouZero by Residual Neural Networks
by: Chen, Yiquan, et al.
Published: (2024)

Histogram-Equalized Quantization for logic-gated Residual Neural Networks
by: Nguyen, Van Thien, et al.
Published: (2025)