Lambda-Skip Connections: the architectural component that prevents Rank Collapse

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Joseph, Federico Arangath, Sieber, Jerome, Zeilinger, Melanie N., Alonso, Carmen Amo
Format:	Preprint
Published:	2024
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

State Space Models as Foundation Models: A Control Theoretic Overview
by: Alonso, Carmen Amo, et al.
Published: (2024)

Design Principles for Sequence Models via Coefficient Dynamics
by: Sieber, Jerome, et al.
Published: (2025)

Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks
by: Sieber, Jerome, et al.
Published: (2024)

Only Large Weights (And Not Skip Connections) Can Prevent the Perils of Rank Collapse
by: Alman, Josh, et al.
Published: (2025)

Task-Level Insights from Eigenvalues across Sequence Models
by: Rickenbach, Rahel, et al.
Published: (2025)

HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
by: Joseph, Federico Arangath, et al.
Published: (2024)

Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction
by: De Santi, Riccardo, et al.
Published: (2024)

LambdaRankIC: Directly Optimizing Rank IC for Financial Prediction
by: Lin, Yan, et al.
Published: (2026)

On the Adversarial Transferability of Generalized "Skip Connections"
by: Wang, Yisen, et al.
Published: (2024)

Stochastic Online Optimization for Cyber-Physical and Robotic Systems
by: Ma, Hao, et al.
Published: (2024)

Can an MLP Absorb Its Own Skip Connection?
by: Mijoski, Antonij, et al.
Published: (2026)

Submodular Reinforcement Learning
by: Prajapat, Manish, et al.
Published: (2023)

Skip-Connected Policy Optimization for Implicit Advantage
by: Teng, Fengwei, et al.
Published: (2026)

SALAAD: Sparse And Low-Rank Adaptation via ADMM for Large Language Model Inference
by: Ma, Hao, et al.
Published: (2026)

Computing Linear Regions in Neural Networks with Skip Connections
by: Joyce, Johnny, et al.
Published: (2025)

Impact of Bottleneck Layers and Skip Connections on the Generalization of Linear Denoising Autoencoders
by: Ham, Jonghyun, et al.
Published: (2025)

Probabilistic Skip Connections for Deterministic Uncertainty Quantification in Deep Neural Networks
by: Jimenez, Felix, et al.
Published: (2025)

Computationally Efficient System Level Tube-MPC for Uncertain Systems
by: Sieber, Jerome, et al.
Published: (2024)

Efficient Skip Connections Realization for Secure Inference on Encrypted Data
by: Drucker, Nir, et al.
Published: (2023)

SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connection
by: Ataiefard, Foozhan, et al.
Published: (2024)

A robust and adaptive MPC formulation for Gaussian process models
by: Dubied, Mathieu, et al.
Published: (2025)

Approximate non-linear model predictive control with safety-augmented neural networks
by: Hose, Henrik, et al.
Published: (2023)

Performance-driven Constrained Optimal Auto-Tuner for MPC
by: Puigjaner, Albert Gassol, et al.
Published: (2025)

Physics-Informed Neural Networks with Skip Connections for Modeling and Control of Gas-Lifted Oil Wells
by: Kittelsen, Jonas Ekeland, et al.
Published: (2024)

The Persistence of Neural Collapse Despite Low-Rank Bias
by: Garrod, Connall, et al.
Published: (2024)

ZNorm: Z-Score Gradient Normalization Accelerating Skip-Connected Network Training without Architectural Modification
by: Yun, Juyoung
Published: (2024)

Rethinking Image Skip Connections in StyleGAN2
by: Park, Seung, et al.
Published: (2024)

Physics-informed learning under mixing: How physical knowledge speeds up learning
by: Scampicchio, Anna, et al.
Published: (2025)

Safe and Near-Optimal Control with Online Dynamics Learning
by: Prajapat, Manish, et al.
Published: (2025)

GenCtrl -- A Formal Controllability Toolkit for Generative Models
by: Cheng, Emily, et al.
Published: (2026)

Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse
by: Baker, Bradley T., et al.
Published: (2024)

Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graph
by: Roth, Andreas, et al.
Published: (2024)

Automatic nonlinear MPC approximation with closed-loop guarantees
by: Tokmak, Abdullah, et al.
Published: (2023)

Finite-Sample-Based Reachability for Safe Control with Gaussian Process Dynamics
by: Prajapat, Manish, et al.
Published: (2025)

No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO
by: Moalla, Skander, et al.
Published: (2024)

Improving Model Representation and Reducing KV Cache via Skip Connections with First Value Heads
by: Wu, Zhoutong, et al.
Published: (2025)

Safe Guaranteed Exploration for Non-linear Systems
by: Prajapat, Manish, et al.
Published: (2024)

The Inlet Rank Collapse in Implicit Neural Representations: Diagnosis and Unified Remedy
by: Zheng, Jianqiao, et al.
Published: (2026)

DEMONSTRATE: Zero-shot Language to Robotic Control via Multi-task Demonstration Learning
by: Rickenbach, Rahel, et al.
Published: (2025)

Zero-Order Optimization for Gaussian Process-based Model Predictive Control
by: Lahr, Amon, et al.
Published: (2022)