Low-Rank Learning by Design: the Role of Network Architecture and Activation Linearity in Gradient Rank Collapse

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Baker, Bradley T., Pearlmutter, Barak A., Miller, Robyn, Calhoun, Vince D., Plis, Sergey M.
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Spectral Introspection Identifies Group Training Dynamics in Deep Neural Networks for Neuroimaging
di: Baker, Bradley T., et al.
Pubblicazione: (2024)

Change of Thought: Adaptive Test-Time Computation
di: Mathur, Mrinal, et al.
Pubblicazione: (2025)

Causal Graph Recovery in Neuroimaging through Answer Set Programming
di: Abavisani, Mohammadsajad, et al.
Pubblicazione: (2025)

Generative forecasting of brain activity enhances Alzheimer's classification and interpretation
di: Gao, Yutong, et al.
Pubblicazione: (2024)

SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learning
di: Ohib, Riyasat, et al.
Pubblicazione: (2024)

Riemannian Gradient Descent for Low-Rank Architectures
di: Knight, Nicholas
Pubblicazione: (2026)

Copula-Linked Parallel ICA: A Method for Coupling Structural and Functional MRI brain Networks
di: Agcaoglu, Oktay, et al.
Pubblicazione: (2024)

Rethinking Functional Brain Connectome Analysis: Do Graph Deep Learning Models Help
di: Han, Keqi, et al.
Pubblicazione: (2025)

Learning Exemplar Representations in Single-Trial EEG Category Decoding
di: Kilgallen, Jack, et al.
Pubblicazione: (2024)

Recent Trends in Modelling the Continuous Time Series using Deep Learning: A Survey
di: Habiba, Mansura, et al.
Pubblicazione: (2024)

The Persistence of Neural Collapse Despite Low-Rank Bias
di: Garrod, Connall, et al.
Pubblicazione: (2024)

From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
di: Jaiswal, Ajay, et al.
Pubblicazione: (2024)

Multiscale Neuroimaging Features for the Identification of Medication Class and Non-Responders in Mood Disorder Treatment
di: Baker, Bradley T., et al.
Pubblicazione: (2024)

Cross-Spectral Witness for Hidden Nonequilibrium Beyond the Scalar Ceiling
di: Bi, Yuda, et al.
Pubblicazione: (2026)

Scaling Laws are Redundancy Laws
di: Bi, Yuda, et al.
Pubblicazione: (2025)

InRank: Incremental Low-Rank Learning
di: Zhao, Jiawei, et al.
Pubblicazione: (2023)

Preventing Rank Collapse in Federated Low-Rank Adaptation with Client Heterogeneity
di: Wu, Fei, et al.
Pubblicazione: (2026)

Mapping minds not averages: a scalable subject-specific manifold learning framework for neuroimaging data
di: Geenjaar, Eloy, et al.
Pubblicazione: (2025)

Conditioning on a Volatility Proxy Compresses the Apparent Timescale of Collective Market Correlation
di: Bi, Yuda, et al.
Pubblicazione: (2026)

LASER: Low-Rank Activation SVD for Efficient Recursion
di: Çakar, Ege, et al.
Pubblicazione: (2026)

Unbiased Gradient Low-Rank Projection
di: Pan, Rui, et al.
Pubblicazione: (2025)

Low Rank Gradients and Where to Find Them
di: Sonthalia, Rishi, et al.
Pubblicazione: (2025)

Probe-Free Low-Rank Activation Intervention
di: Jiang, Chonghe, et al.
Pubblicazione: (2025)

Second-Order Forward-Mode Automatic Differentiation for Optimization
di: Cobb, Adam D., et al.
Pubblicazione: (2024)

Redundancy as a Structural Information Principle for Learning and Generalization
di: Bi, Yuda, et al.
Pubblicazione: (2025)

Multilinear Tensor Low-Rank Approximation for Policy-Gradient Methods in Reinforcement Learning
di: Rozada, Sergio, et al.
Pubblicazione: (2025)

Learning Linear Regression with Low-Rank Tasks in-Context
di: Takanami, Kaito, et al.
Pubblicazione: (2025)

Efficient Low-Rank Matrix Estimation, Experimental Design, and Arm-Set-Dependent Low-Rank Bandits
di: Jang, Kyoungseok, et al.
Pubblicazione: (2024)

Greedy Low-Rank Gradient Compression for Distributed Learning with Convergence Guarantees
di: Chen, Chuyan, et al.
Pubblicazione: (2025)

Flora: Low-Rank Adapters Are Secretly Gradient Compressors
di: Hao, Yongchang, et al.
Pubblicazione: (2024)

Matrix Low-Rank Approximation For Policy Gradient Methods
di: Rozada, Sergio, et al.
Pubblicazione: (2024)

Low-Rank Adapters Initialization via Gradient Surgery for Continual Learning
di: Pasquali, Joana, et al.
Pubblicazione: (2026)

Fundamental limits of Non-Linear Low-Rank Matrix Estimation
di: Mergny, Pierre, et al.
Pubblicazione: (2024)

Provable Emergence of Deep Neural Collapse and Low-Rank Bias in $L^2$-Regularized Nonlinear Networks
di: Zangrando, Emanuele, et al.
Pubblicazione: (2024)

Preventing Representational Rank Collapse in MPNNs by Splitting the Computational Graph
di: Roth, Andreas, et al.
Pubblicazione: (2024)

Reservoir Subspace Injection for Online ICA under Top-n Whitening
di: Xiao, Wenjun, et al.
Pubblicazione: (2026)

Optimizing Brain-Computer Interface Performance: Advancing EEG Signals Channel Selection through Regularized CSP and SPEA II Multi-Objective Optimization
di: Esfahani, M. Moein, et al.
Pubblicazione: (2024)

Efficient Wireless Federated Learning via Low-Rank Gradient Factorization
di: Guo, Mingzhao, et al.
Pubblicazione: (2024)

Hierarchical Spatio-Temporal State-Space Modeling for fMRI Analysis
di: Wei, Yuxiang, et al.
Pubblicazione: (2024)

Personalizing Low-Rank Bayesian Neural Networks Via Federated Learning
di: Zhang, Boning, et al.
Pubblicazione: (2024)