Cover Image

Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Kwon, Soo Min, Zhang, Zekai, Song, Dogyoon, Balzano, Laura, Qu, Qing
Format:	Preprint
Published:	2023
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation
by: Yaras, Can, et al.
Published: (2024)

Out-of-Distribution Generalization of In-Context Learning: A Low-Dimensional Subspace Perspective
by: Kwon, Soo Min, et al.
Published: (2025)

Learning Dynamics of Deep Linear Networks Beyond the Edge of Stability
by: Ghosh, Avrajit, et al.
Published: (2025)

An Overview of Low-Rank Structures in the Training and Adaptation of Large Models
by: Balzano, Laura, et al.
Published: (2025)

Emergent Low-Rank Training Dynamics in MLPs with Smooth Activations
by: Xu, Alec S., et al.
Published: (2026)

Diffusion Models Learn Low-Dimensional Distributions via Subspace Clustering
by: Wang, Peng, et al.
Published: (2024)

Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modeling
by: Li, Xiao, et al.
Published: (2025)

BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference
by: Lee, Changwoo, et al.
Published: (2024)

Understanding Deep Representation Learning via Layerwise Feature Compression and Discrimination
by: Wang, Peng, et al.
Published: (2023)

Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks
by: Zhang, Zixuan, et al.
Published: (2023)

Symmetric Matrix Completion with ReLU Sampling
by: Liu, Huikang, et al.
Published: (2024)

Shallow Diffuse: Robust and Invisible Watermarking through Low-Dimensional Subspaces in Diffusion Models
by: Li, Wenda, et al.
Published: (2024)

Local Linear Recovery Guarantee of Deep Neural Networks at Overparameterization
by: Zhang, Yaoyu, et al.
Published: (2024)

Local Minima Structures in Gaussian Mixture Models
by: Chen, Yudong, et al.
Published: (2020)

Fair Community Detection and Structure Learning in Heterogeneous Graphical Models
by: Tarzanagh, Davoud Ataee, et al.
Published: (2021)

Orthogonal Subspace Clustering: Enhancing High-Dimensional Data Analysis through Adaptive Dimensionality Reduction and Efficient Clustering
by: Wen, Qing-Yuan, et al.
Published: (2026)

An Analytical Model for Overparameterized Learning Under Class Imbalance
by: Mor, Eliav, et al.
Published: (2025)

The Interpolating Information Criterion for Overparameterized Models
by: Hodgkinson, Liam, et al.
Published: (2023)

A Closer Look at Model Collapse: From a Generalization-to-Memorization Perspective
by: Shi, Lianghe, et al.
Published: (2025)

Characterizing Dynamical Stability of Stochastic Gradient Descent in Overparameterized Learning
by: Chemnitz, Dennis, et al.
Published: (2024)

A Spectral Framework for Tracking Communities in Evolving Networks
by: Hume, Jacob, et al.
Published: (2024)

Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime
by: Zhang, Richard Y.
Published: (2021)

Scaling Laws of SignSGD in Linear Regression: When Does It Outperform SGD?
by: Kim, Jihwan, et al.
Published: (2026)

How Does Overparameterization Affect Machine Unlearning of Deep Neural Networks?
by: Alon, Gal, et al.
Published: (2025)

Revisiting Optimism and Model Complexity in the Wake of Overparameterized Machine Learning
by: Patil, Pratik, et al.
Published: (2024)

Analysis of Overparameterization in Continual Learning under a Linear Model
by: Goldfarb, Daniel, et al.
Published: (2025)

Theoretical Insights into Overparameterized Models in Multi-Task and Replay-Based Continual Learning
by: Banayeeanzade, Amin, et al.
Published: (2024)

Unifying Low Dimensional Spectra in Deep Learning
by: Garrod, Connall, et al.
Published: (2024)

The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing
by: Xu, Xingyu, et al.
Published: (2023)

Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matching
by: Fraschini, Andrea, et al.
Published: (2026)

Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing
by: Chen, Siyi, et al.
Published: (2024)

ALPCAH: Subspace Learning for Sample-wise Heteroscedastic Data
by: Cavazos, Javier Salazar, et al.
Published: (2025)

Theoretical Limitations of Ensembles in the Age of Overparameterization
by: Dern, Niclas, et al.
Published: (2024)

How Does Overparameterization Affect Features?
by: Duzgun, Ahmet Cagri, et al.
Published: (2024)

Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models
by: Wu, David X., et al.
Published: (2023)

Task Shift: From Classification to Regression in Overparameterized Linear Models
by: LaBonte, Tyler, et al.
Published: (2025)

Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality
by: Zhang, Ruijia, et al.
Published: (2025)

Algebraic and Statistical Properties of the Ordinary Least Squares Interpolator
by: Shen, Dennis, et al.
Published: (2023)

CoPRIS: Efficient and Stable Reinforcement Learning via Concurrency-Controlled Partial Rollout with Importance Sampling
by: Qu, Zekai, et al.
Published: (2025)

Privacy for Free in the Overparameterized Regime
by: Bombari, Simone, et al.
Published: (2024)