Imagen de Portada

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Kong, Boao, Liang, Junzhu, Liu, Yuxi, Deng, Renjia, Yuan, Kun
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

RoPeSLR: 3D RoPE-driven Sparse-LowRank Attention for Efficient Diffusion Transformers
por: Liu, Yuxi, et al.
Publicado: (2026)

Synergistic Intra- and Cross-Layer Regularization Losses for MoE Expert Specialization
por: Hu, Rizhen, et al.
Publicado: (2026)

MISA: Memory-Efficient LLMs Optimization with Module-wise Importance Sampling
por: Liu, Yuxi, et al.
Publicado: (2025)

On the Convergence of Stochastic Gradient Descent with Perturbed Forward-Backward Passes
por: Kong, Boao, et al.
Publicado: (2026)

Row-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning
por: Liu, Bing, et al.
Publicado: (2025)

BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization
por: Zhang, Hengrui, et al.
Publicado: (2026)

GNMR: Runtime Stability Control for Low-Precision Large Language Model Training
por: Kong, Boao, et al.
Publicado: (2026)

Clapping: Removing Per-sample Storage for Pipeline Parallel Distributed Optimization with Communication Compression
por: Kong, Boao, et al.
Publicado: (2025)

SPARKLE: A Unified Single-Loop Primal-Dual Framework for Decentralized Bilevel Optimization
por: Zhu, Shuchen, et al.
Publicado: (2024)

Decentralized Bilevel Optimization: A Perspective from Transient Iteration Complexity
por: Kong, Boao, et al.
Publicado: (2024)

Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition
por: Zhang, Xitong, et al.
Publicado: (2024)

MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
por: Wang, Sheng, et al.
Publicado: (2024)

LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing
por: Zhang, Ruijie, et al.
Publicado: (2025)

Efficient Pareto Manifold Learning with Low-Rank Structure
por: Chen, Weiyu, et al.
Publicado: (2024)

LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
por: Zhang, Mingyang, et al.
Publicado: (2023)

On Catastrophic Forgetting in Low-Rank Decomposition-Based Parameter-Efficient Fine-Tuning
por: Ahmad, Muhammad, et al.
Publicado: (2026)

Boosting the Accuracy of Stock Market Prediction via Multi-Layer Hybrid MTL Structure
por: Hong, Yuxi
Publicado: (2025)

Joint Tensor-Train Parameterization for Efficient and Expressive Low-Rank Adaptation
por: Qi, Jun, et al.
Publicado: (2025)

Parameter Efficient Continual Learning with Dynamic Low-Rank Adaptation
por: Bhat, Prashant Shivaram, et al.
Publicado: (2025)

ScaLoRA: Optimally Scaled Low-Rank Adaptation for Efficient High-Rank Fine-Tuning
por: Zhang, Yilang, et al.
Publicado: (2025)

Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients
por: Wang, Yezhen, et al.
Publicado: (2025)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
por: Zhao, Jiawei, et al.
Publicado: (2024)

On Generalization Bounds for Neural Networks with Low Rank Layers
por: Pinto, Andrea, et al.
Publicado: (2024)

CLoRA: Parameter-Efficient Continual Learning with Low-Rank Adaptation
por: Muralidhara, Shishir, et al.
Publicado: (2025)

Enhancing Parameter Efficiency and Generalization in Large-Scale Models: A Regularized and Masked Low-Rank Adaptation Approach
por: Mao, Yuzhu, et al.
Publicado: (2024)

A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models
por: Chen, Yiming, et al.
Publicado: (2025)

DropLoRA: Sparse Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
por: Zhang, Haojie
Publicado: (2025)

GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning
por: Jung, Yeonjoon, et al.
Publicado: (2025)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
por: Liu, Ziyue, et al.
Publicado: (2025)

Efficient Generalized Low-Rank Tensor Contextual Bandits
por: Yi, Qianxin, et al.
Publicado: (2023)

Low Rank Multi-Dictionary Selection at Scale
por: Ma, Boya, et al.
Publicado: (2024)

Top-$k$ Feature Importance Ranking
por: Chen, Yuxi, et al.
Publicado: (2025)

Mixture of Distributions Matters: Dynamic Sparse Attention for Efficient Video Diffusion Transformers
por: Liu, Yuxi, et al.
Publicado: (2026)

Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
por: Huang, Ruiquan, et al.
Publicado: (2026)

Low Rank and Sparse Fourier Structure in Recurrent Networks Trained on Modular Addition
por: Rangamani, Akshay
Publicado: (2025)

Fast Forwarding Low-Rank Training
por: Rahamim, Adir, et al.
Publicado: (2024)

Bandwidth-Efficient Adaptive Mixture-of-Experts via Low-Rank Compensation
por: Liu, Zhenyu, et al.
Publicado: (2025)

From PowerSGD to PowerSGD+: Low-Rank Gradient Compression for Distributed Optimization with Convergence Guarantees
por: Xie, Shengping, et al.
Publicado: (2025)

ReLaX-Net: Reusing Layers for Parameter-Efficient Physical Neural Networks
por: Tsuchiyama, Kohei, et al.
Publicado: (2025)

MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning
por: Jiang, Ting, et al.
Publicado: (2024)