Cover Image

LaX: Boosting Low-Rank Training of Foundation Models via Latent Crossing

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Zhang, Ruijie, Liu, Ziyue, Wang, Zhengyang, Zhang, Zheng
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

BOOST: BOttleneck-Optimized Scalable Training Framework for Low-Rank Large Language Models
by: Wang, Zhengyang, et al.
Published: (2025)

CoLA: Compute-Efficient Pre-Training of LLMs via Low-Rank Activation
by: Liu, Ziyue, et al.
Published: (2025)

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning
by: Liu, Ziyue, et al.
Published: (2026)

TEON: Tensorized Orthonormalization Beyond Layer-Wise Muon for Large Language Model Pre-Training
by: Zhang, Ruijie, et al.
Published: (2026)

MuonQ: Enhancing Low-Bit Muon Quantization via Directional Fidelity Optimization
by: Su, Yupeng, et al.
Published: (2026)

MUON+: Towards More Effective Muon via One Additional Normalization Step for LLM Pre-training
by: Zhang, Ruijie, et al.
Published: (2026)

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization
by: Yang, Zi, et al.
Published: (2024)

Federated Low-Rank Adaptation for Foundation Models: A Survey
by: Yang, Yiyuan, et al.
Published: (2025)

Asymmetry in Low-Rank Adapters of Foundation Models
by: Zhu, Jiacheng, et al.
Published: (2024)

Low-Rank Adaptation for Foundation Models: A Comprehensive Review
by: Yang, Menglin, et al.
Published: (2024)

A Stronger Mixture of Low-Rank Experts for Fine-Tuning Foundation Models
by: Sun, Mengyang, et al.
Published: (2025)

CR-Net: Scaling Parameter-Efficient Training with Cross-Layer Low-Rank Structure
by: Kong, Boao, et al.
Published: (2025)

SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
by: Tang, Anke, et al.
Published: (2024)

GaLore$+$: Boosting Low-Rank Adaptation for LLMs with Cross-Head Projection
by: Liao, Xutao, et al.
Published: (2024)

Come Together, But Not Right Now: A Progressive Strategy to Boost Low-Rank Adaptation
by: Zhuang, Zhan, et al.
Published: (2025)

FuRA: Full-Rank Parameter-Efficient Fine-Tuning with Spectral Preconditioning
by: Zhao, Yequan, et al.
Published: (2026)

Breaking the Blocks: Continuous Low-Rank Decomposed Scaling for Unified LLM Quantization and Adaptation
by: Tang, Pingzhi, et al.
Published: (2026)

Training-Free Bayesianization for Low-Rank Adapters of Large Language Models
by: Shi, Haizhou, et al.
Published: (2024)

Dynamic Low-Rank Sparse Adaptation for Large Language Models
by: Huang, Weizhong, et al.
Published: (2025)

G2LoRA: Gradient Orthogonal Low-Rank Adaptation Framework for Graph Continual Learning on Text-Attributed Graphs
by: Wang, Yuhan, et al.
Published: (2026)

ELAS: Efficient Pre-Training of Low-Rank Large Language Models via 2:4 Activation Sparsity
by: Li, Jiaxi, et al.
Published: (2026)

Metric-agnostic Learning-to-Rank via Boosting and Rank Approximation
by: Gomez, Camilo, et al.
Published: (2026)

LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Models
by: Yang, Yifan, et al.
Published: (2024)

Batched Low-Rank Adaptation of Foundation Models
by: Wen, Yeming, et al.
Published: (2023)

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
by: Zhao, Jiawei, et al.
Published: (2024)

Privacy-Preserving Low-Rank Adaptation against Membership Inference Attacks for Latent Diffusion Models
by: Luo, Zihao, et al.
Published: (2024)

Lossless Model Compression via Joint Low-Rank Factorization Optimization
by: Zhang, Boyang, et al.
Published: (2024)

Identifying and Correcting Label Noise for Robust GNNs via Influence Contradiction
by: Ju, Wei, et al.
Published: (2026)

Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition
by: Zhang, Xitong, et al.
Published: (2024)

DeCAF: Decentralized Consensus-And-Factorization for Low-Rank Adaptation of Foundation Models
by: Saadati, Nastaran, et al.
Published: (2025)

Score-Based Model for Low-Rank Tensor Recovery
by: Cheng, Zhengyun, et al.
Published: (2025)

Leveraging Latent Diffusion Models for Training-Free In-Distribution Data Augmentation for Surface Defect Detection
by: Girella, Federico, et al.
Published: (2024)

Low-Rank Optimal Transport through Factor Relaxation with Latent Coupling
by: Halmos, Peter, et al.
Published: (2024)

D-QRELO: Training- and Data-Free Delta Compression for Large Language Models via Quantization and Residual Low-Rank Approximation
by: Li, Junlin, et al.
Published: (2026)

An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models
by: Zhang, Yufeng, et al.
Published: (2022)

Unified Graph Prompt Learning via Low-Rank Graph Message Prompting
by: Wang, Beibei, et al.
Published: (2026)

Transfer Learning with Foundational Models for Time Series Forecasting using Low-Rank Adaptations
by: Germán-Morales, M., et al.
Published: (2024)

FedSLoP: Memory-Efficient Federated Learning with Low-Rank Gradient Projection
by: He, Yutong, et al.
Published: (2026)

Multi-Head Low-Rank Attention
by: Liu, Songtao, et al.
Published: (2026)

InRank: Incremental Low-Rank Learning
by: Zhao, Jiawei, et al.
Published: (2023)