Cover Image

EoRA: Fine-tuning-free Compensation for Compressed LLM with Eigenspace Low-Rank Approximation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Shih-Yang, Khadkevich, Maksim, Fung, Nai Chit, Sakr, Charbel, Yang, Chao-Han Huck, Wang, Chien-Yi, Muralidharan, Saurav, Yin, Hongxu, Cheng, Kwang-Ting, Kautz, Jan, Wang, Yu-Chiang Frank, Molchanov, Pavlo, Chen, Min-Hung
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

DoRA: Weight-Decomposed Low-Rank Adaptation
by: Liu, Shih-Yang, et al.
Published: (2024)

Flextron: Many-in-One Flexible Large Language Model
by: Cai, Ruisi, et al.
Published: (2024)

MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models
by: Fang, Gongfan, et al.
Published: (2024)

Compact Language Models via Pruning and Knowledge Distillation
by: Muralidharan, Saurav, et al.
Published: (2024)

X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2026)

DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning
by: Liu, Shih-Yang, et al.
Published: (2025)

LITA: Language Instructed Temporal-Localization Assistant
by: Huang, De-An, et al.
Published: (2024)

Minifinetuning: Low-Data Generation Domain Adaptation through Corrective Self-Distillation
by: Belcak, Peter, et al.
Published: (2025)

AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One
by: Ranzinger, Mike, et al.
Published: (2023)

FasterViT: Fast Vision Transformers with Hierarchical Attention
by: Hatamizadeh, Ali, et al.
Published: (2023)

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization
by: Huang, Xijie, et al.
Published: (2024)

Revisiting Two Finiteness Phenomena for Mandarin Chinese Complementation Structures: An Empirically Oriented Approach via Systematic Hypothesis Testing
by: Chit‐Fung Lam
Published: (2025)

GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization
by: Liu, Shih-Yang, et al.
Published: (2026)

Adaptive Sharpness-Aware Pruning for Robust Sparse Networks
by: Bair, Anna, et al.
Published: (2023)

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models
by: Heinrich, Greg, et al.
Published: (2024)

Small Language Models are the Future of Agentic AI
by: Belcak, Peter, et al.
Published: (2025)

Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
by: Fu, Yonggan, et al.
Published: (2025)

Step Out and Seek Around: On Warm-Start Training with Incremental Data
by: Shen, Maying, et al.
Published: (2024)

Universal Deep Research: Bring Your Own Model and Strategy
by: Belcak, Peter, et al.
Published: (2025)

Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
by: Fu, Yonggan, et al.
Published: (2025)

ESPACE: Dimensionality Reduction of Activations for Model Compression
by: Sakr, Charbel, et al.
Published: (2024)

FeatSharp: Your Vision Model Features, Sharper
by: Ranzinger, Mike, et al.
Published: (2025)

Advancing Weight and Channel Sparsification with Enhanced Saliency
by: Sun, Xinglong, et al.
Published: (2025)

VILA: On Pre-training for Visual Language Models
by: Lin, Ji, et al.
Published: (2023)

VILA$^2$: VILA Augmented VILA
by: Fang, Yunhao, et al.
Published: (2024)

TwinTURBO: Semi-Supervised Fine-Tuning of Foundation Models via Mutual Information Decompositions for Downstream Task and Latent Spaces
by: Quétant, Guillaume, et al.
Published: (2025)

Organizational Structure in Multi-Campus Community Junior Colleges/Districts.
by: Chang, Nai-Kwang
Published: (1978)

X-VILA: Cross-Modality Alignment for Large Language Model
by: Ye, Hanrong, et al.
Published: (2024)

Scaling Vision Pre-Training to 4K Resolution
by: Shi, Baifeng, et al.
Published: (2025)

Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning
by: Huang, Shih-Cheng, et al.
Published: (2022)

Cultural Differences in Thinking Outside of Box: The Influence of Context‐Independent Versus Context‐Dependent Thinking Styles on Creative Task Performance
by: Wenxia Guo, et al.
Published: (2025)

3D Aware Region Prompted Vision Language Model
by: Cheng, An-Chieh, et al.
Published: (2025)

A deeper look at depth pruning of LLMs
by: Siddiqui, Shoaib Ahmed, et al.
Published: (2024)

C-RADIOv4 (Tech Report)
by: Ranzinger, Mike, et al.
Published: (2026)

Eigenspaces of Newforms with Nontrivial Character
by: Karameris, Markos
Published: (2023)

Eigenspaces of Coleman's Trace Operator
by: DiCapua, Joseph
Published: (2024)

Learning Laplacian Eigenspace with Mass-Aware Neural Operators on Point Clouds
by: Yang, Zherui, et al.
Published: (2026)

Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
by: Diao, Shizhe, et al.
Published: (2025)

COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimation
by: Li, Jiefeng, et al.
Published: (2024)

Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
by: Taghibakhshi, Ali, et al.
Published: (2025)