FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909835051663360 |
|---|---|
| author | Lu, Yu-Chen Chen, Chong-Yan Chang, Chi-Chih Hu, Yu-Fang Wu, Kai-Chiang |
| author_facet | Lu, Yu-Chen Chen, Chong-Yan Chang, Chi-Chih Hu, Yu-Fang Wu, Kai-Chiang |
| contents | Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression can reduce both memory usage and computational demand, but applying a uniform compression ratio across all layers often leads to significant performance degradation, and previous methods perform poorly during decoding. To address these issues, we propose the Fine-grained Low-Rank Compressor (FLRC), which efficiently determines an optimal rank allocation for each layer, and incorporates progressive low-rank decoding to maintain text generation quality. Comprehensive experiments on diverse benchmarks demonstrate the superiority of FLRC, achieving up to a 17% improvement in ROUGE-L on summarization tasks compared to state-of-the-art low-rank compression methods, establishing a more robust and efficient framework to improve LLM inference. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_09332 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference Lu, Yu-Chen Chen, Chong-Yan Chang, Chi-Chih Hu, Yu-Fang Wu, Kai-Chiang Computation and Language Artificial Intelligence Although large language models (LLM) have achieved remarkable performance, their enormous parameter counts hinder deployment on resource-constrained hardware. Low-rank compression can reduce both memory usage and computational demand, but applying a uniform compression ratio across all layers often leads to significant performance degradation, and previous methods perform poorly during decoding. To address these issues, we propose the Fine-grained Low-Rank Compressor (FLRC), which efficiently determines an optimal rank allocation for each layer, and incorporates progressive low-rank decoding to maintain text generation quality. Comprehensive experiments on diverse benchmarks demonstrate the superiority of FLRC, achieving up to a 17% improvement in ROUGE-L on summarization tasks compared to state-of-the-art low-rank compression methods, establishing a more robust and efficient framework to improve LLM inference. |
| title | FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2510.09332 |