Cover Image

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Solgi, Ryan, Madinei, Parsa, Tian, Jiayi, Swaminathan, Rupak, Liu, Jing, Susanj, Nathan, Zhang, Zheng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Saten: Sparse Augmented Tensor Networks for Post-Training Compression of Large Language Models
by: Solgi, Ryan, et al.
Published: (2025)

SharpZO: Hybrid Sharpness-Aware Vision Language Model Prompt Tuning via Forward-Only Passes
by: Yang, Yifan, et al.
Published: (2025)

FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression
by: Tian, Jiayi, et al.
Published: (2025)

RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoning
by: Tian, Jiayi, et al.
Published: (2026)

INTERLACE: Interleaved Layer Pruning and Efficient Adaptation in Large Vision-Language Models
by: Madinei, Parsa, et al.
Published: (2025)

CompAct: Compressed Activations for Memory-Efficient LLM Training
by: Shamshoum, Yara, et al.
Published: (2024)

Wanda++: Pruning Large Language Models via Regional Gradients
by: Yang, Yifan, et al.
Published: (2025)

Low-Rank Adapters Meet Neural Architecture Search for LLM Compression
by: Muñoz, J. Pablo, et al.
Published: (2025)

Do Tensorized Large-Scale Spatiotemporal Dynamic Atmospheric Data Exhibit Low-Rank Properties?
by: Solgi, Ryan, et al.
Published: (2025)

ARChef: An iOS-Based Augmented Reality Cooking Assistant Powered by Multimodal Gemini LLM
by: Vir, Rithik, et al.
Published: (2024)

Revealing the Gap in Human and VLM Scene Perception through Counterfactual Semantic Saliency
by: Wen, Ziqi, et al.
Published: (2026)

ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization
by: Liu, Zechun, et al.
Published: (2025)

Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization
by: Tian, Jiayi, et al.
Published: (2025)

DynSplit-KV: Dynamic Semantic Splitting for KVCache Compression in Efficient Long-Context LLM Inference
by: Ye, Jiancai, et al.
Published: (2026)

Contextually Guided Transformers via Low-Rank Adaptation
by: Zhmoginov, Andrey, et al.
Published: (2025)

Aligning LLMs with Human Uncertainty: A Beta-Bernoulli Calibrator for LLM Forecasting
by: Dai, Hui, et al.
Published: (2026)

Review, Remask, Refine (R3): Process-Guided Block Diffusion for Text Generation
by: Mounier, Nikita, et al.
Published: (2025)

PocketLLM: Ultimate Compression of Large Language Models via Meta Networks
by: Tian, Ye, et al.
Published: (2025)

Merging Beyond: Streaming LLM Updates via Activation-Guided Rotations
by: Yao, Yuxuan, et al.
Published: (2026)

Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation
by: Li, Shiwei, et al.
Published: (2025)

Efficient Pareto Manifold Learning with Low-Rank Structure
by: Chen, Weiyu, et al.
Published: (2024)

LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
by: Zhang, Rongzhi, et al.
Published: (2024)

Fake News Detection After LLM Laundering: Measurement and Explanation
by: Das, Rupak Kumar, et al.
Published: (2025)

Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
by: Saxena, Utkarsh, et al.
Published: (2024)

QUAD: Quantization and Parameter-Efficient Tuning of LLM with Activation Decomposition
by: Hu, Yuxuan, et al.
Published: (2025)

Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
by: Ma, Weiyu, et al.
Published: (2026)

Adaptive Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
by: Ji, Yixin, et al.
Published: (2024)

KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
by: Tian, Yuxuan, et al.
Published: (2025)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

Accelerating the Low-Rank Decomposed Models
by: Hajimolahoseini, Habib, et al.
Published: (2024)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

FishBack: Pullback Fisher Geometry for Optimal Activation Steering in Transformers
by: Wang, Sihan, et al.
Published: (2026)

LASER: Low-Rank Activation SVD for Efficient Recursion
by: Çakar, Ege, et al.
Published: (2026)

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation
by: Sengupta, Ayan, et al.
Published: (2024)

Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences
by: Dimitriadis, Nikolaos, et al.
Published: (2024)

TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly
by: Koike-Akino, Toshiaki, et al.
Published: (2026)

The Evolving Landscape of LLM- and VLM-Integrated Reinforcement Learning
by: Schoepp, Sheila, et al.
Published: (2025)

LoRA+: Efficient Low Rank Adaptation of Large Models
by: Hayou, Soufiane, et al.
Published: (2024)

LANCE: Low Rank Activation Compression for Efficient On-Device Continual Learning
by: Apolinario, Marco Paul E., et al.
Published: (2025)

GRASP: Replace Redundant Layers with Adaptive Singular Parameters for Efficient Model Compression
by: Liu, Kainan, et al.
Published: (2024)