Copertina

Activation Compression in LLMs: Theoretical Analysis and Efficient Algorithm

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Wei, Wen-Da, Fang, Han-Bin, Liu, Yang-Di, Shi, Jiang-Xin, Kwok, James, Li, Yu-Feng
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
di: Shi, Jiang-Xin, et al.
Pubblicazione: (2025)

PRAC: Principal-Random Subspace for LLM Activation Compression and Memory-Efficient Training
di: Li, Yanyi, et al.
Pubblicazione: (2026)

Efficient Pareto Manifold Learning with Low-Rank Structure
di: Chen, Weiyu, et al.
Pubblicazione: (2024)

Learning Scalable Model Soup on a Single GPU: An Efficient Subspace Training Strategy
di: Li, Tao, et al.
Pubblicazione: (2024)

Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging
di: Li, Lujun, et al.
Pubblicazione: (2025)

Communication-Efficient and Privacy-Preserving Decentralized Meta-Learning
di: Yang, Hansi, et al.
Pubblicazione: (2024)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
di: Xi, Haocheng, et al.
Pubblicazione: (2024)

Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
di: Xu, Tian, et al.
Pubblicazione: (2026)

AutoHete: An Automatic and Efficient Heterogeneous Training System for LLMs
di: Zeng, Zihao, et al.
Pubblicazione: (2025)

Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learning
di: Jiang, Yu, et al.
Pubblicazione: (2024)

ReLU$^2$ Wins: Discovering Efficient Activation Functions for Sparse LLMs
di: Zhang, Zhengyan, et al.
Pubblicazione: (2024)

Forward-Backward Reasoning in Large Language Models for Mathematical Verification
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

RAST: Reasoning Activation in LLMs via Small-model Transfer
di: Ouyang, Siru, et al.
Pubblicazione: (2025)

Efficient and Long-Tailed Generalization for Pre-trained Vision-Language Model
di: Shi, Jiang-Xin, et al.
Pubblicazione: (2024)

Efficient Algorithms for Empirical Group Distributionally Robust Optimization and Beyond
di: Yu, Dingzhi, et al.
Pubblicazione: (2024)

Distributed Online Convex Optimization with Efficient Communication: Improved Algorithm and Lower bounds
di: Yang, Sifan, et al.
Pubblicazione: (2026)

Efficient Federated Unlearning with Adaptive Differential Privacy Preservation
di: Jiang, Yu, et al.
Pubblicazione: (2024)

Effective Structured Prompting by Meta-Learning and Representative Verbalizer
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

Efficient Model Compression for Bayesian Neural Networks
di: Saha, Diptarka, et al.
Pubblicazione: (2024)

Orthogonal Subspace Clustering: Enhancing High-Dimensional Data Analysis through Adaptive Dimensionality Reduction and Efficient Clustering
di: Wen, Qing-Yuan, et al.
Pubblicazione: (2026)

Why Do More Experts Fail? A Theoretical Analysis of Model Merging
di: Wang, Zijing, et al.
Pubblicazione: (2025)

Gradient-Based Multi-Objective Deep Learning: Algorithms, Theories, Applications, and Beyond
di: Chen, Weiyu, et al.
Pubblicazione: (2025)

Compress the Easy, Explore the Hard: Difficulty-Aware Entropy Regularization for Efficient LLM Reasoning
di: Luo, Qin-Wen, et al.
Pubblicazione: (2026)

Taming Latency and Bandwidth: A Theoretical Framework and Adaptive Algorithm for Communication-Constrained Training
di: Lu, Rongwei, et al.
Pubblicazione: (2025)

Lyapunov Analysis For Monotonically Forward-Backward Accelerated Algorithms
di: Fu, Mingwei, et al.
Pubblicazione: (2024)

Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
di: Solgi, Ryan, et al.
Pubblicazione: (2025)

BYOM: Building Your Own Multi-Task Model For Free
di: Jiang, Weisen, et al.
Pubblicazione: (2023)

CompAct: Compressed Activations for Memory-Efficient LLM Training
di: Shamshoum, Yara, et al.
Pubblicazione: (2024)

Open-Set Heterogeneous Domain Adaptation: Theoretical Analysis and Algorithm
di: Pham, Thai-Hoang, et al.
Pubblicazione: (2024)

Predictor-Rejector Multi-Class Abstention: Theoretical Analysis and Algorithms
di: Mao, Anqi, et al.
Pubblicazione: (2023)

NLI:Non-uniform Linear Interpolation Approximation of Nonlinear Operations for Efficient LLMs Inference
di: Yu, Jiangyong, et al.
Pubblicazione: (2026)

Theoretical Analysis of Robust Overfitting for Wide DNNs: An NTK Approach
di: Fu, Shaopeng, et al.
Pubblicazione: (2023)

3BASiL: An Algorithmic Framework for Sparse plus Low-Rank Compression of LLMs
di: Makni, Mehdi, et al.
Pubblicazione: (2026)

Delta Decompression for MoE-based LLMs Compression
di: Gu, Hao, et al.
Pubblicazione: (2025)

The Lie of the Average: How Class Incremental Learning Evaluation Deceives You?
di: Lai, Guannan, et al.
Pubblicazione: (2025)

Efficient and Optimal Policy Gradient Algorithm for Corrupted Multi-armed Bandits
di: Liu, Jiayuan, et al.
Pubblicazione: (2025)

LIFT+: Lightweight Fine-Tuning for Long-Tail Learning
di: Shi, Jiang-Xin, et al.
Pubblicazione: (2025)

Decoupling Contrastive Decoding: Robust Hallucination Mitigation in Multimodal Large Language Models
di: Chen, Wei, et al.
Pubblicazione: (2025)

Pareto Merging: Multi-Objective Optimization for Preference-Aware Model Merging
di: Chen, Weiyu, et al.
Pubblicazione: (2024)

A General Error-Theoretical Analysis Framework for Constructing Compression Strategies
di: Zhang, Boyang, et al.
Pubblicazione: (2025)