Image de couverture de livre

ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Yang, Yan, Li, Yixia, Wang, Hongru, Wei, Xuetao, Yu, Jianqiao, Chen, Yun, Chen, Guanhua
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters
par: Yang, Yan, et autres
Publié: (2024)

Enhancing Delta Compression in LLMs via SVD-based Quantization Error Minimization
par: Xiong, Boya, et autres
Publié: (2025)

SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation
par: Li, Yixia, et autres
Publié: (2024)

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning
par: Wang, Hanqing, et autres
Publié: (2024)

PACIT: Unlocking the Power of Examples for Better In-Context Instruction Tuning
par: Xue, Tianci, et autres
Publié: (2023)

G2: Guided Generation for Enhanced Output Diversity in LLMs
par: Ruan, Zhiwen, et autres
Publié: (2025)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
par: Ruan, Zhiwen, et autres
Publié: (2025)

Modeling LLM Unlearning as an Asymmetric Two-Task Learning Problem
par: Xiao, Zeguan, et autres
Publié: (2026)

Compound-QA: A Benchmark for Evaluating LLMs on Compound Questions
par: Hou, Yutao, et autres
Publié: (2024)

From Word to World: Can Large Language Models be Implicit Text-based World Models?
par: Li, Yixia, et autres
Publié: (2025)

LayAlign: Enhancing Multilingual Reasoning in Large Language Models via Layer-Wise Adaptive Fusion and Alignment Strategy
par: Ruan, Zhiwen, et autres
Publié: (2025)

Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems
par: Wu, Wanxing, et autres
Publié: (2026)

One Size Does Not Fit All: A Distribution-Aware Sparsification for More Precise Model Merging
par: Luo, Yingfeng, et autres
Publié: (2025)

Distract Large Language Models for Automatic Jailbreak Attack
par: Xiao, Zeguan, et autres
Publié: (2024)

Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
par: Ruan, Zhiwen, et autres
Publié: (2025)

Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models
par: Ping, Bowen, et autres
Publié: (2024)

Pruning via Merging: Compressing LLMs via Manifold Alignment Based Layer Merging
par: Liu, Deyuan, et autres
Publié: (2024)

Enhancing Uncertainty Estimation in LLMs with Expectation of Aggregated Internal Belief
par: Xiao, Zeguan, et autres
Publié: (2025)

VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models
par: Jiang, Lingjie, et autres
Publié: (2025)

Optimal Brain Restoration for Joint Quantization and Sparsification of LLMs
par: Guo, Hang, et autres
Publié: (2025)

EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance
par: Li, Yingxin, et autres
Publié: (2024)

Be Cautious When Merging Unfamiliar LLMs: A Phishing Model Capable of Stealing Privacy
par: Guo, Zhenyuan, et autres
Publié: (2025)

MTA: A Merge-then-Adapt Framework for Personalized Large Language Model
par: Li, Xiaopeng, et autres
Publié: (2025)

DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging
par: Lin, Tzu-Han, et autres
Publié: (2024)

Extrapolation Merging: Keep Improving With Extrapolation and Merging
par: Lin, Yiguan, et autres
Publié: (2025)

Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging
par: Lu, Zhenyi, et autres
Publié: (2024)

DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens
par: Chen, Shaoshen, et autres
Publié: (2025)

NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation
par: Wang, Xiaoyang, et autres
Publié: (2021)

Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks
par: Wang, Zheng, et autres
Publié: (2024)

Towards Bridging the Reward-Generation Gap in Direct Alignment Algorithms
par: Xiao, Zeguan, et autres
Publié: (2025)

Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages
par: chi, Yongdong, et autres
Publié: (2025)

REAM: Merging Improves Pruning of Experts in LLMs
par: Jha, Saurav, et autres
Publié: (2026)

CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging
par: Yang, Zongzhen, et autres
Publié: (2025)

SE-Merging: A Self-Enhanced Approach for Dynamic Model Merging
par: Chen, Zijun, et autres
Publié: (2025)

STAR: Spectral Truncation and Rescale for Model Merging
par: Lee, Yu-Ang, et autres
Publié: (2025)

PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression
par: Chen, Lizhe, et autres
Publié: (2025)

The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging
par: Lan, Xiaochong, et autres
Publié: (2025)

Importance Sparsification for Sinkhorn Algorithm
par: Li, Mengyu, et autres
Publié: (2023)

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations
par: Lai, Peng, et autres
Publié: (2025)

Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Merging
par: Farn, Hua, et autres
Publié: (2024)