Cover Image

Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Fu, Yao, Li, Runchao, Long, Xianxuan, Yu, Haotian, Han, Xiaotian, Yin, Yu, Li, Pan
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When Truthful Representations Flip Under Deceptive Instructions?
by: Long, Xianxuan, et al.
Published: (2025)

Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models
by: Fu, Yao, et al.
Published: (2024)

Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs
by: Fu, Yao, et al.
Published: (2025)

FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression
by: Li, Runchao, et al.
Published: (2025)

TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning
by: Wei, Zhepei, et al.
Published: (2025)

Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?
by: Wang, Shouren, et al.
Published: (2025)

TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
by: Zhang, Shaolei, et al.
Published: (2024)

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
by: Kolawole, Steven, et al.
Published: (2024)

On Pruning State-Space LLMs
by: Ghattas, Tamer, et al.
Published: (2025)

Compressing LLMs: The Truth is Rarely Pure and Never Simple
by: Jaiswal, Ajay, et al.
Published: (2023)

Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
by: Cao, Mingyu, et al.
Published: (2024)

Tamper-Resistant Safeguards for Open-Weight LLMs
by: Tamirisa, Rishub, et al.
Published: (2024)

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing
by: Le, Qi, et al.
Published: (2025)

MoreauPruner: Robust Pruning of Large Language Models against Weight Perturbations
by: Wang, Zixiao, et al.
Published: (2024)

Pruning Strategies for Backdoor Defense in LLMs
by: Chapagain, Santosh, et al.
Published: (2025)

STUN: Structured-Then-Unstructured Pruning for Scalable MoE Pruning
by: Lee, Jaeseong, et al.
Published: (2024)

Demystifying When Pruning Works via Representation Hierarchies
by: He, Shwai, et al.
Published: (2026)

Fast and Effective Weight Update for Pruned Large Language Models
by: Boža, Vladimír
Published: (2024)

TruthFlow: Truthful LLM Generation via Representation Flow Correction
by: Wang, Hanyu, et al.
Published: (2025)

Adaptive Pruning for Large Language Models with Structural Importance Awareness
by: Zheng, Haotian, et al.
Published: (2024)

VOCABTRIM: Vocabulary Pruning for Efficient Speculative Decoding in LLMs
by: Goel, Raghavv, et al.
Published: (2025)

Pruning Unsafe Tickets: A Resource-Efficient Framework for Safer and More Robust LLMs
by: Si, Wai Man, et al.
Published: (2026)

When Bias Pretends to Be Truth: How Spurious Correlations Undermine Hallucination Detection in LLMs
by: Wang, Shaowen, et al.
Published: (2025)

Efficient Mathematical Reasoning Models via Dynamic Pruning and Knowledge Distillation
by: Yu, Fengming, et al.
Published: (2025)

The Truth Lies Somewhere in the Middle (of the Generated Tokens)
by: Wang, Sophie L., et al.
Published: (2026)

Dynamic Vocabulary Pruning in Early-Exit LLMs
by: Vincenti, Jort, et al.
Published: (2024)

Soft Tokens, Hard Truths
by: Butt, Natasha, et al.
Published: (2025)

LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning
by: Liu, Zihang, et al.
Published: (2025)

DenoiseRotator: Enhance Pruning Robustness for LLMs via Importance Concentration
by: Gu, Tianteng, et al.
Published: (2025)

How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence
by: Du, Hongzhe, et al.
Published: (2025)

IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining
by: Li, Yixiao, et al.
Published: (2025)

From Pruning to Grafting: Dynamic Knowledge Redistribution via Learnable Layer Fusion
by: Pei, Zehua, et al.
Published: (2024)

Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity
by: Yu, Lei, et al.
Published: (2024)

FlexiGPT: Pruning and Extending Large Language Models with Low-Rank Weight Sharing
by: Smith, James Seale, et al.
Published: (2025)

Two-Stage Regularization-Based Structured Pruning for LLMs
by: Feng, Mingkuan, et al.
Published: (2025)

REAM: Merging Improves Pruning of Experts in LLMs
by: Jha, Saurav, et al.
Published: (2026)

The Geometries of Truth Are Orthogonal Across Tasks
by: Azizian, Waiss, et al.
Published: (2025)

Think Before You Prune: Self-Reflective Structured Pruning for Reasoning Language Models
by: Wang, Ziyan, et al.
Published: (2025)

LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblance
by: Haller, Patrick, et al.
Published: (2025)

Non-Linear Inference Time Intervention: Improving LLM Truthfulness
by: Hoscilowicz, Jakub, et al.
Published: (2024)