Cover Image

Winning the Pruning Gamble: A Unified Approach to Joint Sample and Token Pruning for Efficient Supervised Fine-Tuning

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Shaobo, Wang, Jiaming, Zhang, Jiajun, Wang, Cong, Min, Yue, Wen, Zichen, Ren, Xingzhang, Huang, Fei, Jiang, Huiqiang, Lin, Junyang, Liu, Dayiheng, Zhang, Linfeng
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

UNSEEN: Enhancing Dataset Pruning from a Generalization Perspective
by: Xu, Furui, et al.
Published: (2025)

DISA: Offline Importance Sampling for Distribution-Matching LLM-RL
by: Wang, Shaobo, et al.
Published: (2026)

The Missing Piece in Pre-trained Model Evaluation: Reward-Guided Decoding Unlocks Task-Oriented Behavior Without Parameter Updates
by: Wang, Shaobo, et al.
Published: (2026)

Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?
by: Wen, Zichen, et al.
Published: (2025)

Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs
by: Zhou, Yufa, et al.
Published: (2025)

D2Pruner: Debiased Importance and Structural Diversity for MLLM Token Pruning
by: Zhang, Evelyn, et al.
Published: (2025)

Rethinking LLM Evaluation: Can We Evaluate LLMs with 200x Less Data?
by: Wang, Shaobo, et al.
Published: (2025)

Comb, Prune, Distill: Towards Unified Pruning for Vision Model Compression
by: Schmitt, Jonas, et al.
Published: (2024)

ST-Prune: Training-Free Spatio-Temporal Token Pruning for Vision-Language Models in Autonomous Driving
by: Sha, Lin, et al.
Published: (2026)

Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning
by: Wang, Shaobo, et al.
Published: (2025)

Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs
by: Zhang, Qizhe, et al.
Published: (2024)

PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference
by: Li, Ye, et al.
Published: (2024)

UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking
by: Wu, Hao, et al.
Published: (2026)

Towards Joint Quantization and Token Pruning of Vision-Language Models
by: Li, Xinqing, et al.
Published: (2026)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
by: Wang, Shaobo, et al.
Published: (2026)

SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Acceleration
by: Li, Ye, et al.
Published: (2025)

TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model
by: Li, Ao, et al.
Published: (2025)

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding
by: Zhang, Yike, et al.
Published: (2025)

Stop Looking for Important Tokens in Multimodal Language Models: Duplication Matters More
by: Wen, Zichen, et al.
Published: (2025)

Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networks
by: Yue, Yanwei, et al.
Published: (2024)

Diffusion LLM with Native Variable Generation Lengths: Let [EOS] Lead the Way
by: Yang, Yicun, et al.
Published: (2025)

Fast SAM2 with Text-Driven Token Pruning
by: Mandal, Avilasha, et al.
Published: (2025)

CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuning
by: Meng, Fanxu, et al.
Published: (2024)

SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval
by: Lin, Yueqian, et al.
Published: (2024)

Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning
by: Hasan, Adib, et al.
Published: (2024)

Token Pruning for Caching Better: 9 Times Acceleration on Stable Diffusion for Free
by: Zhang, Evelyn, et al.
Published: (2024)

An Empirical Study of Parameter Efficient Fine-tuning on Vision-Language Pre-train Model
by: Tian, Yuxin, et al.
Published: (2024)

GridPrune: From "Where to Look" to "What to Select" in Visual Token Pruning for MLLMs
by: Duan, Yuxiang, et al.
Published: (2025)

HiPrune: Hierarchical Attention for Efficient Token Pruning in Vision-Language Models
by: Liu, Jizhihui, et al.
Published: (2025)

Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Caching
by: Zheng, Zhixin, et al.
Published: (2025)

DivPrune: Diversity-based Visual Token Pruning for Large Multimodal Models
by: Alvar, Saeed Ranjbar, et al.
Published: (2025)

SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
by: Wang, Hanrui, et al.
Published: (2020)

Effective Fine-Tuning with Eigenvector Centrality Based Pruning
by: Chowdhury, Shaif, et al.
Published: (2025)

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning
by: Gu, Naibin, et al.
Published: (2024)

Prune, Don't Rebuild: Efficiently Tuning $α$-Reachable Graphs for Nearest Neighbor Search
by: Zhang, Tian, et al.
Published: (2026)

Layer Pruning with Consensus: A Triple-Win Solution
by: Mugnaini, Leandro Giusti, et al.
Published: (2024)

Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs
by: Kim, Kibum, et al.
Published: (2026)

GradPruner: Gradient-Guided Layer Pruning Enabling Efficient Fine-Tuning and Inference for LLMs
by: Huang, Wei, et al.
Published: (2026)

Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models
by: Ke, Junlong, et al.
Published: (2026)

PAT: Pruning-Aware Tuning for Large Language Models
by: Liu, Yijiang, et al.
Published: (2024)