Image de couverture de livre

Measuring Sample Importance in Data Pruning for Language Models based on Information Entropy

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Kim, Minsang, Baek, Seungjun
Format:	Preprint
Publié:	2024
Sujets:	Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Hierarchical Position Embedding of Graphs with Landmarks and Clustering for Link Prediction
par: Kim, Minsang, et autres
Publié: (2024)

Exploring Large Language Models on Cross-Cultural Values in Connection with Training Methodology
par: Kim, Minsang, et autres
Publié: (2024)

Syntriever: How to Train Your Retriever with Synthetic Data from LLMs
par: Kim, Minsang, et autres
Publié: (2025)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
par: Kim, Minsang, et autres
Publié: (2026)

Entropy Meets Importance: A Unified Head Importance-Entropy Score for Stable and Efficient Transformer Pruning
par: Choi, Minsik, et autres
Publié: (2025)

QPaug: Question and Passage Augmentation for Open-Domain Question Answering of LLMs
par: Kim, Minsang, et autres
Publié: (2024)

Lookahead Sample Reward Guidance for Test-Time Scaling of Diffusion Models
par: Kim, Yeongmin, et autres
Publié: (2026)

ESPO: Entropy Importance Sampling Policy Optimization
par: Sheng, Yuepeng, et autres
Publié: (2025)

IPPRO: Importance-based Pruning with PRojective Offset for Magnitude-indifferent Structural Pruning
par: Jung, Jaeheun, et autres
Publié: (2025)

Adaptive Pruning for Large Language Models with Structural Importance Awareness
par: Zheng, Haotian, et autres
Publié: (2024)

Graph-Assisted Stitching for Offline Hierarchical Reinforcement Learning
par: Baek, Seungho, et autres
Publié: (2025)

Importance Sampling for Nonlinear Models
par: Rajmohan, Prakash Palanivelu, et autres
Publié: (2025)

Development of a Large Language Model-based Multi-Agent Clinical Decision Support System for Korean Triage and Acuity Scale (KTAS)-Based Triage and Treatment Planning in Emergency Departments
par: Han, Seungjun, et autres
Publié: (2024)

Severing Spurious Correlations with Data Pruning
par: Mulchandani, Varun, et autres
Publié: (2025)

On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
par: Shrestha, Safal, et autres
Publié: (2026)

Sample-aware Adaptive Structured Pruning for Large Language Models
par: Kong, Jun, et autres
Publié: (2025)

Pruning and Distilling Mixture-of-Experts into Dense Language Models
par: Kim, Junhyuck, et autres
Publié: (2026)

Toward Adaptive Large Language Models Structured Pruning via Hybrid-grained Weight Importance Assessment
par: Liu, Jun, et autres
Publié: (2024)

SwiftPrune: Hessian-Free Weight Pruning for Large Language Models
par: Kang, Yuhan, et autres
Publié: (2025)

Partial Domain Adaptation via Importance Sampling-based Shift Correction
par: Guo, Cheng-Jun, et autres
Publié: (2025)

Low Variance Off-policy Evaluation with State-based Importance Sampling
par: Bossens, David M., et autres
Publié: (2022)

Hospitality-VQA: Decision-Oriented Informativeness Evaluation for Vision-Language Models
par: Lee, Jeongwoo, et autres
Publié: (2026)

Structured Temporal Causality for Interpretable Multivariate Time Series Anomaly Detection
par: Cho, Dongchan, et autres
Publié: (2025)

Symmetric Pruning of Large Language Models
par: Yi, Kai, et autres
Publié: (2025)

Beware of Calibration Data for Pruning Large Language Models
par: Ji, Yixin, et autres
Publié: (2024)

Exploring Federated Pruning for Large Language Models
par: Guo, Pengxin, et autres
Publié: (2025)

AIS: Adaptive Importance Sampling for Quantized RL
par: Zhou, Jiajun, et autres
Publié: (2026)

GIPO: Gaussian Importance Sampling Policy Optimization
par: Lu, Chengxuan, et autres
Publié: (2026)

Small Contributions, Small Networks: Efficient Neural Network Pruning Based on Relative Importance
par: Hussien, Mostafa, et autres
Publié: (2024)

Preference as Reward, Maximum Preference Optimization with Importance Sampling
par: Jiang, Zaifan, et autres
Publié: (2023)

Beyond Importance Sampling: Rejection-Gated Policy Optimization
par: Sun, Ziwu, et autres
Publié: (2026)

Feature Importance Depends on Properties of the Data: Towards Choosing the Correct Explanations for Your Data and Decision Trees based Models
par: Ayad, Célia Wafa, et autres
Publié: (2025)

Learning Covariance-Based Multi-Scale Representation of Neuroimaging Measures for Alzheimer Classification
par: Baek, Seunghun, et autres
Publié: (2025)

Pruning-based Data Selection and Network Fusion for Efficient Deep Learning
par: Kousar, Humaira, et autres
Publié: (2025)

MPruner: Optimizing Neural Network Size with CKA-Based Mutual Information Pruning
par: Hu, Seungbeom, et autres
Publié: (2024)

A Novel Double Pruning method for Imbalanced Data using Information Entropy and Roulette Wheel Selection for Breast Cancer Diagnosis
par: Bacha, Soufiane, et autres
Publié: (2025)

Large Language Model Pruning
par: Huang, Hanjuan, et autres
Publié: (2024)

TopoPrune: Robust Data Pruning via Unified Latent Space Topology
par: Roy, Arjun, et autres
Publié: (2026)

EMP: Enhance Memory in Data Pruning
par: Xiao, Jinying, et autres
Publié: (2024)

Sparsest Models Elude Pruning: An Exposé of Pruning's Current Capabilities
par: Zhang, Stephen, et autres
Publié: (2024)