Image de couverture de livre

Entropy-Based Data Selection for Language Models

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Li, Hongming, Liu, Yang, Huang, Chao
Format:	Preprint
Publié:	2026
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Revisiting a Pain in the Neck: A Semantic Reasoning Benchmark for Language Models
par: Liu, Yang, et autres
Publié: (2026)

Revisiting a Pain in the Neck: Semantic Phrase Processing Benchmark for Language Models
par: Liu, Yang, et autres
Publié: (2024)

Neuron-Aware Data Selection In Instruction Tuning For Large Language Models
par: Chen, Xin, et autres
Publié: (2026)

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models
par: Zhang, Zhisong, et autres
Publié: (2024)

Entropy-Based Decoding for Retrieval-Augmented Large Language Models
par: Qiu, Zexuan, et autres
Publié: (2024)

Data Selection via Optimal Control for Language Models
par: Gu, Yuxian, et autres
Publié: (2024)

InstructDiff: Domain-Adaptive Data Selection via Differential Entropy for Efficient LLM Fine-Tuning
par: Su, Junyou, et autres
Publié: (2026)

ZPD Detector: Data Selection via Capability-Difficulty Alignment for Large Language Models
par: Yang, Bo, et autres
Publié: (2026)

Enhancing Large Language Models with Domain-Specific Knowledge: The Case in Topological Materials
par: Xu, HuangChao, et autres
Publié: (2024)

Cost-efficient Crowdsourcing for Span-based Sequence Labeling: Worker Selection and Data Augmentation
par: Wang, Yujie, et autres
Publié: (2023)

InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing
par: Li, Shuaiyi, et autres
Publié: (2025)

Entropy-Based Block Pruning for Efficient Large Language Models
par: Yang, Liangwei, et autres
Publié: (2025)

On-the-fly Denoising for Data Augmentation in Natural Language Understanding
par: Fang, Tianqing, et autres
Publié: (2022)

Entropy-Guided Token Dropout: Training Autoregressive Language Models with Limited Domain Data
par: Wang, Jiapeng, et autres
Publié: (2025)

Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models
par: Liu, Ziche, et autres
Publié: (2024)

Entropy in Large Language Models
par: Scharringhausen, Marco
Publié: (2026)

SocREval: Large Language Models with the Socratic Method for Reference-Free Reasoning Evaluation
par: He, Hangfeng, et autres
Publié: (2023)

A Survey on Data Selection for Language Models
par: Albalak, Alon, et autres
Publié: (2024)

Structural-Entropy-Based Sample Selection for Efficient and Effective Learning
par: Xie, Tianchi, et autres
Publié: (2024)

Entropy2Vec: Crosslingual Language Modeling Entropy as End-to-End Learnable Language Representations
par: Irawan, Patrick Amadeus, et autres
Publié: (2025)

Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection
par: Chen, Ruibo, et autres
Publié: (2024)

OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration
par: Wang, Shaobo, et autres
Publié: (2026)

Investigating the Impact of Data Selection Strategies on Language Model Performance
par: Gu, Jiayao, et autres
Publié: (2025)

EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Sampling
par: Zhang, Shimao, et autres
Publié: (2024)

Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model
par: Huang, Chen, et autres
Publié: (2024)

DavIR: Data Selection via Implicit Reward for Large Language Models
par: Zhou, Haotian, et autres
Publié: (2023)

DaMoC: Efficiently Selecting the Optimal Large Language Model for Fine-tuning Domain Tasks Based on Data and Model Compression
par: Huang, Wei, et autres
Publié: (2025)

Knowledge-Driven Feature Selection and Engineering for Genotype Data with Large Language Models
par: Lee, Joseph, et autres
Publié: (2024)

Efficient Pretraining Data Selection for Language Models via Multi-Actor Collaboration
par: Bai, Tianyi, et autres
Publié: (2024)

Thrust: Adaptively Propels Large Language Models with External Knowledge
par: Zhao, Xinran, et autres
Publié: (2023)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
par: Ruan, Zhiwen, et autres
Publié: (2025)

Enhancing Large Language Model Performance with Gradient-Based Parameter Selection
par: Li, Haoling, et autres
Publié: (2024)

CLOMO: Counterfactual Logical Modification with Large Language Models
par: Huang, Yinya, et autres
Publié: (2023)

Efficient Beam Search for Large Language Models Using Trie-Based Decoding
par: Chan, Brian J, et autres
Publié: (2025)

EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models
par: Cheong, Minsoo, et autres
Publié: (2026)

LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
par: Wang, Jingyuan, et autres
Publié: (2025)

Explore Data Left Behind in Reinforcement Learning for Reasoning Language Models
par: Liu, Chenxi, et autres
Publié: (2025)

FroM: Frobenius Norm-Based Data-Free Adaptive Model Merging
par: Li, Zijian, et autres
Publié: (2025)

Influential Language Data Selection via Gradient Trajectory Pursuit
par: Deng, Zhiwei, et autres
Publié: (2024)

Conceptual and Unbiased Reasoning in Language Models
par: Zhou, Ben, et autres
Publié: (2024)