Imagen de Portada

Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Liu, Qi, Sun, Mingdi, He, Yongyi, Zheng, Zhi, Xu, Tong, Zheng, Yi, Wang, Zhefeng, Chen, Enhong
Formato:	Preprint
Publicado:	2026
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

UniMEL: A Unified Framework for Multimodal Entity Linking with Large Language Models
por: Qi, Liu, et al.
Publicado: (2024)

In-Context Former: Lightning-fast Compressing Context for Large Language Model
por: Wang, Xiangfeng, et al.
Publicado: (2024)

Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verification
por: Zheng, Zhi, et al.
Publicado: (2025)

Discovering Decoupled Functional Modules in Large Language Models
por: Yu, Yanke, et al.
Publicado: (2026)

Semantic-aware Token Selection and Resource Optimization for Communication-efficient Split Federated Fine-tuning in Edge Intelligence
por: Qiang, Xianke, et al.
Publicado: (2026)

A Convex-optimization-based Layer-wise Post-training Pruner for Large Language Models
por: Zhao, Pengxiang, et al.
Publicado: (2024)

RKLD: Reverse KL-Divergence-based Knowledge Distillation for Unlearning Personal Information in Large Language Models
por: Wang, Bichen, et al.
Publicado: (2024)

Adaptive Divergence Regularized Policy Optimization for Fine-tuning Generative Models
por: Fan, Jiajun, et al.
Publicado: (2025)

Token-level Data Selection for Safe LLM Fine-tuning
por: Li, Yanping, et al.
Publicado: (2026)

Targeted Efficient Fine-tuning: Optimizing Parameter Updates with Data-Driven Sample Selection
por: Dong, Ming, et al.
Publicado: (2024)

Disentangling Reasoning Tokens and Boilerplate Tokens For Language Model Fine-tuning
por: Ye, Ziang, et al.
Publicado: (2024)

ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning
por: Qin, Xiaohan, et al.
Publicado: (2025)

A Survey on Federated Fine-tuning of Large Language Models
por: Wu, Yebo, et al.
Publicado: (2025)

FASP: Fast and Accurate Structured Pruning of Large Language Models
por: Hu, Hanyu, et al.
Publicado: (2025)

More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing
por: Ma, Xin, et al.
Publicado: (2026)

MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning
por: Zhang, Jingfan, et al.
Publicado: (2024)

Generalized Munchausen Reinforcement Learning using Tsallis KL Divergence
por: Zhu, Lingwei, et al.
Publicado: (2023)

DynLLM: When Large Language Models Meet Dynamic Graph Recommendation
por: Zhao, Ziwei, et al.
Publicado: (2024)

Fine-tuning Large Language Models for Domain-specific Machine Translation
por: Zheng, Jiawei, et al.
Publicado: (2024)

CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models
por: Zhou, Nan, et al.
Publicado: (2026)

Sparse Matrix in Large Language Model Fine-tuning
por: He, Haoze, et al.
Publicado: (2024)

AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding
por: Qi, Haozhe, et al.
Publicado: (2026)

Influence-Preserving Proxies for Gradient-Based Data Selection in LLM Fine-tuning
por: Chen, Sirui, et al.
Publicado: (2026)

Privacy in Fine-tuning Large Language Models: Attacks, Defenses, and Future Directions
por: Du, Hao, et al.
Publicado: (2024)

ByteScience: Bridging Unstructured Scientific Literature and Structured Data with Auto Fine-tuned Large Language Model in Token Granularity
por: Xie, Tong, et al.
Publicado: (2024)

Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuning
por: Ruan, Zhiwen, et al.
Publicado: (2025)

Practical Secure Inference Algorithm for Fine-tuned Large Language Model Based on Fully Homomorphic Encryption
por: Ruoyan, Zhang, et al.
Publicado: (2025)

BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models
por: Ben-Zaken, Elad, et al.
Publicado: (2021)

Random Masking Finds Winning Tickets for Parameter Efficient Fine-tuning
por: Xu, Jing, et al.
Publicado: (2024)

Selecting Large Language Model to Fine-tune via Rectified Scaling Law
por: Lin, Haowei, et al.
Publicado: (2024)

Realistic Unsupervised CLIP Fine-tuning with Universal Entropy Optimization
por: Liang, Jian, et al.
Publicado: (2023)

On the KL-Divergence-based Robust Satisficing Model
por: Yan, Haojie, et al.
Publicado: (2024)

Harmony in Divergence: Towards Fast, Accurate, and Memory-efficient Zeroth-order LLM Fine-tuning
por: Tan, Qitao, et al.
Publicado: (2025)

Towards Context-Robust LLMs: A Gated Representation Fine-tuning Approach
por: Zeng, Shenglai, et al.
Publicado: (2025)

Preference-grounded Token-level Guidance for Language Model Fine-tuning
por: Yang, Shentao, et al.
Publicado: (2023)

Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation
por: Xu, Derong, et al.
Publicado: (2024)

On $f$-Divergence Principled Domain Adaptation: An Improved Framework
por: Wang, Ziqiao, et al.
Publicado: (2024)

DiffuMask: Diffusion Language Model for Token-level Prompt Pruning
por: Zheng, Caleb, et al.
Publicado: (2026)

Fine-tuning Large Language Models for Multigenerator, Multidomain, and Multilingual Machine-Generated Text Detection
por: Xiong, Feng, et al.
Publicado: (2024)

BMFT: Achieving Fairness via Bias-based Weight Masking Fine-tuning
por: Xue, Yuyang, et al.
Publicado: (2024)