Cover Image

Confidence Preservation Property in Knowledge Distillation Abstractions

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Vengertsev, Dmitry, Sherman, Elena
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Confidence-aware Self-Semantic Distillation on Knowledge Graph Embedding
by: Liu, Yichen, et al.
Published: (2022)

Uncertainty Distillation: Teaching Language Models to Express Semantic Confidence
by: Hager, Sophia, et al.
Published: (2025)

Confidence Estimation for Error Detection in Text-to-SQL Systems
by: Somov, Oleg, et al.
Published: (2025)

Knowledge Distillation with Training Wheels
by: Liu, Guanlin, et al.
Published: (2025)

Sinkhorn Distance Minimization for Knowledge Distillation
by: Cui, Xiao, et al.
Published: (2024)

Detecting Conceptual Abstraction in LLMs
by: Regneri, Michaela, et al.
Published: (2024)

LoCa: Logit Calibration for Knowledge Distillation
by: Yang, Runming, et al.
Published: (2024)

Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness
by: Wei, Rongzhe, et al.
Published: (2025)

Knowledge Distillation and Dataset Distillation of Large Language Models: Emerging Trends, Challenges, and Future Directions
by: Fang, Luyang, et al.
Published: (2025)

Efficient Knowledge Injection in LLMs via Self-Distillation
by: Kujanpää, Kalle, et al.
Published: (2024)

In Good GRACEs: Principled Teacher Selection for Knowledge Distillation
by: Panigrahi, Abhishek, et al.
Published: (2025)

Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning
by: Li, Chengye, et al.
Published: (2025)

A Survey on Symbolic Knowledge Distillation of Large Language Models
by: Acharya, Kamal, et al.
Published: (2024)

Relational Knowledge Distillation Using Fine-tuned Function Vectors
by: Kang, Andrea, et al.
Published: (2026)

Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation
by: Ren, Yuxin, et al.
Published: (2023)

Reinforcement Learning-based Knowledge Distillation with LLM-as-a-Judge
by: Shen, Yiyang, et al.
Published: (2026)

Language Model Knowledge Distillation for Efficient Question Answering in Spanish
by: Bazaga, Adrián, et al.
Published: (2023)

Short Data, Long Context: Distilling Positional Knowledge in Transformers
by: Huber, Patrick, et al.
Published: (2026)

X-Token: Projection-Guided Cross-Tokenizer Knowledge Distillation
by: Sreenivas, Sharath Turuvekere, et al.
Published: (2026)

Show Your Work with Confidence: Confidence Bands for Tuning Curves
by: Lourie, Nicholas, et al.
Published: (2023)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models
by: Yang, Junjie, et al.
Published: (2025)

Knowledge Distillation from Large Language Models for Household Energy Modeling
by: Takrouri, Mohannad, et al.
Published: (2025)

Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation
by: Nguyen, Hieu, et al.
Published: (2025)

Implicit Word Reordering with Knowledge Distillation for Cross-Lingual Dependency Parsing
by: Li, Zhuoran, et al.
Published: (2025)

HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models
by: Lee, Seanie, et al.
Published: (2024)

MLKD-BERT: Multi-level Knowledge Distillation for Pre-trained Language Models
by: Zhang, Ying, et al.
Published: (2024)

Multi-modal Anchor Gated Transformer with Knowledge Distillation for Emotion Recognition in Conversation
by: Li, Jie, et al.
Published: (2025)

CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
by: Bae, Seyun, et al.
Published: (2026)

Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning
by: Mao, Zhenjiang, et al.
Published: (2026)

Structural Abstraction as an Inductive Bias for Non-Stationary Language Model Training
by: Rahmati, Elnaz, et al.
Published: (2026)

Multicalibration for Confidence Scoring in LLMs
by: Detommaso, Gianluca, et al.
Published: (2024)

Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels
by: Pangakis, Nicholas, et al.
Published: (2024)

Exploring the Limits of Model Compression in LLMs: A Knowledge Distillation Study on QA Tasks
by: Datta, Joyeeta, et al.
Published: (2025)

Delta Knowledge Distillation for Large Language Models
by: Cao, Yihan, et al.
Published: (2025)

Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction
by: Opiełka, Gustaw, et al.
Published: (2025)

Learning While Staying Curious: Entropy-Preserving Supervised Fine-Tuning via Adaptive Self-Distillation for Large Reasoning Models
by: Wang, Hao, et al.
Published: (2026)

Can We Use Probing to Better Understand Fine-tuning and Knowledge Distillation of the BERT NLU?
by: Hościłowicz, Jakub, et al.
Published: (2023)

Confidence in the Reasoning of Large Language Models
by: Pawitan, Yudi, et al.
Published: (2024)

Are LLM Decisions Faithful to Verbal Confidence?
by: Wang, Jiawei, et al.
Published: (2026)