Cover Image

Flipping Knowledge Distillation: Leveraging Small Models' Expertise to Enhance LLMs in Text Matching

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Mingzhe, Xiang, Jing, Zhang, Qishen, Wan, Kaiyang, Chen, Xiuying
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Multi-Intent Attribute-Aware Text Matching in Searching
by: Li, Mingzhe, et al.
Published: (2024)

Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports
by: Zhu, Qingqing, et al.
Published: (2024)

Flexible and Adaptable Summarization via Expertise Separation
by: Chen, Xiuying, et al.
Published: (2024)

A Cognitive Writing Perspective for Constrained Long-Form Text Generation
by: Wan, Kaiyang, et al.
Published: (2025)

Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generation
by: Li, Mingzhe, et al.
Published: (2022)

Injecting Domain-Specific Knowledge into Large Language Models: A Comprehensive Survey
by: Song, Zirui, et al.
Published: (2025)

Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models
by: Yang, Yuncheng, et al.
Published: (2024)

Self-Enhanced Reasoning Training: Activating Latent Reasoning in Small Models for Enhanced Reasoning Distillation
by: Zhang, Yong, et al.
Published: (2025)

Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMs
by: Gao, Lang, et al.
Published: (2025)

$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models
by: Liao, Huanxuan, et al.
Published: (2024)

Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Models
by: Rao, Jun, et al.
Published: (2024)

Enhancing Knowledge Distillation for LLMs with Response-Priming Prompting
by: Goyal, Vijay, et al.
Published: (2024)

Leveraging Large Language Models for Enhanced NLP Task Performance through Knowledge Distillation and Optimized Training Strategies
by: Huang, Yining, et al.
Published: (2024)

AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications
by: Mu, Honglin, et al.
Published: (2025)

Write Summary Step-by-Step: A Pilot Study of Stepwise Summarization
by: Chen, Xiuying, et al.
Published: (2024)

Hierarchical Knowledge Distillation on Text Graph for Data-limited Attribute Inference
by: Li, Quan, et al.
Published: (2024)

Evaluating, Understanding, and Improving Constrained Text Generation for Large Language Models
by: Chen, Xiang, et al.
Published: (2023)

KnowledgeNavigator: Leveraging Large Language Models for Enhanced Reasoning over Knowledge Graph
by: Guo, Tiezheng, et al.
Published: (2023)

Permute-and-Flip: An optimally stable and watermarkable decoder for LLMs
by: Zhao, Xuandong, et al.
Published: (2024)

Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs
by: Jin, Ruihan, et al.
Published: (2026)

Leveraging Online Data to Enhance Medical Knowledge in a Small Persian Language Model
by: Ghassabi, Mehrdad, et al.
Published: (2025)

LLMs for Doctors: Leveraging Medical LLMs to Assist Doctors, Not Replace Them
by: Xie, Wenya, et al.
Published: (2024)

When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection
by: Gao, Lang, et al.
Published: (2025)

MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities
by: Chen, Jingxue, et al.
Published: (2025)

Advantage-Guided Distillation for Preference Alignment in Small Language Models
by: Gao, Shiping, et al.
Published: (2025)

Efficient Knowledge Distillation: Empowering Small Language Models with Teacher Model Insights
by: Ballout, Mohamad, et al.
Published: (2024)

Fact Finder -- Enhancing Domain Expertise of Large Language Models by Incorporating Knowledge Graphs
by: Steinigen, Daniel, et al.
Published: (2024)

Improving Long Text Understanding with Knowledge Distilled from Summarization Model
by: Liu, Yan, et al.
Published: (2024)

Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval
by: Yoo, HaeJun, et al.
Published: (2026)

Knowledge Distillation of Black-Box Large Language Models
by: Chen, Hongzhan, et al.
Published: (2024)

BAGEL: Benchmarking Animal Knowledge Expertise in Language Models
by: Shen, Jiacheng, et al.
Published: (2026)

SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation
by: Pham, Quang P. M., et al.
Published: (2025)

Distilling Text Style Transfer With Self-Explanation From LLMs
by: Zhang, Chiyu, et al.
Published: (2024)

Facilitating Pornographic Text Detection for Open-Domain Dialogue Systems via Knowledge Distillation of Large Language Models
by: Qiu, Huachuan, et al.
Published: (2024)

Facilitating NSFW Text Detection in Open-Domain Dialogue Systems via Knowledge Distillation
by: Qiu, Huachuan, et al.
Published: (2023)

On the Risk of Evidence Pollution for Malicious Social Text Detection in the Era of LLMs
by: Wan, Herun, et al.
Published: (2024)

Semantic Anchors in In-Context Learning: Why Small LLMs Cannot Flip Their Labels
by: Kumar, Anantha Padmanaban Krishna
Published: (2025)

Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
by: Xing, Wang, et al.
Published: (2026)

Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping
by: Chen, Yijie, et al.
Published: (2025)

Dual-Space Knowledge Distillation with Key-Query Matching for Large Language Models with Vocabulary Mismatch
by: Tsiapali, Stella Eva, et al.
Published: (2026)