Cover Image

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Jennifer, Myrzakhan, Aidar, Luo, Yaxin, Khan, Hassaan Muhammad, Bsharat, Sondos Mahmoud, Shen, Zhiqiang
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Open-LLM-Leaderboard: From Multi-choice to Open-style Questions for LLMs Evaluation, Benchmark, and Arena
by: Myrzakhan, Aidar, et al.
Published: (2024)

Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4
by: Bsharat, Sondos Mahmoud, et al.
Published: (2023)

Prompting Test-Time Scaling Is A Strong LLM Reasoning Data Augmentation
by: Bsharat, Sondos Mahmoud, et al.
Published: (2025)

Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmark
by: Bsharat, Sondos Mahmoud, et al.
Published: (2025)

Sink-Aware Pruning for Diffusion Language Models
by: Myrzakhan, Aidar, et al.
Published: (2026)

Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation
by: Nguyen, Hieu, et al.
Published: (2025)

Dataset Distillation via Committee Voting
by: Cui, Jiacheng, et al.
Published: (2025)

FADRM: Fast and Accurate Data Residual Matching for Dataset Distillation
by: Cui, Jiacheng, et al.
Published: (2025)

Efficient and Stealthy Jailbreak Attacks via Adversarial Prompt Distillation from LLMs to SLMs
by: Li, Xiang, et al.
Published: (2025)

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
by: Padarha, Shreyansh
Published: (2025)

To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation
by: Waheed, Abdul, et al.
Published: (2024)

Toward Robust GraphRAG: Mitigating Retrieval Drift and Hallucination from Imperfect Knowledge Graphs
by: Ma, Yizhuo, et al.
Published: (2026)

Language-Pretraining-Induced Bias: A Strong Foundation for General Vision Tasks
by: Luo, Yaxin, et al.
Published: (2026)

FerKD: Surgical Label Adaptation for Efficient Distillation
by: Shen, Zhiqiang
Published: (2023)

Enhancing Knowledge Distillation for LLMs with Response-Priming Prompting
by: Goyal, Vijay, et al.
Published: (2024)

Distill to Delete: Unlearning in Graph Networks with Knowledge Distillation
by: Sinha, Yash, et al.
Published: (2023)

Listen Like a Teacher: Mitigating Whisper Hallucinations using Adaptive Layer Attention and Knowledge Distillation
by: Tripathi, Kumud, et al.
Published: (2025)

Adaptive Explicit Knowledge Transfer for Knowledge Distillation
by: Park, Hyungkeun, et al.
Published: (2024)

Distilling Text Style Transfer With Self-Explanation From LLMs
by: Zhang, Chiyu, et al.
Published: (2024)

FBI-LLM: Scaling Up Fully Binarized LLMs from Scratch via Autoregressive Distillation
by: Ma, Liqun, et al.
Published: (2024)

Interactive DualChecker for Mitigating Hallucinations in Distilling Large Language Models
by: Wang, Meiyun, et al.
Published: (2024)

SHARP: Unlocking Interactive Hallucination via Stance Transfer in Role-Playing LLMs
by: Kong, Chuyi, et al.
Published: (2024)

From Masks to Pixels and Meaning: A New Taxonomy, Benchmark, and Metrics for VLM Image Tampering
by: Shang, Xinyi, et al.
Published: (2026)

DistilLock: Safeguarding LLMs from Unauthorized Knowledge Distillation on the Edge
by: Mohanty, Asmita, et al.
Published: (2025)

MorphDistill: Distilling Unified Morphological Knowledge from Pathology Foundation Models for Colorectal Cancer Survival Prediction
by: Khan, Hikmat, et al.
Published: (2026)

Mitigating Hallucination on Hallucination in RAG via Ensemble Voting
by: Xie, Zequn, et al.
Published: (2026)

HYDRA-FL: Hybrid Knowledge Distillation for Robust and Accurate Federated Learning
by: Khan, Momin Ahmad, et al.
Published: (2024)

Utility Boundary of Dataset Distillation: Scaling and Configuration-Coverage Laws
by: Luo, Zhengquan, et al.
Published: (2025)

Identify, Isolate, and Purge: Mitigating Hallucinations in LVLMs via Self-Evolving Distillation
by: Li, Wenhao, et al.
Published: (2025)

Flow Augmentation and Knowledge Distillation for Lightweight Face Presentation Attack Detection
by: Jabbar, Muhammad Shahid, et al.
Published: (2026)

Dataset Distillation via Curriculum Data Synthesis in Large Data Era
by: Yin, Zeyuan, et al.
Published: (2023)

Graph Knowledge Distillation to Mixture of Experts
by: Rumiantsev, Pavel, et al.
Published: (2024)

FSKD: Monocular Forest Structure Inference via LiDAR-to-RGBI Knowledge Distillation
by: Khan, Taimur, et al.
Published: (2026)

On Explaining Knowledge Distillation: Measuring and Visualising the Knowledge Transfer Process
by: Adhane, Gereziher, et al.
Published: (2024)

A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation
by: Wu, Lirong, et al.
Published: (2024)

HalluClean: A Unified Framework to Combat Hallucinations in LLMs
by: Zhao, Yaxin, et al.
Published: (2025)

Knowledge Distillation with Adapted Weight
by: Wu, Sirong, et al.
Published: (2025)

KARL: Mitigating Hallucinations in LLMs via Knowledge-Boundary-Aware Reinforcement Learning
by: Gao, Cheng, et al.
Published: (2026)

Air Traffic Flow Prediction with Spatiotemporal Knowledge Distillation Network
by: Zhiqi Shen, et al.
Published: (2024)

Reinforcement Learning-based Knowledge Distillation with LLM-as-a-Judge
by: Shen, Yiyang, et al.
Published: (2026)