Cover Image

Intermediate Distillation: Data-Efficient Distillation from Black-Box LLMs for Information Retrieval

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Li, Zizhong, Zhang, Haopeng, Zhang, Jiawei
Format:	Preprint
Published:	2024
Subjects:	Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Unveiling the Magic: Investigating Attention Distillation in Retrieval-augmented Generation
by: Li, Zizhong, et al.
Published: (2024)

MACA: A Framework for Distilling Trustworthy LLMs into Efficient Retrievers
by: Gudipudi, Satya Swaroop, et al.
Published: (2026)

Distillation for Multilingual Information Retrieval
by: Yang, Eugene, et al.
Published: (2024)

InRanker: Distilled Rankers for Zero-shot Information Retrieval
by: Laitz, Thiago, et al.
Published: (2024)

FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Models
by: Chen, Zhuo, et al.
Published: (2025)

Distillation Enhanced Generative Retrieval
by: Li, Yongqi, et al.
Published: (2024)

Retrieval and Distill: A Temporal Data Shift-Free Paradigm for Online Recommendation System
by: Zheng, Lei, et al.
Published: (2024)

PairDistill: Pairwise Relevance Distillation for Dense Retrieval
by: Huang, Chao-Wei, et al.
Published: (2024)

Interact-RAG: Reason and Interact with the Corpus, Beyond Black-Box Retrieval
by: Hui, Yulong, et al.
Published: (2025)

Teaching Dense Retrieval Models to Specialize with Listwise Distillation and LLM Data Augmentation
by: Tamber, Manveer Singh, et al.
Published: (2025)

Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillation
by: Yang, Eugene, et al.
Published: (2024)

MLPs are Efficient Distilled Generative Recommenders
by: Guo, Zitian, et al.
Published: (2026)

Retrieval-Feedback-Driven Distillation and Preference Alignment for Efficient LLM-based Query Expansion
by: Li, Minghan, et al.
Published: (2026)

The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems
by: Song, Hongru, et al.
Published: (2025)

Preference-Consistent Knowledge Distillation for Recommender System
by: Zhu, Zhangchi, et al.
Published: (2023)

DiSCo: LLM Knowledge Distillation for Efficient Sparse Retrieval in Conversational Search
by: Lupart, Simon, et al.
Published: (2024)

Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance Margins
by: Gienapp, Lukas, et al.
Published: (2024)

Beyond Hard Negatives: The Importance of Score Distribution in Knowledge Distillation for Dense Retrieval
by: Jang, Youngjoon, et al.
Published: (2026)

DELRec: Distilling Sequential Pattern to Enhance LLMs-based Sequential Recommendation
by: Zhang, Haoyi, et al.
Published: (2024)

Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval with Cross-Encoder Listwise Distillation and Synthetic Data
by: Tamber, Manveer Singh, et al.
Published: (2025)

From Reasoning LLMs to BERT: A Two-Stage Distillation Framework for Search Relevance
by: Xia, Runze, et al.
Published: (2025)

Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPA
by: Chen, Teng, et al.
Published: (2026)

Feature Interaction Fusion Self-Distillation Network For CTR Prediction
by: Sang, Lei, et al.
Published: (2024)

BubbleRAG: Evidence-Driven Retrieval-Augmented Generation for Black-Box Knowledge Graphs
by: Pan, Duyi, et al.
Published: (2026)

Distillation Matters: Empowering Sequential Recommenders to Match the Performance of Large Language Model
by: Cui, Yu, et al.
Published: (2024)

Harnessing Pairwise Ranking Prompting Through Sample-Efficient Ranking Distillation
by: Wu, Junru, et al.
Published: (2025)

Causal Distillation for Alleviating Performance Heterogeneity in Recommender Systems
by: Zhang, Shengyu, et al.
Published: (2024)

Document Optimization for Black-Box Retrieval via Reinforcement Learning
by: Uzan, Omri, et al.
Published: (2026)

Contextual Distillation Model for Diversified Recommendation
by: Li, Fan, et al.
Published: (2024)

A Cognitive Distribution and Behavior-Consistent Framework for Black-Box Attacks on Recommender Systems
by: Zhang, Hongyue, et al.
Published: (2026)

FIT-RAG: Black-Box RAG with Factual Information and Token Reduction
by: Mao, Yuren, et al.
Published: (2024)

DIET: Learning to Distill Dataset Continually for Recommender Systems
by: Zhang, Jiaqing, et al.
Published: (2026)

Constructing a Question-Answering Simulator through the Distillation of LLMs
by: Liu, Haipeng, et al.
Published: (2025)

Embedding the Teacher: Distilling vLLM Preferences for Scalable Image Retrieval
by: He, Eric, et al.
Published: (2025)

Continual Collaborative Distillation for Recommender System
by: Lee, Gyuseok, et al.
Published: (2024)

Disentangling Locality and Entropy in Ranking Distillation
by: Parry, Andrew, et al.
Published: (2025)

Knowledge Distillation Approaches for Accurate and Efficient Recommender System
by: Kang, SeongKu
Published: (2024)

Retrieval-Augmented Generation by Evidence Retroactivity in LLMs
by: Xiao, Liang, et al.
Published: (2025)

Context-Efficient Retrieval with Factual Decomposition
by: Li, Yanhong, et al.
Published: (2025)

Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation
by: Cheng, Kai, et al.
Published: (2026)