Cover Image

Don't Throw Away Data: Better Sequence Knowledge Distillation

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Jun, Briakou, Eleftheria, Dadkhahi, Hamid, Agarwal, Rishabh, Cherry, Colin, Cohn, Trevor
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Don't Throw Away Your Pretrained Model
by: Feng, Shangbin, et al.
Published: (2025)

Leveraging Domain Knowledge at Inference Time for LLM Translation: Retrieval versus Generation
by: Li, Bryan, et al.
Published: (2025)

Translating Step-by-Step: Decomposing the Translation Process for Improved Translation Quality of Long-Form Texts
by: Briakou, Eleftheria, et al.
Published: (2024)

On the Implications of Verbose LLM Outputs: A Case Study in Translation Evaluation
by: Briakou, Eleftheria, et al.
Published: (2024)

Don't Ignore the Tail: Decoupling top-K Probabilities for Efficient Language Model Distillation
by: Dasgupta, Sayantan, et al.
Published: (2026)

Don't Throw Away Your Beams: Improving Consistency-based Uncertainties in LLMs via Beam Search
by: Fadeeva, Ekaterina, et al.
Published: (2025)

Don't Throw Away the cookies!
by: Rebecca Weaver
Published: (2024)

Overestimation in LLM Evaluation: A Controlled Large-Scale Study on Data Contamination's Impact on Machine Translation
by: Kocyigit, Muhammed Yusuf, et al.
Published: (2025)

When Better Teachers Don't Make Better Students: Revisiting Knowledge Distillation for CLIP Models in VQA
by: Tuchinda, Pume, et al.
Published: (2025)

Rethinking Cross-lingual Alignment: Balancing Transfer and Cultural Erasure in Multilingual LLMs
by: Han, HyoJung, et al.
Published: (2025)

Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation
by: Kreutzer, Julia, et al.
Published: (2025)

The Accuracy Paradox in RLHF: When Better Reward Models Don't Yield Better Language Models
by: Chen, Yanjun, et al.
Published: (2024)

Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG
by: Sun, Yiqun, et al.
Published: (2026)

Large Reasoning Models Struggle to Transfer Parametric Knowledge Across Scripts
by: Bandarkar, Lucas, et al.
Published: (2026)

EpiCaR: Knowing What You Don't Know Matters for Better Reasoning in LLMs
by: Yeom, Jewon, et al.
Published: (2026)

SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?
by: Li, Senyu, et al.
Published: (2025)

Don't Touch My Diacritics
by: Gorman, Kyle, et al.
Published: (2024)

Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration
by: Feng, Shangbin, et al.
Published: (2024)

Don't Pay Attention
by: Hammoud, Mohammad, et al.
Published: (2025)

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
by: Hans, Abhimanyu, et al.
Published: (2024)

Knowledge Localization in Mixture-of-Experts LLMs Using Cross-Lingual Inconsistency
by: Bandarkar, Lucas, et al.
Published: (2026)

Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models
by: Huang, Zemin, et al.
Published: (2025)

Think, But Don't Overthink: Reproducing Recursive Language Models
by: Wang, Daren
Published: (2026)

Don't Say No: Jailbreaking LLM by Suppressing Refusal
by: Zhou, Yukai, et al.
Published: (2024)

LoraxBench: A Multitask, Multilingual Benchmark Suite for 20 Indonesian Languages
by: Aji, Alham Fikri, et al.
Published: (2025)

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
by: Zhou, Yongchao, et al.
Published: (2023)

Don't Look Away
by: Cohen, Brianne
Published: (2023)

If You Don't Understand It, Don't Use It: Eliminating Trojans with Filters Between Layers
by: Hernandez, Adriano
Published: (2024)

Learning from others' mistakes: Finetuning machine translation models with span-level error annotations
by: Zhang, Lily H., et al.
Published: (2024)

sDPO: Don't Use Your Data All at Once
by: Kim, Dahyun, et al.
Published: (2024)

Beyond "I Don't Know": Evaluating LLM Self-Awareness in Discriminating Data and Model Uncertainty
by: Ren, Jingyi, et al.
Published: (2026)

Don't Do RAG: When Cache-Augmented Generation is All You Need for Knowledge Tasks
by: Chan, Brian J, et al.
Published: (2024)

Decomposed Prompting Does Not Fix Knowledge Gaps, But Helps Models Say "I Don't Know"
by: Madhwal, Dhruv, et al.
Published: (2026)

Do We Know What LLMs Don't Know? A Study of Consistency in Knowledge Probing
by: Zhao, Raoyuan, et al.
Published: (2025)

“Don't Throw the Baby out with the Bathwater”
by: Katherine B. Howell, et al.
Published: (2024)

Don't Command, Cultivate: An Exploratory Study of System-2 Alignment
by: Wang, Yuhang, et al.
Published: (2024)

Hatevolution: What Static Benchmarks Don't Tell Us
by: Di Bonaventura, Chiara, et al.
Published: (2025)

Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling
by: Xu, Wenda, et al.
Published: (2024)

$\textit{SKIntern}$: Internalizing Symbolic Knowledge for Distilling Better CoT Capabilities into Small Language Models
by: Liao, Huanxuan, et al.
Published: (2024)

Smaller, Weaker, Yet Better: Training LLM Reasoners via Compute-Optimal Sampling
by: Bansal, Hritik, et al.
Published: (2024)