Copertina

Distillation Contrastive Decoding: Improving LLMs Reasoning with Contrastive Decoding and Distillation

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Phan, Phuc, Tran, Hieu, Phan, Long
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

DistillSpec: Improving Speculative Decoding via Knowledge Distillation
di: Zhou, Yongchao, et al.
Pubblicazione: (2023)

DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs
di: Ko, Jongwoo, et al.
Pubblicazione: (2025)

Evolutionary Contrastive Distillation for Language Model Alignment
di: Katz-Samuels, Julian, et al.
Pubblicazione: (2024)

DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
di: Chuang, Yung-Sung, et al.
Pubblicazione: (2023)

AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders
di: Hu, Yuezhou, et al.
Pubblicazione: (2025)

Contrastive Decoding for Synthetic Data Generation in Low-Resource Language Modeling
di: Ulm, Jannek, et al.
Pubblicazione: (2025)

Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation
di: Zhang, Hongxiang, et al.
Pubblicazione: (2025)

Reasoning Distillation and Structural Alignment for Improved Code Generation
di: Jalilifard, Amir, et al.
Pubblicazione: (2025)

CONSCENDI: A Contrastive and Scenario-Guided Distillation Approach to Guardrail Models for Virtual Assistants
di: Sun, Albert Yu, et al.
Pubblicazione: (2023)

SUPERNOVA: Eliciting General Reasoning in LLMs with Reinforcement Learning on Natural Instructions
di: Suvarna, Ashima, et al.
Pubblicazione: (2026)

RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
di: Goldstein, Daniel, et al.
Pubblicazione: (2025)

Efficiently Distilling LLMs for Edge Applications
di: Kundu, Achintya, et al.
Pubblicazione: (2024)

KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
di: Xu, Hongling, et al.
Pubblicazione: (2025)

HiSpec: Hierarchical Speculative Decoding for LLMs
di: Kumar, Avinash, et al.
Pubblicazione: (2025)

BOND: Aligning LLMs with Best-of-N Distillation
di: Sessa, Pier Giuseppe, et al.
Pubblicazione: (2024)

Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation
di: Tian, Yijun, et al.
Pubblicazione: (2024)

VLCD: Vision-Language Contrastive Distillation for Accurate and Efficient Automatic Placenta Analysis
di: Mehta, Manas, et al.
Pubblicazione: (2025)

Exploring and Improving Drafts in Blockwise Parallel Decoding
di: Kim, Taehyeon, et al.
Pubblicazione: (2024)

Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
di: Kim, Minsang, et al.
Pubblicazione: (2026)

Draft-Conditioned Constrained Decoding for Structured Generation in LLMs
di: Reddy, Avinash, et al.
Pubblicazione: (2026)

Evolutionary Guided Decoding: Iterative Value Refinement for LLMs
di: Liu, Zhenhua, et al.
Pubblicazione: (2025)

Energy-Aware LLMs: A step towards sustainable AI for downstream applications
di: Tran, Nguyen Phuc, et al.
Pubblicazione: (2025)

Distilling LLMs' Decomposition Abilities into Compact Language Models
di: Tarasov, Denis, et al.
Pubblicazione: (2024)

RelayLLM: Efficient Reasoning via Collaborative Decoding
di: Huang, Chengsong, et al.
Pubblicazione: (2026)

Scalable LLM Reasoning Acceleration with Low-rank Distillation
di: Dong, Harry, et al.
Pubblicazione: (2025)

Structural Rationale Distillation via Reasoning Space Compression
di: Yang, Jialin, et al.
Pubblicazione: (2026)

Agentic-R1: Distilled Dual-Strategy Reasoning
di: Du, Weihua, et al.
Pubblicazione: (2025)

DRAG: Distilling RAG for SLMs from LLMs to Transfer Knowledge and Mitigate Hallucination via Evidence and Graph-based Distillation
di: Chen, Jennifer, et al.
Pubblicazione: (2025)

Nudging: Inference-time Alignment of LLMs via Guided Decoding
di: Fei, Yu, et al.
Pubblicazione: (2024)

What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding
di: Li, Ming, et al.
Pubblicazione: (2025)

Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion
di: Tan, Zhen, et al.
Pubblicazione: (2026)

SAGE-32B: Agentic Reasoning via Iterative Distillation
di: Jha, Basab, et al.
Pubblicazione: (2026)

Enhancing Reasoning Capabilities in SLMs with Reward Guided Dataset Distillation
di: Padarha, Shreyansh
Pubblicazione: (2025)

Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -
di: Fieback, Laura, et al.
Pubblicazione: (2025)

Integrative Decoding: Improve Factuality via Implicit Self-consistency
di: Cheng, Yi, et al.
Pubblicazione: (2024)

Modality Collapse as Mismatched Decoding: Information-Theoretic Limits of Multimodal LLMs
di: Billa, Jayadev
Pubblicazione: (2026)

Accelerating Diffusion LLMs via Adaptive Parallel Decoding
di: Israel, Daniel, et al.
Pubblicazione: (2025)

Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations
di: Luo, Haozheng, et al.
Pubblicazione: (2026)

Through the Lens of Contrast: Self-Improving Visual Reasoning in VLMs
di: Pan, Zhiyu, et al.
Pubblicazione: (2026)

DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching
di: Xu, Zicheng, et al.
Pubblicazione: (2025)