Cover Image

SD$^2$: Self-Distilled Sparse Drafters

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Lasby, Mike, Sinnadurai, Nish, Manohararajah, Valavan, Lie, Sean, Ioannou, Yani, Thangarasa, Vithursan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression
by: Lasby, Mike, et al.
Published: (2025)

Self-Data Distillation for Recovering Quality in Pruned Large Language Models
by: Thangarasa, Vithursan, et al.
Published: (2024)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
by: Saji, Alan, et al.
Published: (2025)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

TapOut: A Bandit-Based Approach to Dynamic Speculative Decoding
by: Sridhar, Aditya, et al.
Published: (2025)

Language Model Circuits Are Sparse in the Neuron Basis
by: Arora, Aryaman, et al.
Published: (2026)

Distilling Self-Consistency into Verbal Confidence: A Pre-Registered Negative Result and Post-Hoc Rescue on Gemma 3 4B
by: Cacioli, Jon-Paul
Published: (2026)

Evaluating Explainable AI Attribution Methods in Neural Machine Translation via Attention-Guided Knowledge Distillation
by: Nourbakhsh, Aria, et al.
Published: (2026)

SimLM: Can Language Models Infer Parameters of Physical Systems?
by: Memery, Sean, et al.
Published: (2023)

Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language
by: Kinas, Remigiusz, et al.
Published: (2026)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
by: Oketunji, Abiodun Finbarrs
Published: (2023)

Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
by: Siegel, Noah Y., et al.
Published: (2025)

Component-Aware Self-Speculative Decoding in Hybrid Language Models
by: Borobia, Hector, et al.
Published: (2026)

The Sufficiency-Conciseness Trade-off in LLM Self-Explanation from an Information Bottleneck Perspective
by: Zahedzadeh, Ali, et al.
Published: (2026)

Low-Resource Court Judgment Summarization for Common Law Systems
by: Liu, Shuaiqi, et al.
Published: (2024)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
by: Ashuach, Tomer, et al.
Published: (2025)

Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thought
by: Tan, Xiaoyu, et al.
Published: (2024)

Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs
by: Anshumann, et al.
Published: (2025)

Align-to-Distill: Trainable Attention Alignment for Knowledge Distillation in Neural Machine Translation
by: Jin, Heegon, et al.
Published: (2024)

ROZA Graphs: Self-Improving Near-Deterministic RAG through Evidence-Centric Feedback
by: Penaroza, Matthew
Published: (2026)

Sparse Regression for Machine Translation
by: Biçici, Ergun
Published: (2024)

TWIZ-v2: The Wizard of Multimodal Conversational-Stimulus
by: Ferreira, Rafael, et al.
Published: (2023)

Fine-Tuned Large Language Models for Logical Translation: Reducing Hallucinations with Lang2Logic
by: Pan, Muyu, et al.
Published: (2025)

Robustness of Large Language Models to Perturbations in Text
by: Singh, Ayush, et al.
Published: (2024)

FIN-bench-v2: A Unified and Robust Benchmark Suite for Evaluating Finnish Large Language Models
by: Kytöniemi, Joona, et al.
Published: (2025)

NinjaLLM: Fast, Scalable and Cost-effective RAG using Amazon SageMaker and AWS Trainium and Inferentia2
by: Xue, Tengfei, et al.
Published: (2024)

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations
by: Salehmohamed, Shoaib Sadiq, et al.
Published: (2026)

From Guessing to Asking: An Approach to Resolving the Persona Knowledge Gap in LLMs during Multi-Turn Conversations
by: Baskar, Sarvesh, et al.
Published: (2025)

Neural Machine Translation for Malayalam Paraphrase Generation
by: Varghese, Christeena, et al.
Published: (2024)

Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbot
by: Lassche, Herman, et al.
Published: (2024)

Predictive Simultaneous Interpretation: Harnessing Large Language Models for Democratizing Real-Time Multilingual Communication
by: Iida, Kurando, et al.
Published: (2024)

Exploring RWKV for Sentence Embeddings: Layer-wise Analysis and Baseline Comparison for Semantic Similarity
by: Pan, Xinghan
Published: (2025)

UA-Legal-Bench: A Benchmark for Evaluating Large Language Models on Ukrainian Legal Reasoning
by: Ovcharov, Volodymyr
Published: (2026)

RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
by: Goldstein, Daniel, et al.
Published: (2025)

MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models
by: Ganesan, Mugilan, et al.
Published: (2025)

Large Language Model (LLM) Bias Index -- LLMBI
by: Oketunji, Abiodun Finbarrs, et al.
Published: (2023)

Grammatically-Guided Sparse Attention for Efficient and Interpretable Transformers
by: Pratyush, Spandan
Published: (2026)

Turning the TIDE: Cross-Architecture Distillation for Diffusion Large Language Models
by: Zhang, Gongbo, et al.
Published: (2026)

Harnessing Negative Signals: Reinforcement Distillation from Teacher Data for LLM Reasoning
by: Xu, Shuyao, et al.
Published: (2025)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
by: Fadli, Samih
Published: (2025)