Imagen de Portada

Attention Drift: What Autoregressive Speculative Decoding Models Learn

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Eldenk, Doğaç, Mohapatra, Payal, Comlek, Yigitcan, Oktay, Kaan, Zhang, Hongyang, Xia, Stephen
Formato:	Preprint
Publicado:	2026
Materias:	Machine Learning Artificial Intelligence I.2.7
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

SSSD: Simply-Scalable Speculative Decoding
por: Marzollo, Michele, et al.
Publicado: (2024)

FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning
por: Zhang, Yizhou, et al.
Publicado: (2025)

SAM Decoding: Speculative Decoding via Suffix Automaton
por: Hu, Yuxuan, et al.
Publicado: (2024)

Component-Aware Self-Speculative Decoding in Hybrid Language Models
por: Borobia, Hector, et al.
Publicado: (2026)

RADLADS: Rapid Attention Distillation to Linear Attention Decoders at Scale
por: Goldstein, Daniel, et al.
Publicado: (2025)

Entropy-Based Measurement of Value Drift and Alignment Work in Large Language Models
por: Fadli, Samih
Publicado: (2025)

Evaluating the Efficacy of Hybrid Deep Learning Models in Distinguishing AI-Generated Text
por: Oketunji, Abiodun Finbarrs
Publicado: (2023)

DIVERSED: Relaxed Speculative Decoding via Dynamic Ensemble Verification
por: Wang, Ziyi, et al.
Publicado: (2026)

SpecExtend: A Drop-in Enhancement for Speculative Decoding of Long Sequences
por: Cha, Jungyoub, et al.
Publicado: (2025)

Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
por: Peng, Songping, et al.
Publicado: (2026)

UNIFERENCE: A Discrete Event Simulation Framework for Developing Distributed AI Models
por: Eldenk, Doğaç, et al.
Publicado: (2026)

Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models
por: Sun, Chendong, et al.
Publicado: (2025)

How Does Unfaithful Reasoning Emerge from Autoregressive Training? A Study of Synthetic Experiments
por: Wang, Fuxin, et al.
Publicado: (2026)

Encoder vs Decoder: Comparative Analysis of Encoder and Decoder Language Models on Multilingual NLU Tasks
por: Nielsen, Dan Saattrup, et al.
Publicado: (2024)

Meta-Learning at Scale for Large Language Models via Low-Rank Amortized Bayesian Meta-Learning
por: Zhang, Liyi, et al.
Publicado: (2025)

Training Dynamics Underlying Language Model Scaling Laws: Loss Deceleration and Zero-Sum Learning
por: Mircea, Andrei, et al.
Publicado: (2025)

CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality
por: Dumitru, Razvan-Gabriel, et al.
Publicado: (2025)

A Performance Evaluation of a Quantized Large Language Model on Various Smartphones
por: Çöplü, Tolga, et al.
Publicado: (2023)

Large Language Model (LLM) Bias Index -- LLMBI
por: Oketunji, Abiodun Finbarrs, et al.
Publicado: (2023)

How LLMs Are Persuaded: A Few Attention Heads, Rerouted
por: Sun, Xiangkun, et al.
Publicado: (2026)

Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents
por: Gao, Heyang, et al.
Publicado: (2025)

Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors
por: Zhang, Zhiwei, et al.
Publicado: (2026)

Beyond Memorization: Violating Privacy Via Inference with Large Language Models
por: Staab, Robin, et al.
Publicado: (2023)

Beyond the Black Box: A Statistical Model for LLM Reasoning and Inference
por: Dalal, Siddhartha, et al.
Publicado: (2024)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
por: Peters, Sydney, et al.
Publicado: (2025)

End-to-End Optimization of LLM-Driven Multi-Agent Search Systems via Heterogeneous-Group-Based Reinforcement Learning
por: Chen, Guanzhong, et al.
Publicado: (2025)

Quantized Side Tuning: Fast and Memory-Efficient Tuning of Quantized Large Language Models
por: Zhang, Zhengxin, et al.
Publicado: (2024)

From Fake Focus to Real Precision: Confusion-Driven Adversarial Attention Learning in Transformers
por: Liu, Yawei
Publicado: (2025)

Softmax Linear Attention: Reclaiming Global Competition
por: Xu, Mingwei, et al.
Publicado: (2026)

Forget Attention: Importance-Aware Attention Is All You Need
por: Shin, Soohyeong, et al.
Publicado: (2026)

Residual Drift Dominates Contradiction in Multi-Turn Constraint Reasoning
por: Kawada, Sebastien
Publicado: (2026)

Dodo: Dynamic Contextual Compression for Decoder-only LMs
por: Qin, Guanghui, et al.
Publicado: (2023)

Adversarial Lens: Exploiting Attention Layers to Generate Adversarial Examples for Evaluation
por: Dhole, Kaustubh
Publicado: (2025)

Assessment of Transformer-Based Encoder-Decoder Model for Human-Like Summarization
por: Nair, Sindhu, et al.
Publicado: (2024)

AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
por: Höth, Max Henning, et al.
Publicado: (2026)

RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
por: Saji, Alan, et al.
Publicado: (2025)

When Words Change the Model: Sensitivity of LLMs for Constraint Programming Modelling
por: Pellegrino, Alessio, et al.
Publicado: (2025)

Mixture of Attention Spans: Optimizing LLM Inference Efficiency with Heterogeneous Sliding-Window Lengths
por: Fu, Tianyu, et al.
Publicado: (2024)

Shattered Compositionality: Counterintuitive Learning Dynamics of Transformers for Arithmetic
por: Zhao, Xingyu, et al.
Publicado: (2026)

CoDA: Coding LM via Diffusion Adaptation
por: Chen, Haolin, et al.
Publicado: (2025)