Cover Image

Beat the long tail: Distribution-Aware Speculative Decoding for RL Training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Shao, Zelei, Srivatsa, Vikranth, Srivastava, Sanjana, Wu, Qingyang, Ariyak, Alpay, Wu, Xiaoxia, Patel, Ameen, Wang, Jue, Liang, Percy, Dao, Tri, Zhang, Ce, Zhang, Yiying, Athiwaratkun, Ben, Xu, Chenfeng, Wang, Junxiong
Format:	Preprint
Published:	2025
Subjects:	Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When RL Meets Adaptive Speculative Training: A Unified Training-Serving System
by: Wang, Junxiong, et al.
Published: (2026)

Cognify: Supercharging Gen-AI Workflows With Hierarchical Autotuning
by: He, Zijian, et al.
Published: (2025)

InferCept: Efficient Intercept Support for Augmented Large Language Model Inference
by: Abhyankar, Reyna, et al.
Published: (2024)

Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining
by: Oncescu, Costin-Andrei, et al.
Published: (2025)

Squeeze Evolve: Unified Multi-Model Orchestration for Verifier-Free Evolution
by: Maheswaran, Monishwaran, et al.
Published: (2026)

Nitsum: Serving Tiered LLM Requests with Adaptive Tensor Parallelism
by: Srivatsa, Vikranth, et al.
Published: (2026)

Preble: Efficient Distributed Prompt Scheduling for LLM Serving
by: Srivatsa, Vikranth, et al.
Published: (2024)

SAW-INT4: System-Aware 4-Bit KV-Cache Quantization for Real-World LLM Serving
by: Jia, Jinda, et al.
Published: (2026)

Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost
by: Xia, Haojun, et al.
Published: (2025)

TClone: Low-Latency Forking of Live GUI Environments for Computer-Use Agents
by: Huang, Yutong, et al.
Published: (2026)

Introspective Diffusion Language Models
by: Yu, Yifan, et al.
Published: (2026)

$V_1$: Unifying Generation and Self-Verification for Parallel Reasoners
by: Singh, Harman, et al.
Published: (2026)

Speculative Speculative Decoding
by: Kumar, Tanishq, et al.
Published: (2026)

Search Your Block Floating Point Scales!
by: Gupta, Tanmaey, et al.
Published: (2026)

Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time
by: Zhang, Zhenyu, et al.
Published: (2025)

Mixture-of-Agents Enhances Large Language Model Capabilities
by: Wang, Junlin, et al.
Published: (2024)

Data Diversification Methods In Alignment Enhance Math Performance In LLMs
by: Dokmeci, Berkan, et al.
Published: (2025)

Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
by: He, Linda, et al.
Published: (2025)

Scaling Speculative Decoding with Lookahead Reasoning
by: Fu, Yichao, et al.
Published: (2025)

Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling Methods
by: Wang, Junlin, et al.
Published: (2025)

CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention
by: Zhou, Zhongzhu, et al.
Published: (2026)

Staircase Streaming for Low-Latency Multi-Agent Inference
by: Wang, Junlin, et al.
Published: (2025)

CDLM: Consistency Diffusion Language Models For Faster Sampling
by: Kim, Minseo, et al.
Published: (2025)

The Mamba in the Llama: Distilling and Accelerating Hybrid Models
by: Wang, Junxiong, et al.
Published: (2024)

Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
by: Zhang, Muru, et al.
Published: (2025)

When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework
by: Xu, Zhen, et al.
Published: (2025)

On tail probability of the covariance matrix in Eldan's stochastic localization
by: Guan, Qingyang
Published: (2025)

A Multi-Model Adaptation of Speculative Decoding for Classification
by: Roy, Somnath, et al.
Published: (2025)

M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models
by: Wang, Junxiong, et al.
Published: (2025)

Improving Model Alignment Through Collective Intelligence of Open-Source LLMS
by: Wang, Junlin, et al.
Published: (2025)

Mamba: Linear-Time Sequence Modeling with Selective State Spaces
by: Gu, Albert, et al.
Published: (2023)

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
by: Dao, Tri, et al.
Published: (2024)

Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimation
by: Liu, Jingyu, et al.
Published: (2025)

Achieving Semantic Consistency: Contextualized Word Representations for Political Text Analysis
by: Zhang, Ruiyu, et al.
Published: (2024)

OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization
by: Zhou, Zhongzhu, et al.
Published: (2026)

Democratizing AI Development: Local LLM Deployment for India's Developer Ecosystem in the Era of Tokenized APIs
by: Udandarao, Vikranth, et al.
Published: (2025)

Movie Revenue Prediction using Machine Learning Models
by: Udandarao, Vikranth, et al.
Published: (2024)

Fast Collaborative Inference via Distributed Speculative Decoding
by: Zheng, Ce, et al.
Published: (2025)

Communication-Efficient Collaborative LLM Inference via Distributed Speculative Decoding
by: Zheng, Ce, et al.
Published: (2025)

Lactate and Lactylation in Immune Cell Function and Autoimmune Diseases: Mechanisms and Therapeutic Potential
by: Yiying Yang, et al.
Published: (2025)