Image de couverture de livre

Accelerating Production LLMs with Combined Token/Embedding Speculators

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Wertheimer, Davis, Rosenkranz, Joshua, Parnell, Thomas, Suneja, Sahil, Ranganathan, Pavithra, Ganti, Raghu, Srivatsa, Mudhakar
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

TP-Aware Dequantization
par: Hoque, Adnan, et autres
Publié: (2024)

Accelerating a Triton Fused Kernel for W4A16 Quantized Inference with SplitK work decomposition
par: Hoque, Adnan, et autres
Publié: (2024)

HadaCore: Tensor Core Accelerated Hadamard Transform Kernel
par: Agarwal, Krish, et autres
Publié: (2024)

A Multi-Model Adaptation of Speculative Decoding for Classification
par: Roy, Somnath, et autres
Publié: (2025)

LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation
par: Liu, Tianyu, et autres
Publié: (2025)

SudokuSens: Enhancing Deep Learning Robustness for IoT Sensing Applications using a Generative Approach
par: Wang, Tianshi, et autres
Publié: (2024)

Flexible and Effective Mixing of Large Language Models into a Mixture of Domain Experts
par: Lee, Rhui Dih, et autres
Publié: (2024)

Speculating LLMs' Chinese Training Data Pollution from Their Tokens
par: Zhang, Qingjie, et autres
Publié: (2025)

SDSAT: Accelerating LLM Inference through Speculative Decoding with Semantic Adaptive Tokens
par: Liu, Chengbo, et autres
Publié: (2024)

Speculate Deep and Accurate: Lossless and Training-Free Acceleration for Offloaded LLMs via Substitute Speculative Decoding
par: Wang, Pei-Shuo, et autres
Publié: (2025)

What Makes Math Word Problems Challenging for LLMs?
par: Srivatsa, KV Aditya, et autres
Publié: (2024)

Confidence, Not Perplexity: A Better Metric for the Creative Era of LLMs
par: Parupudi, V. S. Raghu
Publié: (2025)

Dynamic Speculation Lookahead Accelerates Speculative Decoding of Large Language Models
par: Mamou, Jonathan, et autres
Publié: (2024)

Mirage of Mastery: Memorization Tricks LLMs into Artificially Inflated Self-Knowledge
par: Kale, Sahil
Publié: (2025)

SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning
par: Huang, Haoyu, et autres
Publié: (2026)

Accelerate Speculative Decoding with Sparse Computation in Verification
par: Wang, Jikai, et autres
Publié: (2025)

Continuous Prompt Generation from Linear Combination of Discrete Prompt Embeddings
par: Passigan, Pascal, et autres
Publié: (2023)

TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees
par: Liu, Tianyu, et autres
Publié: (2026)

Flatter Tokens are More Valuable for Speculative Draft Model Training
par: Fan, Jiaming, et autres
Publié: (2026)

SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning
par: Ji, Yicheng, et autres
Publié: (2025)

Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs
par: Kale, Sahil
Publié: (2025)

Hierarchical Verification of Speculative Beams for Accelerating LLM Inference
par: Sen, Jaydip, et autres
Publié: (2025)

Accelerating Speculative Decoding with Block Diffusion Draft Trees
par: Ringel, Liran, et autres
Publié: (2026)

Training-free Dropout Sampling for Semantic Token Acceptance in Speculative Decoding
par: Lee, Jeongtae, et autres
Publié: (2026)

GRIFFIN: Effective Token Alignment for Faster Speculative Decoding
par: Hu, Shijing, et autres
Publié: (2025)

STS: Efficient Sparse Attention with Speculative Token Sparsity
par: Xu, Ceyu, et autres
Publié: (2026)

Frayed RoPE and Long Inputs: A Geometric Perspective
par: Wertheimer, Davis, et autres
Publié: (2026)

LLMs cannot spot math errors, even when allowed to peek into the solution
par: Srivatsa, KV Aditya, et autres
Publié: (2025)

Can LLMs Reliably Simulate Real Students' Abilities in Mathematics and Reading Comprehension?
par: Srivatsa, KV Aditya, et autres
Publié: (2025)

SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration
par: Xia, Heming, et autres
Publié: (2024)

Accelerated Test-Time Scaling with Model-Free Speculative Sampling
par: Song, Woomin, et autres
Publié: (2025)

Optimized Speculative Sampling for GPU Hardware Accelerators
par: Wagner, Dominik, et autres
Publié: (2024)

Efficient Reasoning for LLMs through Speculative Chain-of-Thought
par: Wang, Jikai, et autres
Publié: (2025)

TETRIS: Optimal Draft Token Selection for Batch Speculative Decoding
par: Wu, Zhaoxuan, et autres
Publié: (2025)

Reject Only Critical Tokens: Pivot-Aware Speculative Decoding
par: Ziashahabi, Amir, et autres
Publié: (2025)

Using Span Queries to Optimize for Cache and Attention Locality
par: Castro, Paul, et autres
Publié: (2025)

semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces
par: Ranganathan, Jwalanthi, et autres
Publié: (2025)

A Text is Worth Several Tokens: Text Embedding from LLMs Secretly Aligns Well with The Key Tokens
par: Nie, Zhijie, et autres
Publié: (2024)

SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding
par: Wang, Zhenglin, et autres
Publié: (2024)

Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism
par: Shen, Yuhao, et autres
Publié: (2026)