Copertina

Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gao, Xiangxiang, Xie, Weisheng, Xiang, Yiwei, Ji, Feng
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
di: Li, Yuhui, et al.
Pubblicazione: (2024)

FlashDecoding++: Faster Large Language Model Inference on GPUs
di: Hong, Ke, et al.
Pubblicazione: (2023)

Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decoding
di: Huang, Jianuo, et al.
Pubblicazione: (2026)

ReFusion: A Diffusion Large Language Model with Parallel Autoregressive Decoding
di: Li, Jia-Nan, et al.
Pubblicazione: (2025)

Cascade Speculative Drafting for Even Faster LLM Inference
di: Chen, Ziyi, et al.
Pubblicazione: (2023)

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
di: Zhao, Weilin, et al.
Pubblicazione: (2024)

AR-MAP: Are Autoregressive Large Language Models Implicit Teachers for Diffusion Large Language Models?
di: Lin, Liang, et al.
Pubblicazione: (2026)

PEARL: Parallel Speculative Decoding with Adaptive Draft Length
di: Liu, Tianyu, et al.
Pubblicazione: (2024)

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
di: Wen, Zhuofan, et al.
Pubblicazione: (2024)

Adaptive Draft-Verification for Efficient Large Language Model Decoding
di: Liu, Xukun, et al.
Pubblicazione: (2024)

Exploring and Improving Drafts in Blockwise Parallel Decoding
di: Kim, Taehyeon, et al.
Pubblicazione: (2024)

Blockwise SFT for Diffusion Language Models: Reconciling Bidirectional Attention and Autoregressive Decoding
di: Sun, Bowen, et al.
Pubblicazione: (2025)

Chain of Draft: Thinking Faster by Writing Less
di: Xu, Silei, et al.
Pubblicazione: (2025)

Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
di: Shoham, Ofir Ben
Pubblicazione: (2026)

Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?
di: Li, Pengxiang, et al.
Pubblicazione: (2026)

Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
di: Zhang, Jun, et al.
Pubblicazione: (2023)

Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding
di: Yi, Hanling, et al.
Pubblicazione: (2024)

PARD-2: Target-Aligned Parallel Draft Model for Dual-Mode Speculative Decoding
di: An, Zihao, et al.
Pubblicazione: (2026)

CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit
di: Wang, Kangyu, et al.
Pubblicazione: (2025)

Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding
di: Cho, Sukmin, et al.
Pubblicazione: (2025)

Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models
di: Zou, Shun, et al.
Pubblicazione: (2026)

$\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts
di: Cemri, Mert, et al.
Pubblicazione: (2025)

MineDraft: A Framework for Batch Parallel Speculative Decoding
di: Tang, Zhenwei, et al.
Pubblicazione: (2026)

Learning to Parallel: Accelerating Diffusion Large Language Models via Learnable Parallel Decoding
di: Bao, Wenrui, et al.
Pubblicazione: (2025)

Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference
di: Timor, Nadav, et al.
Pubblicazione: (2024)

Cost-Aware Diffusion Draft Trees for Speculative Decoding
di: Zhang, Shuai, et al.
Pubblicazione: (2026)

Accelerating Speculative Decoding with Block Diffusion Draft Trees
di: Ringel, Liran, et al.
Pubblicazione: (2026)

OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure
di: Wang, Jikai, et al.
Pubblicazione: (2024)

Customizing Language Model Responses with Contrastive In-Context Learning
di: Gao, Xiang, et al.
Pubblicazione: (2024)

Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding
di: Guo, Gabe, et al.
Pubblicazione: (2025)

Plato: Plan to Efficiently Decode for Large Language Model Inference
di: Jin, Shuowei, et al.
Pubblicazione: (2024)

Self Speculative Decoding for Diffusion Large Language Models
di: Gao, Yifeng, et al.
Pubblicazione: (2025)

Enhancing Chemical Reaction and Retrosynthesis Prediction with Large Language Model and Dual-task Learning
di: Lin, Xuan, et al.
Pubblicazione: (2025)

Enhancing Molecular Property Prediction with Knowledge from Large Language Models
di: Zhou, Peng, et al.
Pubblicazione: (2025)

Enhancing Contextual Understanding in Large Language Models through Contrastive Decoding
di: Zhao, Zheng, et al.
Pubblicazione: (2024)

Speculative Decoding for Multi-Sample Inference
di: Li, Yiwei, et al.
Pubblicazione: (2025)

RAPID: Retrieval-Augmented Parallel Inference Drafting for Text-Based Video Event Retrieval
di: Nguyen, Long, et al.
Pubblicazione: (2025)

Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
di: Wu, Pengfei, et al.
Pubblicazione: (2024)

AuroraEdge-V-2B: A Faster And Stronger Edge Visual Large Language Model
di: Chen, Xiang
Pubblicazione: (2026)

From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models
di: Welleck, Sean, et al.
Pubblicazione: (2024)