Imagen de Portada

DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Yao, Jinwei, Chen, Kaiqi, Zhang, Kexun, You, Jiaxuan, Yuan, Binhang, Wang, Zeke, Lin, Tao
Formato:	Preprint
Publicado:	2024
Materias:	Computation and Language Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities
por: Hong, Zhaochen, et al.
Publicado: (2025)

Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decoding
por: Zhang, Kexun, et al.
Publicado: (2023)

Scaling LLM Inference with Optimized Sample Compute Allocation
por: Zhang, Kexun, et al.
Publicado: (2024)

DeFT: Mitigating Data Dependencies for Flexible Communication Scheduling in Distributed Training
por: Meng, Lin, et al.
Publicado: (2025)

LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding
por: Lin, Gang, et al.
Publicado: (2026)

AcademicEval: Live Long-Context LLM Benchmark
por: Zhang, Haozhen, et al.
Publicado: (2025)

TQA-Bench: Evaluating LLMs for Multi-Table Question Answering with Scalable Context and Symbolic Extension
por: Qiu, Zipeng, et al.
Publicado: (2024)

FlashEVA: Accelerating LLM inference via Efficient Attention
por: Kostelec, Juan Gabriel, et al.
Publicado: (2025)

TALEC: Teach Your LLM to Evaluate in Specific Domain with In-house Criteria by Criteria Division and Zero-shot Plus Few-shot
por: Zhang, Kaiqi, et al.
Publicado: (2024)

Table as Thought: Exploring Structured Thoughts in LLM Reasoning
por: Sun, Zhenjie, et al.
Publicado: (2025)

Distribution-Aligned Decoding for Efficient LLM Task Adaptation
por: Hu, Senkang, et al.
Publicado: (2025)

Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learning
por: Zhang, Haozhen, et al.
Publicado: (2025)

Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
por: Zhang, Haozhen, et al.
Publicado: (2024)

Speculative Decoding for Multi-Sample Inference
por: Li, Yiwei, et al.
Publicado: (2025)

Large-Scale Constraint Generation -- Can LLMs Parse Hundreds of Constraints?
por: Boffa, Matteo, et al.
Publicado: (2025)

Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
por: Sun, Shengyin, et al.
Publicado: (2025)

DeFT-Mamba: Universal Multichannel Sound Separation and Polyphonic Audio Classification
por: Lee, Dongheon, et al.
Publicado: (2024)

Paper Copilot: A Self-Evolving and Efficient LLM System for Personalized Academic Assistance
por: Lin, Guanyu, et al.
Publicado: (2024)

CSV-Decode: Certifiable Sub-Vocabulary Decoding for Efficient Large Language Model Inference
por: Liu, Dong, et al.
Publicado: (2025)

PDTrim: Targeted Pruning for Prefill-Decode Disaggregation in Inference
por: Zhang, Hao, et al.
Publicado: (2025)

Plato: Plan to Efficiently Decode for Large Language Model Inference
por: Jin, Shuowei, et al.
Publicado: (2024)

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference
por: Li, Zeping, et al.
Publicado: (2024)

FreeKV: Boosting KV Cache Retrieval for Efficient LLM Inference
por: Liu, Guangda, et al.
Publicado: (2025)

Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency
por: Li, Ruixiao, et al.
Publicado: (2025)

Goose: Anisotropic Speculation Trees for Training-Free Speculative Decoding
por: Jin, Tao, et al.
Publicado: (2026)

BUZZ: Beehive-structured Sparse KV Cache with Segmented Heavy Hitters for Efficient LLM Inference
por: Zhao, Junqi, et al.
Publicado: (2024)

Reward-Guided Speculative Decoding for Efficient LLM Reasoning
por: Liao, Baohao, et al.
Publicado: (2025)

FlashThink: An Early Exit Method For Efficient Reasoning
por: Jiang, Guochao, et al.
Publicado: (2025)

CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
por: Zheng, Wenhao, et al.
Publicado: (2025)

Cerberus: Efficient Inference with Adaptive Parallel Decoding and Sequential Knowledge Enhancement
por: Liu, Yuxuan, et al.
Publicado: (2024)

Beyond Facts: Evaluating Intent Hallucination in Large Language Models
por: Hao, Yijie, et al.
Publicado: (2025)

JoyAI-LLM Flash: Advancing Mid-Scale LLMs with Token Efficiency
por: Cai, Aichen, et al.
Publicado: (2026)

KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning
por: Zhang, Kaiqi, et al.
Publicado: (2024)

FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
por: Lu, Yu-Chen, et al.
Publicado: (2025)

Local Success Does Not Compose: Benchmarking Large Language Models for Compositional Formal Verification
por: Xu, Xu, et al.
Publicado: (2025)

FlashDecoding++: Faster Large Language Model Inference on GPUs
por: Hong, Ke, et al.
Publicado: (2023)

I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Models
por: Hu, Xing, et al.
Publicado: (2024)

Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference
por: Simonds, Toby
Publicado: (2025)

Speculative Decoding with CTC-based Draft Model for LLM Inference Acceleration
por: Wen, Zhuofan, et al.
Publicado: (2024)

Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
por: Timor, Nadav, et al.
Publicado: (2025)