DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference

Documenti analoghi

• ConsistencyChecker: Tree-based Evaluation of LLM Generalization Capabilities
  di: Hong, Zhaochen, et al.
  Pubblicazione: (2025)
• Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decoding
  di: Zhang, Kexun, et al.
  Pubblicazione: (2023)
• Scaling LLM Inference with Optimized Sample Compute Allocation
  di: Zhang, Kexun, et al.
  Pubblicazione: (2024)
• DeFT: Mitigating Data Dependencies for Flexible Communication Scheduling in Distributed Training
  di: Meng, Lin, et al.
  Pubblicazione: (2025)
• LycheeDecode: Accelerating Long-Context LLM Inference via Hybrid-Head Sparse Decoding
  di: Lin, Gang, et al.
  Pubblicazione: (2026)