Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism

Documenti analoghi

• FIRP: Faster LLM inference via future intermediate representation prediction
  di: Wu, Pengfei, et al.
  Pubblicazione: (2024)
• Graph-Structured Speculative Decoding
  di: Gong, Zhuocheng, et al.
  Pubblicazione: (2024)
• Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration
  di: Wu, Pengfei, et al.
  Pubblicazione: (2024)
• Dynamic Fisher-weighted Model Merging via Bayesian Optimization
  di: Lee, Sanwoo, et al.
  Pubblicazione: (2025)
• Ltri-LLM: Streaming Long Context Inference for LLMs with Training-Free Dynamic Triangular Attention Pattern
  di: Tang, Hongyin, et al.
  Pubblicazione: (2024)