Informed Routing in LLMs: Smarter Token-Level Computation for Faster Inference

Documenti analoghi

• Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the Edge
  di: Shen, Xuan, et al.
  Pubblicazione: (2023)
• Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference
  di: Warner, Benjamin, et al.
  Pubblicazione: (2024)
• TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
  di: Wu, Wei, et al.
  Pubblicazione: (2024)
• An Expert is Worth One Token: Synergizing Multiple Expert LLMs as Generalist via Expert Token Routing
  di: Chai, Ziwei, et al.
  Pubblicazione: (2024)
• CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
  di: Zheng, Wenhao, et al.
  Pubblicazione: (2025)