Cover Image

KV Prediction for Improved Time to First Token

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Horton, Maxwell, Cao, Qingqing, Sun, Chenfan, Jin, Yanzi, Mehta, Sachin, Rastegari, Mohammad, Nabi, Moin
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

OpenELM: An Efficient Language Model Family with Open Training and Inference Framework
by: Mehta, Sachin, et al.
Published: (2024)

LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inference
by: Fu, Qichen, et al.
Published: (2024)

CatLIP: CLIP-level Visual Recognition Accuracy with 2.7x Faster Pre-training on Web-scale Image-Text Data
by: Mehta, Sachin, et al.
Published: (2024)

Diffusion Models as Masked Audio-Video Learners
by: Nunez, Elvis, et al.
Published: (2023)

KV-Runahead: Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
by: Cho, Minsik, et al.
Published: (2024)

Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers
by: Wen, Yuxin, et al.
Published: (2024)

From Dense to Dynamic: Token-Difficulty Driven MoEfication of Pre-Trained LLMs
by: Nishu, Kumari, et al.
Published: (2025)

Superposition Prompting: Improving and Accelerating Retrieval-Augmented Generation
by: Merth, Thomas, et al.
Published: (2024)

Bytes Are All You Need: Transformers Operating Directly On File Bytes
by: Horton, Maxwell, et al.
Published: (2023)

IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
by: Liu, Ruikang, et al.
Published: (2024)

BTR: Binary Token Representations for Efficient Retrieval Augmented Language Models
by: Cao, Qingqing, et al.
Published: (2023)

G-KV: Decoding-Time KV Cache Eviction with Global Attention
by: Liao, Mengqi, et al.
Published: (2025)

KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
by: Yuan, Aomufei, et al.
Published: (2025)

CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning
by: Cao, Qingqing, et al.
Published: (2024)

HySparse: A Hybrid Sparse Attention Architecture with Oracle Token Selection and KV Cache Sharing
by: Gao, Yizhao, et al.
Published: (2026)

Do Compressed LLMs Forget Knowledge? An Experimental Study with Practical Implications
by: Hoang, Duc N. M, et al.
Published: (2023)

One Token Is Enough: Improving Diffusion Language Models with a Sink Token
by: Zhang, Zihou, et al.
Published: (2026)

Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference
by: Feng, Yuan, et al.
Published: (2024)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
by: Samragh, Mohammad, et al.
Published: (2024)

CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
by: Lin, Xiaolin, et al.
Published: (2025)

FarSSiBERT: A Novel Transformer-based Model for Semantic Similarity Measurement of Persian Social Networks Informal Texts
by: Sadjadi, Seyed Mojtaba, et al.
Published: (2024)

SeedLM: Compressing LLM Weights into Seeds of Pseudo-Random Generators
by: Shafipour, Rasoul, et al.
Published: (2024)

R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
by: Cai, Zefan, et al.
Published: (2025)

AhaKV: Adaptive Holistic Attention-Driven KV Cache Eviction for Efficient Inference of Large Language Models
by: Gu, Yifeng, et al.
Published: (2025)

Crystal-KV: Efficient KV Cache Management for Chain-of-Thought LLMs via Answer-First Principle
by: Wang, Zihan, et al.
Published: (2026)

SemShareKV: Efficient KVCache Sharing for Semantically Similar Prompts via Token-Level LSH Matching
by: Zhao, Xinye, et al.
Published: (2025)

Lossless KV Cache Compression to 2%
by: Yang, Zhen, et al.
Published: (2024)

SentenceKV: Efficient LLM Inference via Sentence-Level Semantic KV Caching
by: Zhu, Yuxuan, et al.
Published: (2025)

LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
by: Liang, Manlai, et al.
Published: (2025)

Learning Private Representations through Entropy-based Adversarial Training
by: Klein, Tassilo, et al.
Published: (2025)

Speculative Streaming: Fast LLM Inference without Auxiliary Models
by: Bhendawade, Nikhil, et al.
Published: (2024)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

KVSculpt: KV Cache Compression as Distillation
by: Jiang, Bo, et al.
Published: (2026)

PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
by: Cai, Zefan, et al.
Published: (2024)

Finish First, Perfect Later: Test-Time Token-Level Cross-Validation for Diffusion Large Language Models
by: Tian, Runchu, et al.
Published: (2025)

One Size Does Not Fit All: Token-Wise Adaptive Compression for KV Cache
by: Lu, Liming, et al.
Published: (2026)

TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
by: Wu, Wei, et al.
Published: (2024)

DepthKV: Layer-Dependent KV Cache Pruning for Long-Context LLM Inference
by: Dehghanighobadi, Zahra, et al.
Published: (2026)

DeltaKV: Residual-Based KV Cache Compression via Long-Range Similarity
by: Hao, Jitai, et al.
Published: (2026)

Fixing the Broken Compass: Diagnosing and Improving Inference-Time Reward Modeling
by: Li, Jiachun, et al.
Published: (2025)