Cover Image

Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Nawrot, Piotr, Łańcucki, Adrian, Chochowski, Marcin, Tarjan, David, Ponti, Edoardo M.
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Inference-Time Hyper-Scaling with KV Cache Compression
by: Łańcucki, Adrian, et al.
Published: (2025)

The Sparse Frontier: Sparse Attention Trade-offs in Transformer LLMs
by: Nawrot, Piotr, et al.
Published: (2025)

KV Cache Transform Coding for Compact Storage in LLM Inference
by: Staniszewski, Konrad, et al.
Published: (2025)

Scaling Fine-Grained MoE Beyond 50B Parameters: Empirical Evaluation and Practical Insights
by: Krajewski, Jakub, et al.
Published: (2025)

Cross-Lingual and Cross-Cultural Variation in Image Descriptions
by: Berger, Uri, et al.
Published: (2024)

Efficient Safety Retrofitting Against Jailbreaking for LLMs
by: Garcia-Gasulla, Dario, et al.
Published: (2025)

Retrofitting Large Language Models with Dynamic Tokenization
by: Feher, Darius, et al.
Published: (2024)

Zero-Shot Tokenizer Transfer
by: Minixhofer, Benjamin, et al.
Published: (2024)

Universal Cross-Tokenizer Distillation via Approximate Likelihood Matching
by: Minixhofer, Benjamin, et al.
Published: (2025)

A Grounded Typology of Word Classes
by: Haley, Coleman, et al.
Published: (2024)

Fine-tuning Large Language Models with Sequential Instructions
by: Hu, Hanxu, et al.
Published: (2024)

Is Information Density Uniform when Utterances are Grounded on Perception and Discourse?
by: Gay, Matteo, et al.
Published: (2026)

Probing the Emergence of Cross-lingual Alignment during LLM Training
by: Wang, Hetong, et al.
Published: (2024)

Bootstrapping Action-Grounded Visual Dynamics in Unified Vision-Language Models
by: Qiu, Yifu, et al.
Published: (2025)

Emergent Communication Pretraining for Few-Shot Machine Translation
by: Li, Yaoyiran, et al.
Published: (2020)

The Impact of Inference Acceleration on Bias of LLMs
by: Kirsten, Elisabeth, et al.
Published: (2024)

Emergent Inference-Time Semantic Contamination via In-Context Priming
by: Abram, Marcin
Published: (2026)

Post-hoc Reward Calibration: A Case Study on Length Bias
by: Huang, Zeyu, et al.
Published: (2024)

Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
by: Taghibakhshi, Ali, et al.
Published: (2025)

Towards Modular LLMs by Building and Reusing a Library of LoRAs
by: Ostapenko, Oleksiy, et al.
Published: (2024)

Minitron-SSM: Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning
by: Taghibakhshi, Ali, et al.
Published: (2025)

Embedding Retrofitting: Data Engineering for better RAG
by: Sharma, Anantha
Published: (2026)

Scaling Sparse Fine-Tuning to Large Language Models
by: Ansell, Alan, et al.
Published: (2024)

ChunkLLM: A Lightweight Pluggable Framework for Accelerating LLMs Inference
by: Ouyang, Haojie, et al.
Published: (2025)

Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference
by: Chen, Hao Mark, et al.
Published: (2024)

Sample-efficient Integration of New Modalities into Large Language Models
by: İnce, Osman Batur, et al.
Published: (2025)

Dynamic Compressing Prompts for Efficient Inference of Large Language Models
by: Hu, Jinwu, et al.
Published: (2025)

LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
by: Jiang, Huiqiang, et al.
Published: (2023)

Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
by: Song, Yuerong, et al.
Published: (2025)

Understanding the Physics of Key-Value Cache Compression for LLMs through Attention Dynamics
by: Ananthanarayanan, Samhruth, et al.
Published: (2026)

DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokens
by: Chen, Shaoshen, et al.
Published: (2025)

ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
by: Liu, Xin, et al.
Published: (2025)

DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling
by: Tuli, Shikhar, et al.
Published: (2024)

Evaluation of Code LLMs on Geospatial Code Generation
by: Gramacki, Piotr, et al.
Published: (2024)

APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUs
by: Huang, Yuxiang, et al.
Published: (2025)

FairSteer: Inference Time Debiasing for LLMs with Dynamic Activation Steering
by: Li, Yichen, et al.
Published: (2025)

Retrofitting Small Multilingual Models for Retrieval: Matching 7B Performance with 300M Parameters
by: Tu, Lifu, et al.
Published: (2025)

DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs
by: Zhou, Xiabin, et al.
Published: (2024)

Mixtures of In-Context Learners
by: Hong, Giwon, et al.
Published: (2024)

SlimInfer: Accelerating Long-Context LLM Inference via Dynamic Token Pruning
by: Long, Lingkun, et al.
Published: (2025)