Cover Image

Trellis: Learning to Compress Key-Value Memory in Attention Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Karami, Mahdi, Behrouz, Ali, Kacham, Praneeth, Mirrokni, Vahab
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Lattice: Learning to Efficiently Compress the Memory
by: Karami, Mahdi, et al.
Published: (2025)

Titans: Learning to Memorize at Test Time
by: Behrouz, Ali, et al.
Published: (2024)

PolySketchFormer: Fast Transformers via Sketching Polynomial Kernels
by: Kacham, Praneeth, et al.
Published: (2023)

TNT: Improving Chunkwise Training for Test-Time Memorization
by: Li, Zeman, et al.
Published: (2025)

MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling
by: Karami, Mahdi, et al.
Published: (2025)

ATLAS: Learning to Optimally Memorize the Context at Test Time
by: Behrouz, Ali, et al.
Published: (2025)

Best of Both Worlds: Advantages of Hybrid Graph Sequence Models
by: Behrouz, Ali, et al.
Published: (2024)

PolarQuant: Quantizing KV Caches with Polar Transformation
by: Han, Insu, et al.
Published: (2025)

Efficient Data Selection at Scale via Influence Distillation
by: Nikdan, Mahdi, et al.
Published: (2025)

ECO: Quantized Training without Full-Precision Master Weights
by: Nikdan, Mahdi, et al.
Published: (2026)

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
by: Li, Zeman, et al.
Published: (2024)

It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimization
by: Behrouz, Ali, et al.
Published: (2025)

Nested Learning: The Illusion of Deep Learning Architectures
by: Behrouz, Ali, et al.
Published: (2025)

Memory Caching: RNNs with Growing Memory
by: Behrouz, Ali, et al.
Published: (2026)

Synthetic Text Generation for Training Large Language Models via Gradient Matching
by: Nguyen, Dang, et al.
Published: (2025)

LoMA: Lossless Compressed Memory Attention
by: Wang, Yumeng, et al.
Published: (2024)

Hydra: Dual Exponentiated Memory for Multivariate Time Series Analysis
by: Meskin, Asal, et al.
Published: (2025)

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
by: Brandon, William, et al.
Published: (2024)

LML-DAP: Language Model Learning a Dataset for Data-Augmented Prediction
by: Vadlapati, Praneeth
Published: (2024)

Learning to Evict from Key-Value Cache
by: Moschella, Luca, et al.
Published: (2026)

LevAttention: Time, Space, and Streaming Efficient Algorithm for Heavy Attentions
by: Kannan, Ravindran, et al.
Published: (2024)

Key-Value Means: Transformers with Expandable Block-Recurrent Compressed Memory
by: Goldstein, Daniel, et al.
Published: (2026)

Compressed Context Memory For Online Language Model Interaction
by: Kim, Jang-Hyun, et al.
Published: (2023)

Causal Attention with Lookahead Keys
by: Song, Zhuoqing, et al.
Published: (2025)

Sampling and Loss Weights in Multi-Domain Training
by: Salmani, Mahdi, et al.
Published: (2025)

Coupled Query-Key Dynamics for Attention
by: Gahtan, Barak, et al.
Published: (2026)

ZoomR: Memory Efficient Reasoning through Multi-Granularity Key Value Retrieval
by: Yang, David H., et al.
Published: (2026)

Optimal Communication for Classic Functions in the Coordinator Model and Beyond
by: Esfandiari, Hossein, et al.
Published: (2024)

SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Models
by: Duanmu, Haojie, et al.
Published: (2024)

Probing the Limits of Compressive Memory: A Study of Infini-Attention in Small-Scale Pretraining
by: Huang, Ruizhe, et al.
Published: (2025)

DREaM: Drug-Drug Relation Extraction via Transfer Learning Method
by: Fata, Ali, et al.
Published: (2025)

Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks
by: Sabbaghi, Mahdi, et al.
Published: (2024)

SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression
by: Wang, Xin, et al.
Published: (2024)

Auto-Regressive Masked Diffusion Models
by: Karami, Mahdi, et al.
Published: (2026)

Theoretical Perspectives on Data Quality and Synergistic Effects in Pre- and Post-Training Reasoning Models
by: Javanmard, Adel, et al.
Published: (2026)

PriorBoost: An Adaptive Algorithm for Learning from Aggregate Responses
by: Javanmard, Adel, et al.
Published: (2024)

Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling
by: Karami, Mahdi, et al.
Published: (2024)

SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
by: S, Santhosh G, et al.
Published: (2025)

CompAct: Compressed Activations for Memory-Efficient LLM Training
by: Shamshoum, Yara, et al.
Published: (2024)

AttentionPredictor: Temporal Patterns Matter for KV Cache Compression
by: Yang, Qingyue, et al.
Published: (2025)