Cover Image

MELODI: Exploring Memory Compression for Long Contexts

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Yinpeng, Hutchins, DeLesley, Jansen, Aren, Zhmoginov, Andrey, Racz, David, Andersen, Jesper
Format:	Preprint
Published:	2024
Subjects:	Machine Learning Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Long Context In-Context Compression by Getting to the Gist of Gisting
by: Petrov, Aleksandar, et al.
Published: (2025)

Goal-Directed Search Outperforms Goal-Agnostic Memory Compression in Long-Context Memory Tasks
by: Zheng, Yicong, et al.
Published: (2025)

CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling
by: Zhao, Runsong, et al.
Published: (2026)

Learning to Remember: End-to-End Training of Memory Agents for Long-Context Reasoning
by: Zhang, Kehao, et al.
Published: (2026)

MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning
by: Liu, Dong, et al.
Published: (2026)

Alignment Dynamics in LLM Fine-Tuning
by: Huang, Yuhan, et al.
Published: (2026)

SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training
by: Li, Zhouyang, et al.
Published: (2025)

RMAAT: Astrocyte-Inspired Memory Compression and Replay for Efficient Long-Context Transformers
by: Mia, Md Zesun Ahmed, et al.
Published: (2026)

COSMIR: Chain Orchestrated Structured Memory for Iterative Reasoning over Long Context
by: Gupta, Naman, et al.
Published: (2025)

GeneZip: Region-Aware Compression for Long Context DNA Modeling
by: Zhao, Jianan, et al.
Published: (2026)

Latent Context Compilation: Distilling Long Context into Compact Portable Memory
by: Li, Zeju, et al.
Published: (2026)

SCRIPT: Implementing an Intelligent Tutoring System for Programming in a German University Context
by: Deriyeva, Alina, et al.
Published: (2026)

Long Context Modeling with Ranked Memory-Augmented Retrieval
by: Alselwi, Ghadir, et al.
Published: (2025)

DNN Memory Footprint Reduction via Post-Training Intra-Layer Multi-Precision Quantization
by: Ghavami, Behnam, et al.
Published: (2024)

Context Distillation as Latent Memory Management
by: Zheng, Ziyang, et al.
Published: (2026)

Lattice: Learning to Efficiently Compress the Memory
by: Karami, Mahdi, et al.
Published: (2025)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

Compressing Many-Shots in In-Context Learning
by: Khatri, Devvrit, et al.
Published: (2025)

ContextEvolve: Multi-Agent Context Compression for Systems Code Optimization
by: Su, Hongyuan, et al.
Published: (2026)

Why Do Time Series Models Need Long Context Windows?
by: Butera, Luca, et al.
Published: (2026)

MemAgent: Reshaping Long-Context LLM with Multi-Conv RL-based Memory Agent
by: Yu, Hongli, et al.
Published: (2025)

Memory-Efficient Fine-Tuning via Low-Rank Activation Compression
by: Shi, Jiang-Xin, et al.
Published: (2025)

Long Term Memory: The Foundation of AI Self-Evolution
by: Jiang, Xun, et al.
Published: (2024)

Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting
by: Chen, Wei, et al.
Published: (2024)

PolicyLong: Towards On-Policy Context Extension
by: Jia, Junlong, et al.
Published: (2026)

COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training
by: Xi, Haocheng, et al.
Published: (2024)

Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context
by: Joo, Taejong, et al.
Published: (2025)

VSPrefill: Vertical-Slash Sparse Attention with Lightweight Indexing for Long-Context Prefilling
by: Guanzhong, Chen
Published: (2026)

Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information
by: Zhu, Yifan, et al.
Published: (2026)

DySCo: Dynamic Semantic Compression for Effective Long-term Time Series Forecasting
by: Ao, Xiang, et al.
Published: (2026)

MOM: Memory-Efficient Offloaded Mini-Sequence Inference for Long Context Language Models
by: Zhang, Junyang, et al.
Published: (2025)

Exploring Context Generalizability in Citywide Crowd Mobility Prediction: An Analytic Framework and Benchmark
by: Chen, Liyue, et al.
Published: (2021)

KV Admission: Learning What to Write for Efficient Long-Context Inference
by: Huang, Yen-Chieh, et al.
Published: (2025)

Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
by: Song, Woomin, et al.
Published: (2024)

Long-Context Attention Benchmark: From Kernel Efficiency to Distributed Context Parallelism
by: Bu, Tao, et al.
Published: (2025)

Task-Core Memory Management and Consolidation for Long-term Continual Learning
by: Huai, Tianyu, et al.
Published: (2025)

Robust Finite-Memory Policy Gradients for Hidden-Model POMDPs
by: Galesloot, Maris F. L., et al.
Published: (2025)

OjaKV: Context-Aware Online Low-Rank KV Cache Compression
by: Zhu, Yuxuan, et al.
Published: (2025)

NeuZip: Memory-Efficient Training and Inference with Dynamic Compression of Neural Networks
by: Hao, Yongchang, et al.
Published: (2024)

StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigation
by: Chen, Zhiyuan, et al.
Published: (2026)