Imagen de Portada

Short window attention enables long-term memorization

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Cabannes, Loïc, Beck, Maximilian, Szilvasy, Gergely, Douze, Matthijs, Lomeli, Maria, Copet, Jade, Mazaré, Pierre-Emmanuel, Synnaeve, Gabriel, Jégou, Hervé
Formato:	Preprint
Publicado:	2025
Materias:	Machine Learning Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Stochastic activations
por: Lomeli, Maria, et al.
Publicado: (2025)

Inference-time sparse attention with asymmetric indexing
por: Mazaré, Pierre-Emmanuel, et al.
Publicado: (2025)

Self-Pruned Key-Value Attention: Learning When to Write by Predicting Future Utility
por: Szilvasy, Gergely, et al.
Publicado: (2026)

Vector search with small radiuses
por: Szilvasy, Gergely, et al.
Publicado: (2024)

The Faiss library
por: Douze, Matthijs, et al.
Publicado: (2024)

Machine learning and high dimensional vector search
por: Douze, Matthijs
Publicado: (2025)

RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning
por: Gehring, Jonas, et al.
Publicado: (2024)

Simple and Controllable Music Generation
por: Copet, Jade, et al.
Publicado: (2023)

Functional Invariants to Watermark Large Transformers
por: Fernandez, Pierre, et al.
Publicado: (2023)

Towards a Neural Debugger for Python
por: Beck, Maximilian, et al.
Publicado: (2026)

SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution
por: Wei, Yuxiang, et al.
Publicado: (2025)

Masked Audio Generation using a Single Non-Autoregressive Transformer
por: Ziv, Alon, et al.
Publicado: (2024)

Watermark Anything with Localized Messages
por: Sander, Tom, et al.
Publicado: (2024)

Watermarking Makes Language Models Radioactive
por: Sander, Tom, et al.
Publicado: (2024)

Qinco2: Vector Compression and Search with Improved Implicit Neural Codebooks
por: Vallaeys, Théophane, et al.
Publicado: (2025)

Moshi: a speech-text foundation model for real-time dialogue
por: Défossez, Alexandre, et al.
Publicado: (2024)

Evaluation data contamination in LLMs: how do we measure it and (when) does it matter?
por: Singh, Aaditya K., et al.
Publicado: (2024)

Textually Pretrained Speech Language Models
por: Hassid, Michael, et al.
Publicado: (2023)

Automatic Textbook Formalization
por: Gloeckle, Fabian, et al.
Publicado: (2026)

KVzap: Fast, Adaptive, and Faithful KV Cache Pruning
por: Jegou, Simon, et al.
Publicado: (2026)

Lossless Compression of Vector IDs for Approximate Nearest Neighbor Search
por: Severo, Daniel, et al.
Publicado: (2025)

Audio Conditioning for Music Generation via Discrete Bottleneck Features
por: Rouard, Simon, et al.
Publicado: (2024)

Successful reintroduction of species: improving on windows of opportunity for biodiversity repair
por: Ann‐Kathrin Tielke, et al.
Publicado: (2024)

An Independence-promoting Loss for Music Generation with Language Models
por: Lemercier, Jean-Marie, et al.
Publicado: (2024)

BigO(Bench) -- Can LLMs Generate Code with Controlled Time and Space Complexity?
por: Chambon, Pierre, et al.
Publicado: (2025)

Neutral Residues: Revisiting Adapters for Model Extension
por: Talla, Franck Signe, et al.
Publicado: (2024)

Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
por: Devoto, Alessio, et al.
Publicado: (2025)

La hospitalidad en profesores memorables universitarios
por: Luis Gabriel Porta
Publicado: (2017)

Robust Offset-free Kernelized Data-Driven Predictive Control for Nonlinear Systems
por: Mazare, Mahmood, et al.
Publicado: (2025)

Noise-Tolerant Hybrid Approach for Data-Driven Predictive Control
por: Mazare, Mahmood, et al.
Publicado: (2025)

Short and long‐term effects of experimental varicocele
por: Aram Minas, et al.
Publicado: (2025)

TestGenEval: A Real World Unit Test Generation and Test Completion Benchmark
por: Jain, Kush, et al.
Publicado: (2024)

Getting the most out of your tokenizer for pre-training and domain adaptation
por: Dagan, Gautier, et al.
Publicado: (2024)

Surgical stabilization technique and long‐term outcome of traumatic lateral shoulder luxation in a dog
por: Maria Podsiedlik, et al.
Publicado: (2025)

Disentangling generalization and memorization in large language models using chess
por: Pleiss, Leonard S., et al.
Publicado: (2026)

Touring sampling with pushforward maps
por: Cabannes, Vivien, et al.
Publicado: (2023)

The Galerkin method beats Graph-Based Approaches for Spectral Algorithms
por: Cabannes, Vivien, et al.
Publicado: (2023)

RA-DIT: Retrieval-Augmented Dual Instruction Tuning
por: Lin, Xi Victoria, et al.
Publicado: (2023)

Residual Quantization with Implicit Neural Codebooks
por: Huijben, Iris A. M., et al.
Publicado: (2024)

Losing dimensions: Geometric memorization in generative diffusion
por: Achilli, Beatrice, et al.
Publicado: (2024)