Cover Image

Capacity Matters: a Proof-of-Concept for Transformer Memorization on Real-World Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Changalidis, Anton, Härmä, Aki
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Empirical Capacity Model for Self-Attention Neural Networks
by: Härmä, Aki, et al.
Published: (2024)

A Systematic Review on the Generative AI Applications in Human Medical Genomics
by: Changalidis, Anton, et al.
Published: (2025)

Too Big to Think: Capacity, Memorization, and Generalization in Pre-Trained Transformers
by: Barron, Joshua, et al.
Published: (2025)

Survey on biomarkers in human vocalizations
by: Härmä, Aki, et al.
Published: (2024)

Memorization in Attention-only Transformers
by: Dana, Léo, et al.
Published: (2024)

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
by: Hans, Abhimanyu, et al.
Published: (2024)

What Matters in Memorizing and Recalling Facts? Multifaceted Benchmarks for Knowledge Probing in Language Models
by: Zhao, Xin, et al.
Published: (2024)

Data Compressibility Quantifies LLM Memorization
by: Huang, Yizhan, et al.
Published: (2025)

Bidirectional LMs are Better Knowledge Memorizers? A Benchmark for Real-world Knowledge Injection
by: Zhang, Yuwei, et al.
Published: (2025)

DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios
by: Meng, Jinxiang, et al.
Published: (2026)

Generative AI-Based Virtual Assistant using Retrieval-Augmented Generation: An evaluation study for bachelor projects
by: Verşebeniuc, Dumitru, et al.
Published: (2026)

Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs
by: Wu, Qinyuan, et al.
Published: (2025)

Impact of Layer Norm on Memorization and Generalization in Transformers
by: Singhal, Rishi, et al.
Published: (2025)

Private Memorization Editing: Turning Memorization into a Defense to Strengthen Data Privacy in Large Language Models
by: Ruzzetti, Elena Sofia, et al.
Published: (2025)

Theoretical Proof that Auto-regressive Language Models Collapse when Real-world Data is a Finite Set
by: Wang, Lecheng, et al.
Published: (2024)

OWL: Probing Cross-Lingual Recall of Memorized Texts via World Literature
by: Srivastava, Alisha, et al.
Published: (2025)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
by: Xiao, Jianfei, et al.
Published: (2026)

Exploring Robustness of Multilingual LLMs on Real-World Noisy Data
by: Aliakbarzadeh, Amirhossein, et al.
Published: (2025)

Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Models
by: Grishina, Anastasiia, et al.
Published: (2025)

Memorization or Interpolation ? Detecting LLM Memorization through Input Perturbation Analysis
by: Djiré, Albérick Euraste, et al.
Published: (2025)

Long-Term Ad Memorability: Understanding & Generating Memorable Ads
by: SI, Harini, et al.
Published: (2023)

Continual Memorization of Factoids in Language Models
by: Chen, Howard, et al.
Published: (2024)

Elephants Never Forget: Testing Language Models for Memorization of Tabular Data
by: Bordt, Sebastian, et al.
Published: (2024)

Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks
by: Chang, Ting-Yun, et al.
Published: (2023)

Quantifying Memorization and Detecting Training Data of Pre-trained Language Models using Japanese Newspaper
by: Ishihara, Shotaro, et al.
Published: (2024)

Memorization or Reasoning? Exploring the Idiom Understanding of LLMs
by: Kim, Jisu, et al.
Published: (2025)

Generalization or Memorization: Dynamic Decoding for Mode Steering
by: Zhang, Xuanming
Published: (2025)

Mitigating Memorization in LLMs using Activation Steering
by: Suri, Manan, et al.
Published: (2025)

Memorization Dynamics in Knowledge Distillation for Language Models
by: Borkar, Jaydeep, et al.
Published: (2026)

On Memorization of Large Language Models in Logical Reasoning
by: Xie, Chulin, et al.
Published: (2024)

A Content-Based Novelty Measure for Scholarly Publications: A Proof of Concept
by: Wang, Haining
Published: (2024)

Evaluating LLM Alignment on Personality Inference from Real-World Interview Data
by: Zhu, Jianfeng, et al.
Published: (2025)

Universal Abstraction: Harnessing Frontier Models to Structure Real-World Data at Scale
by: Wong, Cliff, et al.
Published: (2025)

Retrieval-Augmented Generation of Pediatric Speech-Language Pathology vignettes: A Proof-of-Concept Study
by: Liu, Yilan
Published: (2025)

Assessing and Mitigating Data Memorization Risks in Fine-Tuned Large Language Models
by: Ramakrishnan, Badrinath, et al.
Published: (2025)

Cram Less to Fit More: Training Data Pruning Improves Memorization of Facts
by: Ye, Jiayuan, et al.
Published: (2026)

How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translation
by: Meng, Yan, et al.
Published: (2024)

Data-centric NLP Backdoor Defense from the Lens of Memorization
by: Wang, Zhenting, et al.
Published: (2024)

Data Cartography for Detecting Memorization Hotspots and Guiding Data Interventions in Generative Models
by: Patel, Laksh, et al.
Published: (2025)

Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework
by: Luo, Xiaoyu, et al.
Published: (2026)