Cover Image

Continual Memorization of Factoids in Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chen, Howard, Geng, Jiayi, Bhaskar, Adithya, Friedman, Dan, Chen, Danqi
Format:	Preprint
Published:	2024
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
by: Bhaskar, Adithya, et al.
Published: (2024)

Finding Transformer Circuits with Edge Pruning
by: Bhaskar, Adithya, et al.
Published: (2024)

Extracting Rule-based Descriptions of Attention Features in Transformers
by: Friedman, Dan, et al.
Published: (2025)

Language Models that Think, Chat Better
by: Bhaskar, Adithya, et al.
Published: (2025)

Improving Language Understanding from Screenshots
by: Gao, Tianyu, et al.
Published: (2024)

Cache Me If You Can: How Many KVs Do You Need for Effective Long-Context LMs?
by: Bhaskar, Adithya, et al.
Published: (2025)

Long-Context Language Modeling with Parallel Context Encoding
by: Yen, Howard, et al.
Published: (2024)

Representing Rule-based Chatbots with Transformers
by: Friedman, Dan, et al.
Published: (2024)

Interpretability Illusions in the Generalization of Simplified Models
by: Friedman, Dan, et al.
Published: (2023)

Unintentional Unalignment: Likelihood Displacement in Direct Preference Optimization
by: Razin, Noam, et al.
Published: (2024)

How to Train Long-Context Language Models (Effectively)
by: Gao, Tianyu, et al.
Published: (2024)

MinosEval: Distinguishing Factoid and Non-Factoid for Tailored Open-Ended QA Evaluation with LLMs
by: Fan, Yongqi, et al.
Published: (2025)

DySCO: Dynamic Attention-Scaling Decoding for Long-Context Language Models
by: Ye, Xi, et al.
Published: (2026)

Are Large Language Models Reliable AI Scientists? Assessing Reverse-Engineering of Black-Box Systems
by: Geng, Jiayi, et al.
Published: (2025)

Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks
by: Lee, Yoonsang, et al.
Published: (2026)

Mitigating Memorization in LLMs using Activation Steering
by: Suri, Manan, et al.
Published: (2025)

Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
by: Chen, Howard, et al.
Published: (2025)

Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision
by: He, Yinghui, et al.
Published: (2026)

LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation
by: Ye, Xi, et al.
Published: (2025)

On Memorization of Large Language Models in Logical Reasoning
by: Xie, Chulin, et al.
Published: (2024)

Query-Focused Retrieval Heads Improve Long-Context Reasoning and Re-ranking
by: Zhang, Wuwei, et al.
Published: (2025)

TriviaHG: A Dataset for Automatic Hint Generation from Factoid Questions
by: Mozafari, Jamshid, et al.
Published: (2024)

Accumulating Context Changes the Beliefs of Language Models
by: Geng, Jiayi, et al.
Published: (2025)

QuRating: Selecting High-Quality Data for Training Language Models
by: Wettig, Alexander, et al.
Published: (2024)

Lory: Fully Differentiable Mixture-of-Experts for Autoregressive Language Model Pre-training
by: Zhong, Zexuan, et al.
Published: (2024)

HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly
by: Yen, Howard, et al.
Published: (2024)

A Multi-Perspective Analysis of Memorization in Large Language Models
by: Chen, Bowen, et al.
Published: (2024)

Metadata Conditioning Accelerates Language Model Pre-training
by: Gao, Tianyu, et al.
Published: (2025)

MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions
by: Zhong, Zexuan, et al.
Published: (2023)

Memorization Dynamics in Knowledge Distillation for Language Models
by: Borkar, Jaydeep, et al.
Published: (2026)

Language Models as Science Tutors
by: Chevalier, Alexis, et al.
Published: (2024)

Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning
by: Xia, Mengzhou, et al.
Published: (2023)

LINKAGE: Listwise Ranking among Varied-Quality References for Non-Factoid QA Evaluation via LLMs
by: Yang, Sihui, et al.
Published: (2024)

Multilingual Non-Factoid Question Answering with Answer Paragraph Selection
by: Mishra, Ritwik, et al.
Published: (2024)

Typed-RAG: Type-Aware Decomposition of Non-Factoid Questions for Retrieval-Augmented Generation
by: Lee, DongGeon, et al.
Published: (2025)

An Evaluation on Large Language Model Outputs: Discourse and Memorization
by: de Wynter, Adrian, et al.
Published: (2023)

Retentive or Forgetful? Diving into the Knowledge Memorizing Mechanism of Language Models
by: Cao, Boxi, et al.
Published: (2023)

Scaling Laws for Fact Memorization of Large Language Models
by: Lu, Xingyu, et al.
Published: (2024)

Memorization in Language Models through the Lens of Intrinsic Dimension
by: Arnold, Stefan
Published: (2025)

Mitigating Memorization In Language Models
by: Sakarvadia, Mansi, et al.
Published: (2024)