Cover Image

Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chang, Ting-Yun, Thomason, Jesse, Jia, Robin
Format:	Preprint
Published:	2023
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Models
by: Chang, Ting-Yun, et al.
Published: (2024)

Language Models can Infer Action Semantics for Symbolic Planners from Environment Feedback
by: Zhu, Wang, et al.
Published: (2024)

Guess or Recall? Training CNNs to Classify and Localize Memorization in LLMs
by: Dentan, Jérémie, et al.
Published: (2025)

Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization
by: Tseng, Yu-Min, et al.
Published: (2024)

Why Do Some Inputs Break Low-Bit LLM Quantization?
by: Chang, Ting-Yun, et al.
Published: (2025)

PDDL-Mind: Large Language Models are Capable on Belief Reasoning with Reliable State Tracking
by: Zhu, Wang Bill, et al.
Published: (2026)

A Tale of Two Structures: Do LLMs Capture the Fractal Complexity of Language?
by: Alabdulmohsin, Ibrahim, et al.
Published: (2025)

PSALM-V: Automating Symbolic Planning in Interactive Visual Environments with Large Language Models
by: Zhu, Wang Bill, et al.
Published: (2025)

Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
by: Srinivasan, Tejas, et al.
Published: (2025)

Large Language Models Do Multi-Label Classification Differently
by: Ma, Marcus, et al.
Published: (2025)

Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
by: Ren, Richard, et al.
Published: (2024)

Efficient End-to-End Visual Document Understanding with Rationale Distillation
by: Zhu, Wang, et al.
Published: (2023)

When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retraction
by: Yang, Yuqing, et al.
Published: (2025)

Localizing Paragraph Memorization in Language Models
by: Stoehr, Niklas, et al.
Published: (2024)

Unveiling Over-Memorization in Finetuning LLMs for Reasoning Tasks
by: Ruan, Zhiwen, et al.
Published: (2025)

Phonological Representation Learning for Isolated Signs Improves Out-of-Vocabulary Generalization
by: Kezar, Lee, et al.
Published: (2025)

TwoStep: Multi-agent Task Planning using Classical Planners and Large Language Models
by: Bai, David, et al.
Published: (2024)

LocalBench: Benchmarking LLMs on County-Level Local Knowledge and Reasoning
by: Gao, Zihan, et al.
Published: (2025)

Short-Context Dominance: How Much Local Context Natural Language Actually Needs?
by: Vakilian, Vala, et al.
Published: (2025)

Few-Shot VQA with Frozen LLMs: A Tale of Two Approaches
by: Sterner, Igor, et al.
Published: (2024)

Do LLMs Really Memorize Personally Identifiable Information? Revisiting PII Leakage with a Cue-Controlled Memorization Framework
by: Luo, Xiaoyu, et al.
Published: (2026)

Instructional Goal-Aligned Question Generation for Student Evaluation in Virtual Lab Settings: How Closely Do LLMs Actually Align?
by: Knipper, R. Alexander, et al.
Published: (2025)

WinoViz: Probing Visual Properties of Objects Under Different States
by: Jin, Woojeong, et al.
Published: (2024)

Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs
by: Hans, Abhimanyu, et al.
Published: (2024)

What Do Claim Verification Datasets Actually Test? A Reasoning Trace Analysis
by: Rao, Delip, et al.
Published: (2026)

When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs
by: Kamoi, Ryo, et al.
Published: (2024)

Words that make SENSE: Sensorimotor Norms in Learned Lexical Token Representations
by: Gupta, Abhinav, et al.
Published: (2026)

Benchmarking Chinese Commonsense Reasoning of LLMs: From Chinese-Specifics to Reasoning-Memorization Correlations
by: Sun, Jiaxing, et al.
Published: (2024)

Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs
by: Wu, Qinyuan, et al.
Published: (2025)

Iterative Formalization and Planning in Partially Observable Environments
by: Gong, Liancheng, et al.
Published: (2025)

Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs
by: Kassem, Aly M., et al.
Published: (2024)

Memorization or Reasoning? Exploring the Idiom Understanding of LLMs
by: Kim, Jisu, et al.
Published: (2025)

Mitigating Memorization in LLMs using Activation Steering
by: Suri, Manan, et al.
Published: (2025)

Decomposing the Delta: What Do Models Actually Learn from Preference Pairs?
by: Lee, Chia-Hsuan, et al.
Published: (2026)

From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centered
by: Devic, Siddartha, et al.
Published: (2025)

Memorization and Knowledge Injection in Gated LLMs
by: Pan, Xu, et al.
Published: (2025)

QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation
by: Hong, Mengze, et al.
Published: (2025)

Benchmarking Local LLMs for Natural-Language-to-SQL Querying in Biopharmaceutical Manufacturing: An Empirical Benchmark on Consumer-Grade Hardware
by: Bhetwal, Sagar, et al.
Published: (2026)

Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows
by: Balashov, Yuri, et al.
Published: (2026)

Unique Hard Attention: A Tale of Two Sides
by: Jerad, Selim, et al.
Published: (2025)