Cover Image

How Do Large Language Models Acquire Factual Knowledge During Pretraining?

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Chang, Hoyeon, Park, Jinho, Ye, Seonghyeon, Yang, Sohee, Seo, Youngkyung, Chang, Du-Seong, Seo, Minjoon
Format:	Preprint
Published:	2024
Subjects:	Computation and Language I.2.7
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Train-Attention: Meta-Learning Where to Focus in Continual Knowledge Learning
by: Seo, Yeongbin, et al.
Published: (2024)

Integrating Emotional and Linguistic Models for Ethical Compliance in Large Language Models
by: Chang, Edward Y.
Published: (2024)

Comprehensiveness Metrics for Automatic Evaluation of Factual Recall in Text Generation
by: Dejl, Adam, et al.
Published: (2025)

Internal Reasoning vs. External Control: A Thermodynamic Analysis of Sycophancy in Large Language Models
by: Chang, Edward Y.
Published: (2025)

Accurate Retraining-free Pruning for Pretrained Encoder-based Language Models
by: Park, Seungcheol, et al.
Published: (2023)

Factual Dialogue Summarization via Learning from Large Language Models
by: Zhu, Rongxin, et al.
Published: (2024)

Right for Right Reasons: Large Language Models for Verifiable Commonsense Knowledge Graph Question Answering
by: Toroghi, Armin, et al.
Published: (2024)

OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs
by: Wang, Yuxia, et al.
Published: (2024)

Quantifying Genuine Awareness in Hallucination Prediction Beyond Question-Side Shortcuts
by: Seo, Yeongbin, et al.
Published: (2025)

Uncovering Biases with Reflective Large Language Models
by: Chang, Edward Y.
Published: (2024)

Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
by: Seo, Yeongbin, et al.
Published: (2025)

CRISP: Persistent Concept Unlearning via Sparse Autoencoders
by: Ashuach, Tomer, et al.
Published: (2025)

Robust Uncertainty Quantification for Factual Generation of Large Language Models
by: Zhang, Yuhao, et al.
Published: (2026)

Text-Based Approaches to Item Difficulty Modeling in Large-Scale Assessments: A Systematic Review
by: Peters, Sydney, et al.
Published: (2025)

Precise Length Control in Large Language Models
by: Butcher, Bradley, et al.
Published: (2024)

Large Language Models for Biomedical Article Classification
by: Proboszcz, Jakub, et al.
Published: (2026)

Towards Fundamental Language Models: Does Linguistic Competence Scale with Model Size?
by: Collado-Montañez, Jaime, et al.
Published: (2025)

Prompt-Time Symbolic Knowledge Capture with Large Language Models
by: Çöplü, Tolga, et al.
Published: (2024)

Strategy Adaptation in Large Language Model Werewolf Agents
by: Nakamori, Fuya, et al.
Published: (2025)

Socially Responsible Data for Large Multilingual Language Models
by: Smart, Andrew, et al.
Published: (2024)

Large Language Models for Persian $ \leftrightarrow $ English Idiom Translation
by: Rezaeimanesh, Sara, et al.
Published: (2024)

Qomhra: A Bilingual Irish and English Large Language Model
by: McInerney, Joseph, et al.
Published: (2025)

Dialect Normalization using Large Language Models and Morphological Rules
by: Dimakis, Antonios, et al.
Published: (2025)

Towards Human Understanding of Paraphrase Types in Large Language Models
by: Meier, Dominik, et al.
Published: (2024)

RUQuant: Towards Refining Uniform Quantization for Large Language Models
by: Liu, Han, et al.
Published: (2026)

Does Localization Inform Unlearning? A Rigorous Examination of Local Parameter Attribution for Knowledge Unlearning in Language Models
by: Lee, Hwiyeong, et al.
Published: (2025)

Beyond Token Length: Step Pruner for Efficient and Accurate Reasoning in Large Language Models
by: Wu, Canhui, et al.
Published: (2025)

A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models
by: Peláez-González, Carlos, et al.
Published: (2025)

Personality, Role, and Expressive Style in Large Language Models: An Interactionist Analysis
by: Nagao, Moe, et al.
Published: (2026)

ConPET: Continual Parameter-Efficient Tuning for Large Language Models
by: Song, Chenyang, et al.
Published: (2023)

Aligning Large Language Models for Faithful Integrity Against Opposing Argument
by: Zhao, Yong, et al.
Published: (2025)

OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs
by: Iqbal, Hasan, et al.
Published: (2024)

LoRS: Efficient Low-Rank Adaptation for Sparse Large Language Model
by: Hu, Yuxuan, et al.
Published: (2025)

German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data
by: Klöser, Lars, et al.
Published: (2024)

LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models
by: Fuad, Kazi Ahmed Asif, et al.
Published: (2024)

Prompt-Time Ontology-Driven Symbolic Knowledge Capture with Large Language Models
by: Çöplü, Tolga, et al.
Published: (2024)

Extracting Probabilistic Knowledge from Large Language Models for Bayesian Network Parameterization
by: Nafar, Aliakbar, et al.
Published: (2025)

From Facts to Folklore: Evaluating Large Language Models on Bengali Cultural Knowledge
by: Chowdhury, Nafis, et al.
Published: (2025)

Mixup Model Merge: Enhancing Model Merging Performance through Randomized Linear Interpolation
by: Zhou, Yue, et al.
Published: (2025)

When Self-Reference Fails to Close: Matrix-Level Dynamics in Large Language Models
by: Bae, Ji Ho
Published: (2026)