Cover Image

Beyond Public Access in LLM Pre-Training Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Rosenblat, Sruly, O'Reilly, Tim, Strauss, Ilan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The Attribution Crisis in LLM Search Results
by: Strauss, Ilan, et al.
Published: (2025)

Real-World Gaps in AI Governance Research
by: Strauss, Ilan, et al.
Published: (2025)

LLM-Supported Natural Language to Bash Translation
by: Westenfelder, Finnian, et al.
Published: (2025)

Disentangling concept semantics via multilingual averaging in Sparse Autoencoders
by: O'Reilly, Cliff, et al.
Published: (2025)

Experiments in News Bias Detection with Pre-Trained Neural Transformers
by: Menzner, Tim, et al.
Published: (2024)

Unifying Structured Data as Graph for Data-to-Text Pre-Training
by: Li, Shujie, et al.
Published: (2024)

Rethinking Reflection in Pre-Training
by: AI, Essential, et al.
Published: (2025)

Crown, Frame, Reverse: Layer-Wise Scaling Variants for LLM Pre-Training
by: Baroian, Andrei, et al.
Published: (2025)

Towards an automatic method for generating topical vocabulary test forms for specific reading passages
by: Flor, Michael, et al.
Published: (2025)

Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training
by: Jiang, Changhao, et al.
Published: (2025)

Reinforcement Learning on Pre-Training Data
by: Li, Siheng, et al.
Published: (2025)

Continual Pre-Training for Cross-Lingual LLM Adaptation: Enhancing Japanese Language Capabilities
by: Fujii, Kazuki, et al.
Published: (2024)

Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training
by: Du, Wenyu, et al.
Published: (2024)

"According to ...": Prompting Language Models Improves Quoting from Pre-Training Data
by: Weller, Orion, et al.
Published: (2023)

What Is The Political Content in LLMs' Pre- and Post-Training Data?
by: Ceron, Tanise, et al.
Published: (2025)

Investigating Gender Bias in LLM-Generated Stories via Psychological Stereotypes
by: Masoudian, Shahed, et al.
Published: (2025)

Bridging Linguistic Gaps: Cross-Lingual Mapping in Pre-Training and Dataset for Enhanced Multilingual LLM Performance
by: Zheng, Weihua, et al.
Published: (2026)

Beyond Perplexity: A Geometric and Spectral Study of Low-Rank Pre-Training
by: Shivagunde, Namrata, et al.
Published: (2026)

Unraveling Emotions with Pre-Trained Models
by: Pajón-Sanmartín, Alejandro, et al.
Published: (2025)

SlimPajama-DC: Understanding Data Combinations for LLM Training
by: Shen, Zhiqiang, et al.
Published: (2023)

Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication
by: Chung, Philip, et al.
Published: (2024)

Untie the Knots: An Efficient Data Augmentation Strategy for Long-Context Pre-Training in Language Models
by: Tian, Junfeng, et al.
Published: (2024)

ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training
by: Zhuo, Le, et al.
Published: (2024)

Evolution of Concepts in Language Model Pre-Training
by: Ge, Xuyang, et al.
Published: (2025)

Beyond Memorization: The Challenge of Random Memory Access in Language Models
by: Zhu, Tongyao, et al.
Published: (2024)

Scaling LLM Pre-training with Vocabulary Curriculum
by: Yu, Fangyuan
Published: (2025)

D$^3$: Dynamic Directional Graph-Constrained Data Scheduling for LLM Training
by: Xu, Yuanjian, et al.
Published: (2026)

Beyond Translation: LLM-Based Data Generation for Multilingual Fact-Checking
by: Chung, Yi-Ling, et al.
Published: (2025)

DEFT: Data Efficient Fine-Tuning for Pre-Trained Language Models via Unsupervised Core-Set Selection
by: Das, Devleena, et al.
Published: (2023)

Revealing the Inherent Instructability of Pre-Trained Language Models
by: An, Seokhyun, et al.
Published: (2024)

AutoScale: Scale-Aware Data Mixing for Pre-Training LLMs
by: Kang, Feiyang, et al.
Published: (2024)

Investigating Cost-Efficiency of LLM-Generated Training Data for Conversational Semantic Frame Analysis
by: Matta, Shiho, et al.
Published: (2024)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
by: Aynetdinov, Ansar, et al.
Published: (2025)

Pre-Trained Language Models for Keyphrase Prediction: A Review
by: Umair, Muhammad, et al.
Published: (2024)

Characterizing Stereotypical Bias from Privacy-preserving Pre-Training
by: Arnold, Stefan, et al.
Published: (2024)

Reparameterized LLM Training via Orthogonal Equivalence Transformation
by: Qiu, Zeju, et al.
Published: (2025)

LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation
by: Kim, Eunsu, et al.
Published: (2024)

AdapterSwap: Continuous Training of LLMs with Data Removal and Access-Control Guarantees
by: Fleshman, William, et al.
Published: (2024)

MTQE.en-he: Machine Translation Quality Estimation for English-Hebrew
by: Rosenbaum, Andy, et al.
Published: (2026)

Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance
by: Djuhera, Aladin, et al.
Published: (2025)