:: Library Catalog

Cover Image

Saved in:

Bibliographic Details
Main Author:	Yu, Fangyuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	https://arxiv.org/abs/2502.17910
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution
by: Dineen, Jacob, et al.
Published: (2026)

Bridging the Fairness Gap: Enhancing Pre-trained Models with LLM-Generated Sentences
by: Yu, Liu, et al.
Published: (2025)

Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training
by: Bergsma, Shane, et al.
Published: (2025)

Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
by: Tao, Chaofan, et al.
Published: (2024)

Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
by: Kang, Feiyang, et al.
Published: (2025)

Vocabulary embeddings organize linguistic structure early in language model training
by: Papadimitriou, Isabel, et al.
Published: (2025)

RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programs
by: Bi, Baolong, et al.
Published: (2025)

Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement
by: Ye, Liqin, et al.
Published: (2025)

Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies
by: Salhan, Suchir, et al.
Published: (2024)

Automated Detection of Pre-training Text in Black-box LLMs
by: Hu, Ruihan, et al.
Published: (2025)

Boosting Explainability through Selective Rationalization in Pre-trained Language Models
by: Yuan, Libing, et al.
Published: (2025)

Pre-Training Curriculum for Multi-Token Prediction in Language Models
by: Aynetdinov, Ansar, et al.
Published: (2025)

How to inject knowledge efficiently? Knowledge Infusion Scaling Law for Pre-training Large Language Models
by: Lv, Kangtao, et al.
Published: (2025)

Memorization-Compression Cycles Improve Generalization
by: Yu, Fangyuan
Published: (2025)

Crown, Frame, Reverse: Layer-Wise Scaling Variants for LLM Pre-Training
by: Baroian, Andrei, et al.
Published: (2025)

Adaptive Layer-skipping in Pre-trained LLMs
by: Luo, Xuan, et al.
Published: (2025)

Exploring the Benefit of Activation Sparsity in Pre-training
by: Zhang, Zhengyan, et al.
Published: (2024)

PreCog: Exploring the Relation between Memorization and Performance in Pre-trained Language Models
by: Ranaldi, Leonardo, et al.
Published: (2023)

Probing Language Models for Pre-training Data Detection
by: Liu, Zhenhua, et al.
Published: (2024)

Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scale
by: Zhou, Fan, et al.
Published: (2024)

Scaling Smart: Accelerating Large Language Model Pre-training with Small Model Initialization
by: Samragh, Mohammad, et al.
Published: (2024)

A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs
by: Shelmanov, Artem, et al.
Published: (2025)

NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data
by: Bogdanov, Sergei, et al.
Published: (2024)

Topic Over Source: The Key to Effective Data Mixing for Language Models Pre-training
by: Peng, Jiahui, et al.
Published: (2025)

DARC: Decoupled Asymmetric Reasoning Curriculum for LLM Evolution
by: Fan, Shengda, et al.
Published: (2026)

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
by: Qian, Chen, et al.
Published: (2024)

Can Pre-trained Language Models Understand Chinese Humor?
by: Chen, Yuyan, et al.
Published: (2024)

Pre-trained Large Language Models for Financial Sentiment Analysis
by: Luo, Wei, et al.
Published: (2024)

Spike No More: Stabilizing the Pre-training of Large Language Models
by: Takase, Sho, et al.
Published: (2023)

Parallel Structures in Pre-training Data Yield In-Context Learning
by: Chen, Yanda, et al.
Published: (2024)

Memory Reviving, Continuing Learning and Beyond: Evaluation of Pre-trained Encoders and Decoders for Multimodal Machine Translation
by: Yu, Zhuang, et al.
Published: (2025)

SAMGPT: Text-free Graph Foundation Model for Multi-domain Pre-training and Cross-domain Adaptation
by: Yu, Xingtong, et al.
Published: (2025)

EEG2TEXT: Open Vocabulary EEG-to-Text Decoding with EEG Pre-Training and Multi-View Transformer
by: Liu, Hanwen, et al.
Published: (2024)

Reasoning Curriculum: Bootstrapping Broad LLM Reasoning from Math
by: Pang, Bo, et al.
Published: (2025)

SSG: Logit-Balanced Vocabulary Partitioning for LLM Watermarking
by: Gu, Chenxi, et al.
Published: (2026)

Optimal Turkish Subword Strategies at Scale: Systematic Evaluation of Data, Vocabulary, Morphology Interplay
by: Altinok, Duygu
Published: (2026)

Rethinking the Chain-of-Thought: The Roles of In-Context Learning and Pre-trained Priors
by: Yang, Hao, et al.
Published: (2025)

DataMan: Data Manager for Pre-training Large Language Models
by: Peng, Ru, et al.
Published: (2025)

Pre-training LLMs using human-like development data corpus
by: Bhardwaj, Khushi, et al.
Published: (2023)

DocMamba: Efficient Document Pre-training with State Space Model
by: Hu, Pengfei, et al.
Published: (2024)