Cover Image

Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Liu, Lei, Zhu, Hao, Shen, Yue, Chu, Zhixuan, Wang, Jian, Gu, Jinjie, Ren, Kui
Format:	Preprint
Published:	2025
Subjects:	Machine Learning Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models
by: Ankner, Zachary, et al.
Published: (2024)

Rethinking Perplexity: Revealing the Impact of Input Length on Perplexity Evaluation in LLMs
by: Cheng, Letian, et al.
Published: (2026)

Perplexity Trap: PLM-Based Retrievers Overrate Low Perplexity Documents
by: Wang, Haoyu, et al.
Published: (2025)

Improving Pretraining Data Using Perplexity Correlations
by: Thrush, Tristan, et al.
Published: (2024)

Rethinking GSPO: The Perplexity-Entropy Equivalence
by: Liu, Chi
Published: (2025)

How Well Can Knowledge Edit Methods Edit Perplexing Knowledge?
by: Ge, Huaizhi, et al.
Published: (2024)

Mapping Overlaps in Benchmarks through Perplexity in the Wild
by: Wu, Siyang, et al.
Published: (2025)

Beyond Perplexity: A Geometric and Spectral Study of Low-Rank Pre-Training
by: Shivagunde, Namrata, et al.
Published: (2026)

What is Wrong with Perplexity for Long-context Language Modeling?
by: Fang, Lizhe, et al.
Published: (2024)

Demystifying Prompts in Language Models via Perplexity Estimation
by: Gonen, Hila, et al.
Published: (2022)

Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language Models
by: Bell, Tyler, et al.
Published: (2024)

Can Perplexity Predict Fine-tuning Performance? An Investigation of Tokenization Effects on Sequential Language Models for Nepali
by: Luitel, Nishant, et al.
Published: (2024)

Efficient Response Generation Strategy Selection for Fine-Tuning Large Language Models Through Self-Aligned Perplexity
by: Ren, Xuan, et al.
Published: (2025)

SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction
by: Dai, Lu, et al.
Published: (2025)

CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models
by: Gu, Jiawei, et al.
Published: (2024)

Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
by: Xu, Zhichao, et al.
Published: (2024)

Confidence, Not Perplexity: A Better Metric for the Creative Era of LLMs
by: Parupudi, V. S. Raghu
Published: (2025)

Momentum Point-Perplexity Mechanics in Large Language Models
by: Tomaz, Lorenzo, et al.
Published: (2025)

Perplexity Cannot Always Tell Right from Wrong
by: Veličković, Petar, et al.
Published: (2026)

Perplexity
by: Mahmoud, Abdalla
Published: (2026)

Perplexed
by: Tarek Zieneldien
Published: (2024)

PLPP: Prompt Learning with Perplexity Is Self-Distillation for Vision-Language Models
by: Liu, Biao, et al.
Published: (2024)

On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation
by: Hsu, Chan-Jan, et al.
Published: (2026)

Low-Perplexity LLM-Generated Sequences and Where To Find Them
by: Wuhrmann, Arthur, et al.
Published: (2025)

Beyond Perplexity: Let the Reader Select Retrieval Summaries via Spectrum Projection Score
by: Hu, Zhanghao, et al.
Published: (2025)

Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimization
by: Mavromatis, Costas, et al.
Published: (2024)

destroR: Attacking Transfer Models with Obfuscous Examples to Discard Perplexity
by: Ahmed, Saadat Rafid, et al.
Published: (2025)

Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
by: Seo, Yeongbin, et al.
Published: (2025)

Inconsistent Tokenizations Cause Language Models to be Perplexed by Japanese Grammar
by: Gambardella, Andrew, et al.
Published: (2025)

Contrastive Perplexity for Controlled Generation: An Application in Detoxifying Large Language Models
by: Klein, Tassilo, et al.
Published: (2024)

Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives
by: Baker, Mohammed Abu, et al.
Published: (2026)

When Perplexity Lies: Generation-Focused Distillation of Hybrid Sequence Models
by: Kostelec, Juan Gabriel, et al.
Published: (2026)

Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection
by: Narayanasamy, Priyadarshan, et al.
Published: (2026)

Mitigating Forgetting in LLM Fine-Tuning via Low-Perplexity Token Learning
by: Wu, Chao-Chung, et al.
Published: (2025)

Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?
by: Hu, Yutong, et al.
Published: (2024)

Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models
by: Cui, Yingqian, et al.
Published: (2025)

A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions
by: Liu, Lei, et al.
Published: (2024)

The Serial Perplex.
by: Blackwell, Maree Macon, et al.
Published: (1978)

Beyond Perplexity: A Lightweight Benchmark for Knowledge Retention in Supervised Fine-Tuning
by: Shabgahi, Soheil Zibakhsh, et al.
Published: (2026)

Token-Level Adversarial Prompt Detection Based on Perplexity Measures and Contextual Information
by: Hu, Zhengmian, et al.
Published: (2023)