Image de couverture de livre

Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Roque, Matthew Theodore, Velasco, Dan John
Format:	Preprint
Publié:	2025
Sujets:	Computation and Language Artificial Intelligence
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text
par: Velasco, Dan John, et autres
Publié: (2025)

Rethinking the Role of Text Complexity in Language Model Pretraining
par: Velasco, Dan John, et autres
Publié: (2025)

Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning
par: Zhang, Yang, et autres
Publié: (2025)

Edit-Constrained Decoding for Sentence Simplification
par: Zetsu, Tatsuya, et autres
Publié: (2024)

Large Language Models for Biomedical Text Simplification: Promising But Not There Yet
par: Li, Zihao, et autres
Publié: (2024)

Preference Curriculum: LLMs Should Always Be Pretrained on Their Preferred Data
par: Zhang, Xuemiao, et autres
Publié: (2025)

Difficulty Estimation and Simplification of French Text Using LLMs
par: Jamet, Henri, et autres
Publié: (2024)

MuTSE: A Human-in-the-Loop Multi-use Text Simplification Evaluator
par: Roscan, Rares-Alexandru, et autres
Publié: (2026)

MultiLS: A Multi-task Lexical Simplification Framework
par: North, Kai, et autres
Publié: (2024)

How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
par: Luo, Kairong, et autres
Publié: (2025)

A Knowledge-Injected Curriculum Pretraining Framework for Question Answering
par: Lin, Xin, et autres
Publié: (2024)

Text and Audio Simplification: Human vs. ChatGPT
par: Leroy, Gondy, et autres
Publié: (2024)

Automated Feedback Loops to Protect Text Simplification with Generative AI from Information Loss
par: Nandiraju, Abhay Kumara Sri Krishna, et autres
Publié: (2025)

Large Language Models as Quasi-crystals: Coherence Without Repetition in Generative Text
par: Guevara-Vela, Jose Manuel
Publié: (2025)

SimplifyMyText: An LLM-Based System for Inclusive Plain Language Text Simplification
par: Färber, Michael, et autres
Publié: (2025)

Health Text Simplification: An Annotated Corpus for Digestive Cancer Education and Novel Strategies for Reinforcement Learning
par: Rahman, Md Mushfiqur, et autres
Publié: (2024)

Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilities
par: Gao, Yingqiang, et autres
Publié: (2025)

Fighting Against the Repetitive Training and Sample Dependency Problem in Few-shot Named Entity Recognition
par: Tian, Chang, et autres
Publié: (2024)

Beyond Line-Level Filtering for the Pretraining Corpora of LLMs
par: Park, Chanwoo, et autres
Publié: (2025)

TACL: Threshold-Adaptive Curriculum Learning Strategy for Enhancing Medical Text Understanding
par: Ren, Mucheng, et autres
Publié: (2025)

A Severity-Based Curriculum Learning Strategy for Arabic Medical Text Generation
par: Alansary, Ahmed, et autres
Publié: (2026)

Contrastive Token Learning with Similarity Decay for Repetition Suppression in Machine Translation
par: Dai, Huangyu, et autres
Publié: (2024)

A Neural Model for Word Repetition
par: Dager, Daniel, et autres
Publié: (2025)

Repetition over Diversity: High-Signal Data Filtering for Sample-Efficient German Language Modeling
par: Aynetdinov, Ansar, et autres
Publié: (2026)

Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL
par: Wang, Yihan, et autres
Publié: (2026)

The Overlooked Repetitive Lengthening Form in Sentiment Analysis
par: Wang, Lei, et autres
Publié: (2026)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web
par: Yan, Yibo, et autres
Publié: (2023)

APIO: Automatic Prompt Induction and Optimization for Grammatical Error Correction and Text Simplification
par: Chernodub, Artem, et autres
Publié: (2025)

Aligning Sentence Simplification with ESL Learner's Proficiency for Language Acquisition
par: Li, Guanlin, et autres
Publié: (2025)

ALEXSIS-PT: A New Resource for Portuguese Lexical Simplification
par: North, Kai, et autres
Publié: (2022)

Harnessing the Intrinsic Knowledge of Pretrained Language Models for Challenging Text Classification Settings
par: Gao, Lingyu
Publié: (2024)

Do Repetitions Matter? Strengthening Reliability in LLM Evaluations
par: Gonzalez, Miguel Angel Alvarado, et autres
Publié: (2025)

Dynamic Masking Rate Schedules for MLM Pretraining
par: Ankner, Zachary, et autres
Publié: (2023)

Curriculum Learning with Quality-Driven Data Selection
par: Wu, Biao, et autres
Publié: (2024)

On Linear Representations and Pretraining Data Frequency in Language Models
par: Merullo, Jack, et autres
Publié: (2025)

Optimizing Pretraining Data Mixtures with LLM-Estimated Utility
par: Held, William, et autres
Publié: (2025)

Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learning
par: Kim, Jisu, et autres
Publié: (2024)

Learning Dynamics of Meta-Learning in Small Model Pretraining
par: Africa, David Demitri, et autres
Publié: (2025)

Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
par: Cen, Zhepeng, et autres
Publié: (2025)

Looks can be Deceptive: Distinguishing Repetition Disfluency from Reduplication
par: Ahmad, Arif, et autres
Publié: (2024)