Cover Image

Ultra-FineWeb: Efficient Data Filtering and Verification for High-Quality LLM Training Data

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Wang, Yudong, Fu, Zixuan, Cai, Jie, Tang, Peijun, Lyu, Hongya, Fang, Yewei, Zheng, Zhi, Zhou, Jie, Zeng, Guoyang, Xiao, Chaojun, Han, Xu, Liu, Zhiyuan
Format:	Preprint
Published:	2025
Subjects:	Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
by: Penedo, Guilherme, et al.
Published: (2024)

FineWeb-zhtw: Scalable Curation of Traditional Chinese Text Data from the Web
by: Lin, Cheng-Wei, et al.
Published: (2024)

Data Science and Technology Towards AGI Part I: Tiered Data Management
by: Wang, Yudong, et al.
Published: (2026)

FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
by: Penedo, Guilherme, et al.
Published: (2025)

Densing Law of LLMs
by: Xiao, Chaojun, et al.
Published: (2024)

DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuning
by: Zhang, Junbo, et al.
Published: (2026)

UltraEval-Audio: A Unified Framework for Comprehensive Evaluation of Audio Foundation Models
by: Shi, Qundong, et al.
Published: (2026)

NOSA: Native and Offloadable Sparse Attention
by: Huang, Yuxiang, et al.
Published: (2025)

Training Data Provenance Verification: Did Your Model Use Synthetic Data from My Generative Model for Training?
by: Xie, Yuechen, et al.
Published: (2025)

Leveraging Web-Crawled Data for High-Quality Fine-Tuning
by: Zhou, Jing, et al.
Published: (2024)

AutoPureData: Automated Filtering of Undesirable Web Data to Update LLM Knowledge
by: Vadlapati, Praneeth
Published: (2024)

PyBench: Evaluating LLM Agent on various real-world coding tasks
by: Zhang, Yaolun, et al.
Published: (2024)

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
by: Huang, Yuxiang, et al.
Published: (2024)

Data Verification is the Future of Quantum Computing Copilots
by: Song, Junhao, et al.
Published: (2026)

CatMemo at the FinLLM Challenge Task: Fine-Tuning Large Language Models using Data Fusion in Financial Applications
by: Cao, Yupeng, et al.
Published: (2024)

FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering
by: Henriksson, Erik, et al.
Published: (2025)

Statistical-Geometric Degeneracy in UAV Search: A Physics-Aware Asymmetric Filtering Approach
by: Ren, Zhiyuan, et al.
Published: (2026)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
by: Saada, Thiziri Nait, et al.
Published: (2025)

MEMO: Fine-grained Tensor Management For Ultra-long Context LLM Training
by: Zhao, Pinxue, et al.
Published: (2024)

MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
by: Hu, Shengding, et al.
Published: (2024)

Filter-then-Weight: Online Data Selection and Reweighting for LLM Fine-Tuning
by: Wang, Fangxin, et al.
Published: (2026)

Improving Romanian LLM Pretraining Data using Diversity and Quality Filtering
by: Negoita, Vlad, et al.
Published: (2025)

Analysis of an age‐space structured tuberculosis model with treatment and relapse
by: Jinliang Wang, et al.
Published: (2024)

MiniCPM4: Ultra-Efficient LLMs on End Devices
by: MiniCPM Team, et al.
Published: (2025)

Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding
by: Zhao, Weilin, et al.
Published: (2024)

Quotation-Based Data Retention Mechanism for Data Privacy in LLM-Empowered Network Services
by: Han, Bin, et al.
Published: (2025)

The Long-Term Effects of Data Selection in LLM Fine-Tuning
by: Yang, Yuxin, et al.
Published: (2026)

SmolKalam: Ensemble Quality-Filtered Translation at Scale for High Quality Arabic Post-Training Data
by: Alrashed, Sultan, et al.
Published: (2025)

Scaling Web Agent Training through Automatic Data Generation and Fine-grained Evaluation
by: Logeswaran, Lajanugen, et al.
Published: (2026)

Does Training with Synthetic Data Truly Protect Privacy?
by: Zhao, Yunpeng, et al.
Published: (2025)

Kernel-based Method for Detecting Structural Break in Distribution of Functional Data
by: Sang, Peijun, et al.
Published: (2025)

Restricted Tweedie stochastic block models
by: Jie Jian, et al.
Published: (2025)

Learning Shared and Source-specific Subspaces across Multiple Data Sources for Functional Data
by: Zhang, Chi, et al.
Published: (2025)

InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by: Xiao, Chaojun, et al.
Published: (2024)

Massive Supervised Fine-tuning Experiments Reveal How Data, Layer, and Training Factors Shape LLM Alignment Quality
by: Harada, Yuto, et al.
Published: (2025)

Order Determination for Functional Data
by: Zhang, Chi, et al.
Published: (2025)

Correcting Auto-Differentiation in Neural-ODE Training
by: Xu, Yewei, et al.
Published: (2023)

Towards Reliable Verification of Unauthorized Data Usage in Personalized Text-to-Image Diffusion Models
by: Li, Boheng, et al.
Published: (2024)

Imbalanced Data Clustering using Equilibrium K-Means
by: He, Yudong
Published: (2024)

Bootstrap Nonparametric Inference under Data Integration
by: Shang, Zuofeng, et al.
Published: (2025)