Image de couverture de livre

GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Zhang, Jifan, Luo, Ziyue, Liu, Jia, Shroff, Ness, Nowak, Robert
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Prediction-Assisted Online Distributed Deep Learning Workload Scheduling in GPU Clusters
par: Luo, Ziyue, et autres
Publié: (2025)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
par: Ali, Mehdi, et autres
Publié: (2025)

Constraint-Rectified Training for Efficient Chain-of-Thought
par: Wu, Qinhang, et autres
Publié: (2026)

FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Models
par: Nourzad, Fatemeh, et autres
Publié: (2025)

Benchmarking LLMs' Judgments with No Gold Standard
par: Xu, Shengwei, et autres
Publié: (2024)

Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning?
par: Ju, Peizhong, et autres
Publié: (2024)

From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models
par: Liang, Yuchen, et autres
Publié: (2026)

AHA: Human-Assisted Out-of-Distribution Generalization and Detection
par: Bai, Haoyue, et autres
Publié: (2024)

Large Language Models Achieve Gold Medal Performance at the International Olympiad on Astronomy & Astrophysics (IOAA)
par: Pinheiro, Lucas Carrit Delgado, et autres
Publié: (2025)

Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach
par: Liang, Yuchen, et autres
Publié: (2024)

Detecting Pretraining Data from Large Language Models
par: Shi, Weijia, et autres
Publié: (2023)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
par: Saada, Thiziri Nait, et autres
Publié: (2025)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
par: Chen, Zhixun, et autres
Publié: (2025)

Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
par: Matton, Katie, et autres
Publié: (2025)

LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
par: Wang, Yiming, et autres
Publié: (2025)

Procedural Pretraining: Warming Up Language Models with Abstract Data
par: Jiang, Liangze, et autres
Publié: (2026)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
par: Sam, Dylan, et autres
Publié: (2025)

Pretraining Large Language Models with NVFP4
par: NVIDIA, et autres
Publié: (2025)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
par: Wang, Xinyi, et autres
Publié: (2024)

SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors
par: Lyu, Bohan, et autres
Publié: (2025)

On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
par: Zhang, Meishan, et autres
Publié: (2025)

Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
par: Behnia, Tina, et autres
Publié: (2025)

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
par: Bhaskar, Adithya, et autres
Publié: (2024)

Data Mixing for Large Language Models Pretraining: A Survey and Outlook
par: Chen, Zhuo, et autres
Publié: (2026)

Language Models Improve When Pretraining Data Matches Target Tasks
par: Mizrahi, David, et autres
Publié: (2025)

Temporal Entailment Pretraining for Clinical Language Models over EHR Data
par: Tanaka, Tatsunori, et autres
Publié: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
par: Foroutan, Negar, et autres
Publié: (2025)

Using Hallucinations to Bypass GPT4's Filter
par: Lemkin, Benjamin
Publié: (2024)

On Training Data Influence of GPT Models
par: Chai, Yekun, et autres
Publié: (2024)

SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
par: Puvvada, Krishna C., et autres
Publié: (2025)

Can General-Purpose Large Language Models Generalize to English-Thai Machine Translation ?
par: Chiaranaipanich, Jirat, et autres
Publié: (2024)

Chameleon: A Flexible Data-mixing Framework for Language Model Pretraining and Finetuning
par: Xie, Wanyun, et autres
Publié: (2025)

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine
par: Zhang, Kai, et autres
Publié: (2026)

Measuring and Reducing LLM Hallucination without Gold-Standard Answers
par: Wei, Jiaheng, et autres
Publié: (2024)

Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models
par: Ruis, Laura, et autres
Publié: (2024)

Zero-to-Strong Generalization: Eliciting Strong Capabilities of Large Language Models Iteratively without Gold Labels
par: Liu, Chaoqun, et autres
Publié: (2024)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
par: Bhatt, Gantavya, et autres
Publié: (2024)

Sharp Convergence Rates for Masked Diffusion Models
par: Liang, Yuchen, et autres
Publié: (2026)

Optimizing Large Language Model Training Using FP4 Quantization
par: Wang, Ruizhe, et autres
Publié: (2025)

Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dual
par: Li, Yining, et autres
Publié: (2026)