Copertina

GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Zhang, Jifan, Luo, Ziyue, Liu, Jia, Shroff, Ness, Nowak, Robert
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Prediction-Assisted Online Distributed Deep Learning Workload Scheduling in GPU Clusters
di: Luo, Ziyue, et al.
Pubblicazione: (2025)

Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models
di: Ali, Mehdi, et al.
Pubblicazione: (2025)

Constraint-Rectified Training for Efficient Chain-of-Thought
di: Wu, Qinhang, et al.
Pubblicazione: (2026)

FIRM: Federated In-client Regularized Multi-objective Alignment for Large Language Models
di: Nourzad, Fatemeh, et al.
Pubblicazione: (2025)

Can We Theoretically Quantify the Impacts of Local Updates on the Generalization Performance of Federated Learning?
di: Ju, Peizhong, et al.
Pubblicazione: (2024)

Large Language Models Achieve Gold Medal Performance at the International Olympiad on Astronomy & Astrophysics (IOAA)
di: Pinheiro, Lucas Carrit Delgado, et al.
Pubblicazione: (2025)

From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models
di: Liang, Yuchen, et al.
Pubblicazione: (2026)

Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach
di: Liang, Yuchen, et al.
Pubblicazione: (2024)

AHA: Human-Assisted Out-of-Distribution Generalization and Detection
di: Bai, Haoyue, et al.
Pubblicazione: (2024)

Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations
di: Matton, Katie, et al.
Pubblicazione: (2025)

Benchmarking LLMs' Judgments with No Gold Standard
di: Xu, Shengwei, et al.
Pubblicazione: (2024)

Detecting Pretraining Data from Large Language Models
di: Shi, Weijia, et al.
Pubblicazione: (2023)

LLMs as Scalable, General-Purpose Simulators For Evolving Digital Agent Training
di: Wang, Yiming, et al.
Pubblicazione: (2025)

MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
di: Chen, Zhixun, et al.
Pubblicazione: (2025)

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining
di: Saada, Thiziri Nait, et al.
Pubblicazione: (2025)

Pretraining Large Language Models with NVFP4
di: NVIDIA, et al.
Pubblicazione: (2025)

Generalization v.s. Memorization: Tracing Language Models' Capabilities Back to Pretraining Data
di: Wang, Xinyi, et al.
Pubblicazione: (2024)

Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dual
di: Li, Yining, et al.
Pubblicazione: (2026)

SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors
di: Lyu, Bohan, et al.
Pubblicazione: (2025)

Sharp Convergence Rates for Masked Diffusion Models
di: Liang, Yuchen, et al.
Pubblicazione: (2026)

Procedural Pretraining: Warming Up Language Models with Abstract Data
di: Jiang, Liangze, et al.
Pubblicazione: (2026)

Analyzing Similarity Metrics for Data Selection for Language Model Pretraining
di: Sam, Dylan, et al.
Pubblicazione: (2025)

Using Hallucinations to Bypass GPT4's Filter
di: Lemkin, Benjamin
Pubblicazione: (2024)

An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models
di: Bhatt, Gantavya, et al.
Pubblicazione: (2024)

Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers
di: Liang, Yuchen, et al.
Pubblicazione: (2024)

On The Role of Pretrained Language Models in General-Purpose Text Embeddings: A Survey
di: Zhang, Meishan, et al.
Pubblicazione: (2025)

Revisiting Multilingual Data Mixtures in Language Model Pretraining
di: Foroutan, Negar, et al.
Pubblicazione: (2025)

Facts in Stats: Impacts of Pretraining Diversity on Language Model Generalization
di: Behnia, Tina, et al.
Pubblicazione: (2025)

The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
di: Bhaskar, Adithya, et al.
Pubblicazione: (2024)

Data Mixing for Large Language Models Pretraining: A Survey and Outlook
di: Chen, Zhuo, et al.
Pubblicazione: (2026)

Language Models Improve When Pretraining Data Matches Target Tasks
di: Mizrahi, David, et al.
Pubblicazione: (2025)

Temporal Entailment Pretraining for Clinical Language Models over EHR Data
di: Tanaka, Tatsunori, et al.
Pubblicazione: (2025)

Discrete Diffusion Models: Novel Analysis and New Sampler Guarantees
di: Liang, Yuchen, et al.
Pubblicazione: (2025)

Can General-Purpose Large Language Models Generalize to English-Thai Machine Translation ?
di: Chiaranaipanich, Jirat, et al.
Pubblicazione: (2024)

Causal Reasoning and Large Language Models: Opening a New Frontier for Causality
di: Kıcıman, Emre, et al.
Pubblicazione: (2023)

Measuring and Reducing LLM Hallucination without Gold-Standard Answers
di: Wei, Jiaheng, et al.
Pubblicazione: (2024)

How to Find the Exact Pareto Front for Multi-Objective MDPs?
di: Li, Yining, et al.
Pubblicazione: (2024)

Provably Efficient Multi-Objective Bandit Algorithms under Preference-Centric Customization
di: Cao, Linfeng, et al.
Pubblicazione: (2025)

On Training Data Influence of GPT Models
di: Chai, Yekun, et al.
Pubblicazione: (2024)

SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
di: Puvvada, Krishna C., et al.
Pubblicazione: (2025)