Copertina

Quantifying the Effect of Test Set Contamination on Generative Evaluations

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Schaeffer, Rylan, Kazdan, Joshua, Abbasi, Baber, Liu, Ken Ziyu, Miranda, Brando, Ahmed, Ahmed, Berez, Fazl, Puri, Abhay, Biderman, Stella, Mireshghallah, Niloofar, Koyejo, Sanmi
Natura:	Preprint
Pubblicazione:	2026
Soggetti:	Machine Learning Computation and Language
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

Understanding Adversarial Transfer: Why Representation-Space Attacks Fail Where Data-Space Attacks Succeed
di: Gupta, Isha, et al.
Pubblicazione: (2025)

Position: Model Collapse Does Not Mean What You Think
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Pretraining Scaling Laws for Generative Evaluations of Language Models
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

No, of Course I Can! Deeper Fine-Tuning Attacks That Bypass Token-Level Safety Mechanisms
di: Kazdan, Joshua, et al.
Pubblicazione: (2025)

Scale Dependent Data Duplication
di: Kazdan, Joshua, et al.
Pubblicazione: (2026)

Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Investigating Data Contamination for Pre-training Language Models
di: Jiang, Minhao, et al.
Pubblicazione: (2024)

In-Context Learning of Energy Functions
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Efficient Prediction of Pass@k Scaling in Large Language Models
di: Kazdan, Joshua, et al.
Pubblicazione: (2025)

Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
di: Kazdan, Joshua, et al.
Pubblicazione: (2024)

Consensus is Not Verification: Why Crowd Wisdom Strategies Fail for LLM Truthfulness
di: Denisov-Blanch, Yegor, et al.
Pubblicazione: (2026)

ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
di: Obbad, Elyas, et al.
Pubblicazione: (2024)

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data
di: Miranda, Brando, et al.
Pubblicazione: (2023)

Evaluating the Robustness of Chinchilla Compute-Optimal Scaling
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Min-p, Max Exaggeration: A Critical Analysis of Min-p Sampling in Language Models
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

The Utility and Complexity of in- and out-of-Distribution Machine Unlearning
di: Allouah, Youssef, et al.
Pubblicazione: (2024)

How Do Large Language Monkeys Get Their Power (Laws)?
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

Quantifying Variance in Evaluation Benchmarks
di: Madaan, Lovish, et al.
Pubblicazione: (2024)

Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track
di: Schaeffer, Rylan, et al.
Pubblicazione: (2025)

What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
di: Lecomte, Victor, et al.
Pubblicazione: (2023)

Quantifying the Importance of Data Alignment in Downstream Model Performance
di: Chawla, Krrish, et al.
Pubblicazione: (2025)

Best-of-N Jailbreaking
di: Hughes, John, et al.
Pubblicazione: (2024)

Pantograph: A Machine-to-Machine Interaction Interface for Advanced Theorem Proving, High Level Reasoning, and Data Extraction in Lean 4
di: Aniva, Leni, et al.
Pubblicazione: (2024)

Is Pre-training Truly Better Than Meta-Learning?
di: Miranda, Brando, et al.
Pubblicazione: (2023)

In-Situ Behavioral Evaluation for LLM Fairness, Not Standardized-Test Scores
di: Tang, Zeyu, et al.
Pubblicazione: (2026)

Chain-of-Thought Hijacking
di: Zhao, Jianli, et al.
Pubblicazione: (2025)

Extracting books from production language models
di: Ahmed, Ahmed, et al.
Pubblicazione: (2026)

Position: Privacy Is Not Just Memorization!
di: Mireshghallah, Niloofar, et al.
Pubblicazione: (2025)

Synthetic Data Can Mislead Evaluations: Membership Inference as Machine Text Detection
di: Naseh, Ali, et al.
Pubblicazione: (2025)

SpecEval: Evaluating Model Adherence to Behavior Specifications
di: Ahmed, Ahmed, et al.
Pubblicazione: (2025)

On Fairness of Low-Rank Adaptation of Large Models
di: Ding, Zhoujie, et al.
Pubblicazione: (2024)

Sharpe Ratio-Guided Active Learning for Preference Optimization in RLHF
di: Belakaria, Syrine, et al.
Pubblicazione: (2025)

Lean-ing on Quality: How High-Quality Data Beats Diverse Multilingual Data in AutoFormalization
di: Chan, Willy, et al.
Pubblicazione: (2025)

Scalable Ensembling For Mitigating Reward Overoptimisation
di: Ahmed, Ahmed M., et al.
Pubblicazione: (2024)

Causally Inspired Regularization Enables Domain General Representations
di: Salaudeen, Olawale, et al.
Pubblicazione: (2024)

Let's Measure Information Step-by-Step: AI-Based Evaluation Beyond Vibes
di: Robertson, Zachary, et al.
Pubblicazione: (2025)

CURE: Cultural Understanding and Reasoning Evaluation - A Framework for "Thick" Culture Alignment Evaluation in LLMs
di: Vo, Truong, et al.
Pubblicazione: (2025)

KGGen: Extracting Knowledge Graphs from Plain Text with Language Models
di: Mo, Belinda, et al.
Pubblicazione: (2025)

Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
di: Schaeffer, Rylan, et al.
Pubblicazione: (2024)

Putnam-AXIOM: A Functional and Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs
di: Gulati, Aryan, et al.
Pubblicazione: (2025)