Image de couverture de livre

Training on the Test Task Confounds Evaluation and Emergence

Fuente: arXiv

Enregistré dans:

Détails bibliographiques
Auteurs principaux:	Dominguez-Olmedo, Ricardo, Dorner, Florian E., Hardt, Moritz
Format:	Preprint
Publié:	2024
Sujets:	Computation and Language Artificial Intelligence Machine Learning
Accès en ligne:	Acceder al recurso
Tags:	Ajouter un tag Pas de tags, Soyez le premier à ajouter un tag!

Documents similaires

Train-before-Test Harmonizes Language Model Rankings
par: Zhang, Guanhua, et autres
Publié: (2025)

Lawma: The Power of Specialization for Legal Annotation
par: Dominguez-Olmedo, Ricardo, et autres
Publié: (2024)

Answer Matching Outperforms Multiple Choice for Language Model Evaluation
par: Chandak, Nikhil, et autres
Publié: (2025)

Computational Arbitrage in AI Model Markets
par: Olmedo, Ricardo, et autres
Publié: (2026)

Test-Time Training on Nearest Neighbors for Large Language Models
par: Hardt, Moritz, et autres
Publié: (2023)

FutureSim: Replaying World Events to Evaluate Adaptive Agents
par: Goel, Shashwat, et autres
Publié: (2026)

Whose Preferences? Differences in Fairness Preferences and Their Impact on the Fairness of AI Utilizing Human Feedback
par: Lerner, Emilia Agis, et autres
Publié: (2024)

Foundational Automatic Evaluators: Scaling Multi-Task Generative Evaluator Training for Reasoning-Centric Domains
par: Xu, Austin, et autres
Publié: (2025)

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder Decoder Perspective
par: Han, Seungwook, et autres
Publié: (2024)

Probing the Emergence of Cross-lingual Alignment during LLM Training
par: Wang, Hetong, et autres
Publié: (2024)

In-Place Test-Time Training
par: Feng, Guhao, et autres
Publié: (2026)

Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences
par: Ghoshal, Asish, et autres
Publié: (2022)

Self-Trained Verification for Training- and Test-Time Self-Improvement
par: Wu, Chen Henry, et autres
Publié: (2026)

Don't Label Twice: Quantity Beats Quality when Comparing Binary Classifiers on a Budget
par: Dorner, Florian E., et autres
Publié: (2024)

XL-Suite: Cross-Lingual Synthetic Training and Evaluation Data for Open-Ended Generation
par: Iyer, Vivek, et autres
Publié: (2025)

The Unreasonable Effectiveness of Easy Training Data for Hard Tasks
par: Hase, Peter, et autres
Publié: (2024)

Emergence of Episodic Memory in Transformers: Characterizing Changes in Temporal Structure of Attention Scores During Training
par: Mistry, Deven Mahesh, et autres
Publié: (2025)

Test-Time Detoxification without Training or Learning Anything
par: Saglam, Baturay, et autres
Publié: (2026)

The Surprising Effectiveness of Test-Time Training for Few-Shot Learning
par: Akyürek, Ekin, et autres
Publié: (2024)

SR-TTT: Surprisal-Aware Residual Test-Time Training
par: P, Swamynathan V
Publié: (2026)

Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
par: Xu, Xiaoyue, et autres
Publié: (2024)

Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet
par: Zhao, James Xu, et autres
Publié: (2025)

Evaluating Language-Model Agents on Realistic Autonomous Tasks
par: Kinniment, Megan, et autres
Publié: (2023)

Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards
par: Padula, Alexander G., et autres
Publié: (2024)

Query-Conditioned Test-Time Self-Training for Large Language Models
par: Song, Chaehee, et autres
Publié: (2026)

MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
par: von Oswald, Johannes, et autres
Publié: (2025)

A Dataset for Evaluating LLM-based Evaluation Functions for Research Question Extraction Task
par: Fujisaki, Yuya, et autres
Publié: (2024)

Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Models
par: Zhou, Chenxi, et autres
Publié: (2025)

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task
par: Li, Kenneth, et autres
Publié: (2022)

CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
par: Feng, Jie, et autres
Publié: (2024)

Visualizing Uncertainty in Translation Tasks: An Evaluation of LLM Performance and Confidence Metrics
par: Park, Jin Hyun, et autres
Publié: (2025)

DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks
par: Zhu, Kaijie, et autres
Publié: (2023)

Enhancing LLM Reasoning via Critique Models with Test-Time and Training-Time Supervision
par: Xi, Zhiheng, et autres
Publié: (2024)

Becoming Experienced Judges: Selective Test-Time Learning for Evaluators
par: Jwa, Seungyeon, et autres
Publié: (2025)

Learning from Relevant Subgoals in Successful Dialogs using Iterative Training for Task-oriented Dialog Systems
par: Kaiser, Magdalena, et autres
Publié: (2024)

Challenging the Validity of Personality Tests for Large Language Models
par: Sühr, Tom, et autres
Publié: (2023)

Training and Evaluating Language Models with Template-based Data Generation
par: Zhang, Yifan
Publié: (2024)

Culinary Class Wars: Evaluating LLMs using ASH in Cuisine Transfer Task
par: Lee, Hoonick, et autres
Publié: (2024)

TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
par: Lin, Weizhe, et autres
Publié: (2025)

Universal Response and Emergence of Induction in LLMs
par: Luick, Niclas
Publié: (2024)