Cover Image

Improving Model Evaluation using SMART Filtering of Benchmark Datasets

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Gupta, Vipul, Ross, Candace, Pantoja, David, Passonneau, Rebecca J., Ung, Megan, Williams, Adina
Format:	Preprint
Published:	2024
Subjects:	Computation and Language Artificial Intelligence Machine Learning
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Bias
by: Gupta, Vipul, et al.
Published: (2023)

Changing Answer Order Can Decrease MMLU Accuracy
by: Gupta, Vipul, et al.
Published: (2024)

Sociodemographic Bias in Language Models: A Survey and Forward Path
by: Gupta, Vipul, et al.
Published: (2023)

VerAs: Verify then Assess STEM Lab Reports
by: Atil, Berk, et al.
Published: (2024)

Benchmarking Cognitive Biases in Large Language Models as Evaluators
by: Koo, Ryan, et al.
Published: (2023)

SMART: Submodular Data Mixture Strategy for Instruction Tuning
by: Renduchintala, H S V N S Kowndinya, et al.
Published: (2024)

SMART: Self-Aware Agent for Tool Overuse Mitigation
by: Qian, Cheng, et al.
Published: (2025)

MORPHOGEN: A Multilingual Benchmark for Evaluating Gender-Aware Morphological Generation
by: Agarwal, Mehul, et al.
Published: (2026)

The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More
by: Kitouni, Ouail, et al.
Published: (2024)

CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
by: Brandfonbrener, David, et al.
Published: (2024)

SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees
by: Jo, Saehan, et al.
Published: (2024)

Paloma: A Benchmark for Evaluating Language Model Fit
by: Magnusson, Ian, et al.
Published: (2023)

Structured Prompts Improve Evaluation of Language Models
by: Aali, Asad, et al.
Published: (2025)

Cluster-based Adaptive Retrieval: Dynamic Context Selection for RAG Applications
by: Xu, Yifan, et al.
Published: (2025)

Joint Training for Selective Prediction
by: Li, Zhaohui, et al.
Published: (2024)

Fine-Grained Benchmark Generation for Comprehensive Evaluation of Foundation Models
by: Islam, Mohammed Saidul, et al.
Published: (2026)

Episodic Memories Generation and Evaluation Benchmark for Large Language Models
by: Huet, Alexis, et al.
Published: (2025)

How Contaminated Is Your Benchmark? Quantifying Dataset Leakage in Large Language Models with Kernel Divergence
by: Choi, Hyeong Kyu, et al.
Published: (2025)

VITAL: A New Dataset for Benchmarking Pluralistic Alignment in Healthcare
by: Shetty, Anudeex, et al.
Published: (2025)

Recovered in Translation: Efficient Pipeline for Automated Translation of Benchmarks and Datasets
by: Yukhymenko, Hanna, et al.
Published: (2026)

Attention Flows: Tracing LLM Conceptual Engagement via Story Summaries
by: Hicke, Rebecca M. M., et al.
Published: (2026)

Line Goes Up? Inherent Limitations of Benchmarks for Evaluating Large Language Models
by: Fodor, James
Published: (2025)

AssertBench: A Benchmark for Evaluating Self-Assertion in Large Language Models
by: Lee, Jaeho, et al.
Published: (2025)

ArenaBencher: Automatic Benchmark Evolution via Multi-Model Competitive Evaluation
by: Liu, Qin, et al.
Published: (2025)

Reliable Evaluation and Benchmarks for Statement Autoformalization
by: Poiroux, Auguste, et al.
Published: (2024)

Branch-Solve-Merge Improves Large Language Model Evaluation and Generation
by: Saha, Swarnadeep, et al.
Published: (2023)

An Evaluation Benchmark for Autoformalization in Lean4
by: Gulati, Aryan, et al.
Published: (2024)

Fluid Language Model Benchmarking
by: Hofmann, Valentin, et al.
Published: (2025)

Model Unlearning Objectives Vary for Distinct Language Functions
by: Atil, Berk, et al.
Published: (2026)

CTBench: A Comprehensive Benchmark for Evaluating Language Model Capabilities in Clinical Trial Design
by: Neehal, Nafis, et al.
Published: (2024)

AECBench: A Hierarchical Benchmark for Knowledge Evaluation of Large Language Models in the AEC Field
by: Liang, Chen, et al.
Published: (2025)

MLB: A Scenario-Driven Benchmark for Evaluating Large Language Models in Clinical Applications
by: He, Qing, et al.
Published: (2026)

Comprehensive Modeling and Question Answering of Cancer Clinical Practice Guidelines using LLMs
by: Gupta, Bhumika, et al.
Published: (2025)

Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Series
by: Chang, Ching, et al.
Published: (2025)

StyleRec: A Benchmark Dataset for Prompt Recovery in Writing Style Transformation
by: Liu, Shenyang, et al.
Published: (2025)

AfriStereo: A Culturally Grounded Dataset for Evaluating Stereotypical Bias in Large Language Models
by: Beux, Yann Le, et al.
Published: (2025)

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets
by: Indurthi, Sathish Reddy, et al.
Published: (2024)

BEExAI: Benchmark to Evaluate Explainable AI
by: Sithakoul, Samuel, et al.
Published: (2024)

ECG-Reasoning-Benchmark: A Benchmark for Evaluating Clinical Reasoning Capabilities in ECG Interpretation
by: Oh, Jungwoo, et al.
Published: (2026)

DrBenchmark: A Large Language Understanding Evaluation Benchmark for French Biomedical Domain
by: Labrak, Yanis, et al.
Published: (2024)