Cover Image

Cost-of-Pass: An Economic Framework for Evaluating Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Erol, Mehmet Hamza, El, Batu, Suzgun, Mirac, Yuksekgonul, Mert, Zou, James
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding
by: Suzgun, Mirac, et al.
Published: (2024)

Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory
by: Suzgun, Mirac, et al.
Published: (2025)

Inefficiencies of Meta Agents for Agent Design
by: El, Batu, et al.
Published: (2025)

Do Language Models Know When They're Hallucinating References?
by: Agrawal, Ayush, et al.
Published: (2023)

Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
by: Dahl, Matthew, et al.
Published: (2024)

Belief in the Machine: Investigating Epistemological Blind Spots of Language Models
by: Suzgun, Mirac, et al.
Published: (2024)

Sparse Reward Subsystem in Large Language Models
by: Xu, Guowei, et al.
Published: (2026)

Diversity of Thought Improves Reasoning Abilities of LLMs
by: Naik, Ranjita, et al.
Published: (2023)

How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis
by: Bianchi, Federico, et al.
Published: (2024)

TextGrad: Automatic "Differentiation" via Text
by: Yuksekgonul, Mert, et al.
Published: (2024)

Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models
by: Yuksekgonul, Mert, et al.
Published: (2023)

Moloch's Bargain: Emergent Misalignment When LLMs Compete for Audiences
by: El, Batu, et al.
Published: (2025)

Free Lunch for Pass@$k$? Low Cost Diverse Sampling for Diffusion Language Models
by: Lamont, Sean, et al.
Published: (2026)

Can LLM feedback enhance review quality? A randomized study of 20K reviews at ICLR 2025
by: Thakkar, Nitya, et al.
Published: (2025)

Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions
by: Bianchi, Federico, et al.
Published: (2023)

SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning
by: Zhao, Wanjia, et al.
Published: (2025)

metaTextGrad: Automatically optimizing language model optimizers
by: Xu, Guowei, et al.
Published: (2025)

Don't Pass@k: A Bayesian Framework for Large Language Model Evaluation
by: Hariri, Mohsen, et al.
Published: (2025)

Evaluating Semantic and Syntactic Understanding in Large Language Models for Payroll Systems
by: Maclean, Hendrika, et al.
Published: (2026)

SiLVERScore: Semantically-Aware Embeddings for Sign Language Generation Evaluation
by: Imai, Saki, et al.
Published: (2025)

Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruption
by: Erol, Mehmet Kaan
Published: (2026)

A Multi-Domain Red Teaming Framework for Safety, Robustness, and Fairness Evaluation of Medical Large Language Models
by: Feier, Andrei Marian, et al.
Published: (2026)

Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content
by: Bianchi, Federico, et al.
Published: (2024)

Role-Playing Evaluation for Large Language Models
by: Boudouri, Yassine El, et al.
Published: (2025)

Luna: An Evaluation Foundation Model to Catch Language Model Hallucinations with High Accuracy and Low Cost
by: Belyi, Masha, et al.
Published: (2024)

Evaluating Commercial AI Chatbots as News Intermediaries
by: Suzgun, Mirac, et al.
Published: (2026)

A Benchmark for Learning to Translate a New Language from One Grammar Book
by: Tanzer, Garrett, et al.
Published: (2023)

Cleansing the Artificial Mind: A Self-Reflective Detoxification Framework for Large Language Models
by: Zhang, Kaituo, et al.
Published: (2026)

Accounting for Sycophancy in Language Model Uncertainty Estimation
by: Sicilia, Anthony, et al.
Published: (2024)

Leveraging Computerized Adaptive Testing for Cost-effective Evaluation of Large Language Models in Medical Benchmarking
by: Zheng, Tianpeng, et al.
Published: (2026)

SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding
by: Zhang, Yazhou, et al.
Published: (2024)

FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination
by: Zhou, Pengfei, et al.
Published: (2024)

Math Neurosurgery: Isolating Language Models' Math Reasoning Abilities Using Only Forward Passes
by: Christ, Bryan R., et al.
Published: (2024)

MirrorStories: Reflecting Diversity through Personalized Narrative Generation with Large Language Models
by: Yunusov, Sarfaroz, et al.
Published: (2024)

A System for Name and Address Parsing with Large Language Models
by: Tarannum, Adeeba, et al.
Published: (2026)

Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers
by: Kleidermacher, Hannah Calzi, et al.
Published: (2025)

Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs
by: Cherepanova, Valeriia, et al.
Published: (2024)

ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models
by: Elangovan, Aparna, et al.
Published: (2024)

CurLL: A Developmental Framework to Evaluate Continual Learning in Language Models
by: Kalyan, Pavan, et al.
Published: (2025)

CreativityPrism: A Holistic Evaluation Framework for Large Language Model Creativity
by: Hou, Zhaoyi Joey, et al.
Published: (2025)