Cover Image

Beyond Metrics: A Critical Analysis of the Variability in Large Language Model Evaluation Frameworks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Pimentel, Marco AF, Christophe, Clément, Raha, Tathagata, Munjal, Prateek, Kanithi, Praveen K, Khan, Shadab
Format:	Preprint
Published:	2024
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Med42-v2: A Suite of Clinical LLMs
by: Christophe, Clément, et al.
Published: (2024)

Beyond Fine-tuning: Unleashing the Potential of Continuous Pretraining for Clinical LLMs
by: Christophe, Clément, et al.
Published: (2024)

Bridging Language Barriers in Healthcare: A Study on Arabic LLMs
by: Saadi, Nada, et al.
Published: (2025)

MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applications
by: Kanithi, Praveenkumar, et al.
Published: (2024)

Overalignment in Frontier LLMs: An Empirical Study of Sycophantic Behaviour in Healthcare
by: Christophe, Clément, et al.
Published: (2026)

Named Clinical Entity Recognition Benchmark
by: Abdul, Wadood M, et al.
Published: (2024)

Cross-Examination Framework: A Task-Agnostic Diagnostic for Information Fidelity in Text-to-Text Generation
by: Raha, Tathagata, et al.
Published: (2026)

Building Trust in Clinical LLMs: Bias Analysis and Dataset Transparency
by: Maslenkova, Svetlana, et al.
Published: (2025)

Med42 -- Evaluating Fine-Tuning Strategies for Medical LLMs: Full-Parameter vs. Parameter-Efficient Approaches
by: Christophe, Clément, et al.
Published: (2024)

Do Instruction-Tuned Models Always Perform Better Than Base Models? Evidence from Math and Domain-Shifted Benchmarks
by: Munjal, Prateek, et al.
Published: (2026)

Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Models
by: Chhikara, Prateek
Published: (2025)

iREL at SemEval-2024 Task 9: Improving Conventional Prompting Methods for Brain Teasers
by: Gupta, Harshit, et al.
Published: (2024)

CriticEval: Evaluating Large Language Model as Critic
by: Lan, Tian, et al.
Published: (2024)

Wavelet GPT: Wavelet Inspired Large Language Models
by: Verma, Prateek
Published: (2024)

Whisper-GPT: A Hybrid Representation Audio Large Language Model
by: Verma, Prateek
Published: (2024)

Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
by: Askari, Raha, et al.
Published: (2025)

Model Provenance Testing for Large Language Models
by: Nikolic, Ivica, et al.
Published: (2025)

Large Language Models Are Active Critics in NLG Evaluation
by: Xu, Shuying, et al.
Published: (2024)

Investigating Critical Period Effects in Language Acquisition through Neural Language Models
by: Constantinescu, Ionut, et al.
Published: (2024)

Gene42: Long-Range Genomic Foundation Model With Dense Attention
by: Vishniakov, Kirill, et al.
Published: (2025)

Towards Signal Processing In Large Language Models
by: Verma, Prateek, et al.
Published: (2024)

Beyond Words: On Large Language Models Actionability in Mission-Critical Risk Analysis
by: Esposito, Matteo, et al.
Published: (2024)

Evaluating Large Language Models on Urdu Idiom Translation
by: Khan, Muhammad Farmal, et al.
Published: (2025)

Adaptive Large Language Models By Layerwise Attention Shortcuts
by: Verma, Prateek, et al.
Published: (2024)

Beyond Fixed: Training-Free Variable-Length Denoising for Diffusion Large Language Models
by: Li, Jinsong, et al.
Published: (2025)

Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
by: Lyu, Chenyang, et al.
Published: (2024)

Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations
by: Jalouzot, Louis, et al.
Published: (2024)

Can Large Language Models Infer Causal Relationships from Real-World Text?
by: Saklad, Ryan, et al.
Published: (2025)

Adversarial Text Purification: A Large Language Model Approach for Defense
by: Moraffah, Raha, et al.
Published: (2024)

Beyond One-Size-Fits-All Pruning via Evolutionary Metric Search for Large Language Models
by: Liu, Shuqi, et al.
Published: (2025)

A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations
by: Laskar, Md Tahmid Rahman, et al.
Published: (2024)

Strategic Innovation Management in the Age of Large Language Models Market Intelligence, Adaptive R&D, and Ethical Governance
by: Aghaei, Raha, et al.
Published: (2025)

Objective Metrics for Evaluating Large Language Models Using External Data Sources
by: Du, Haoze, et al.
Published: (2025)

FLAME: Financial Large-Language Model Assessment and Metrics Evaluation
by: Guo, Jiayu, et al.
Published: (2025)

Beyond Facts: Evaluating Intent Hallucination in Large Language Models
by: Hao, Yijie, et al.
Published: (2025)

Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages
by: Anand, Srija, et al.
Published: (2026)

Exploring Large Language Models in Healthcare: Insights into Corpora Sources, Customization Strategies, and Evaluation Metrics
by: Yang, Shuqi, et al.
Published: (2025)

Enhanced Electronic Health Records Text Summarization Using Large Language Models
by: Madzime, Ruvarashe, et al.
Published: (2024)

Large Language Models Implicitly Learn to See and Hear Just By Reading
by: Verma, Prateek, et al.
Published: (2025)

Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages
by: Yari, Amir Hossein, et al.
Published: (2025)