Cover Image

LLM-Evaluation Tropes: Perspectives on the Validity of LLM-Evaluations

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Dietz, Laura, Zendel, Oleg, Bailey, Peter, Clarke, Charles, Cotterill, Ellese, Dalton, Jeff, Hasibi, Faegheh, Sanderson, Mark, Craswell, Nick
Format:	Preprint
Published:	2025
Subjects:	Information Retrieval H.3.3
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

Criteria-Based LLM Relevance Judgments
by: Farzi, Naghmeh, et al.
Published: (2025)

Best in Tau@LLMJudge: Criteria-Based Relevance Evaluation with Llama3
by: Farzi, Naghmeh, et al.
Published: (2024)

PUB: An LLM-Enhanced Personality-Driven User Behaviour Simulator for Recommender System Evaluation
by: Ma, Chenglong, et al.
Published: (2025)

Does UMBRELA Work on Other LLMs?
by: Farzi, Naghmeh, et al.
Published: (2025)

Diagnosing LLM-based Rerankers in Cold-Start Recommender Systems: Coverage, Exposure and Practical Mitigations
by: Lemdiasova, Ekaterina, et al.
Published: (2026)

LLM-based relevance assessment still can't replace human relevance assessment
by: Clarke, Charles L. A., et al.
Published: (2024)

MIRA: An LLM-Assisted Benchmark for Multi-Category Integrated Retrieval
by: Türkmen, Mehmet Deniz, et al.
Published: (2026)

LLM-based Query Expansion Fails for Unfamiliar and Ambiguous Queries
by: Abe, Kenya, et al.
Published: (2025)

An Intrinsic Framework of Information Retrieval Evaluation Measures
by: Giner, Fernando
Published: (2023)

Evaluating Contrastive Feedback for Effective User Simulations
by: Kruff, Andreas Konstantin, et al.
Published: (2025)

CreAgent: Towards Long-Term Evaluation of Recommender System under Platform-Creator Information Asymmetry
by: Ye, Xiaopeng, et al.
Published: (2025)

Comparing the Utility, Preference, and Performance of Course Material Search Functionality and Retrieval-Augmented Generation Large Language Model (RAG-LLM) AI Chatbots in Information-Seeking Tasks
by: Pasquarelli, Leonardo, et al.
Published: (2024)

LLM-Assisted Relevance Assessments: When Should We Ask LLMs for Help?
by: Takehi, Rikiya, et al.
Published: (2024)

Boosting LLM-based Relevance Modeling with Distribution-Aware Robust Learning
by: Liu, Hong, et al.
Published: (2024)

RobustExplain: Evaluating Robustness of LLM-Based Explanation Agents for Recommendation
by: Zhang, Guilin, et al.
Published: (2026)

Revisiting Text Ranking in Deep Research
by: Meng, Chuan, et al.
Published: (2026)

Re-Rankers as Relevance Judges
by: Meng, Chuan, et al.
Published: (2026)

Likert or Not: LLM Absolute Relevance Judgments on Fine-Grained Ordinal Scales
by: Godfrey, Charles, et al.
Published: (2025)

Less LLM, More Documents: Searching for Improved RAG
by: Ning, Jingjie, et al.
Published: (2025)

Online Item Cold-Start Recommendation with Popularity-Aware Meta-Learning
by: Luo, Yunze, et al.
Published: (2024)

Exploring Content-Based and Meta-Data Analysis for Detecting Fake News Infodemic: A case study on COVID-19
by: Ajao, Oluwaseun, et al.
Published: (2024)

Content-Aware Tweet Location Inference using Quadtree Spatial Partitioning and Jaccard-Cosine Word Embedding
by: Ajao, Oluwaseun, et al.
Published: (2024)

ColBERT's [MASK]-based Query Augmentation: Effects of Quadrupling the Query Input Length
by: Giacalone, Ben, et al.
Published: (2024)

Cluster-based Graph Collaborative Filtering
by: Liu, Fan, et al.
Published: (2024)

Improving E-commerce Search with Category-Aligned Retrieval
by: Aliev, Rauf
Published: (2025)

Hyena Operator for Fast Sequential Recommendation
by: Liu, Jiahao, et al.
Published: (2026)

Text-like Encoding of Collaborative Information in Large Language Models for Recommendation
by: Zhang, Yang, et al.
Published: (2024)

Graph Reasoning for Explainable Cold Start Recommendation
by: Frej, Jibril, et al.
Published: (2024)

MIND Your Language: A Multilingual Dataset for Cross-lingual News Recommendation
by: Iana, Andreea, et al.
Published: (2024)

Information Retrieval for Climate Impact
by: de Rijke, Maarten, et al.
Published: (2025)

Reverse-Engineering the Retrieval Process in GenIR Models
by: Reusch, Anja, et al.
Published: (2025)

CSMF: Cascaded Selective Mask Fine-Tuning for Multi-Objective Embedding-Based Retrieval
by: Deng, Hao, et al.
Published: (2025)

HeterRec: Heterogeneous Information Transformer for Scalable Sequential Recommendation
by: Deng, Hao, et al.
Published: (2025)

Doc2Query++: Topic-Coverage based Document Expansion and its Application to Dense Retrieval via Dual-Index Fusion
by: Kuo, Tzu-Lin, et al.
Published: (2025)

Comprehensive List Generation for Multi-Generator Reranking
by: Yang, Hailan, et al.
Published: (2025)

Value Function Decomposition in Markov Recommendation Process
by: Wang, Xiaobei, et al.
Published: (2025)

Formalized Information Needs Improve Large-Language-Model Relevance Judgments
by: Keller, Jüri, et al.
Published: (2026)

From Item-Only to Query-Item: Query-Conditioned Generative Search with QGS in Quark
by: Song, Yanglong, et al.
Published: (2026)

Relevance Filtering for Embedding-based Retrieval
by: Rossi, Nicholas, et al.
Published: (2024)

KamerRaad: Enhancing Information Retrieval in Belgian National Politics through Hierarchical Summarization and Conversational Interfaces
by: Rogiers, Alexander, et al.
Published: (2024)