Cover Image

Batched Self-Consistency Improves LLM Relevance Assessment and Ranking

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Korikov, Anton, Du, Pan, Sanner, Scott, Rekabsaz, Navid
Format:	Preprint
Published:	2025
Subjects:	Information Retrieval
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

ReBOL: Retrieval via Bayesian Optimization with Batched LLM Relevance Observations and Query Reformulation
by: Korikov, Anton, et al.
Published: (2026)

Multimodal Item Scoring for Natural Language Recommendation via Gaussian Process Regression with LLM Relevance Judgments
by: Liu, Yifan, et al.
Published: (2025)

Multi-Aspect Reviewed-Item Retrieval via LLM Query Decomposition and Aspect Fusion
by: Korikov, Anton, et al.
Published: (2024)

Bayesian Active Learning with Gaussian Processes Guided by LLM Relevance Scoring for Dense Passage Retrieval
by: Kim, Junyoung, et al.
Published: (2026)

A Simple but Effective Elaborative Query Reformulation Approach for Natural Language Recommendation
by: Wen, Qianfeng, et al.
Published: (2025)

Elaborative Subtopic Query Reformulation for Broad and Indirect Queries in Travel Destination Recommendation
by: Wen, Qianfeng, et al.
Published: (2024)

Multi-modal Generative Models in Recommendation System
by: Ramisa, Arnau, et al.
Published: (2024)

A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)
by: Deldjoo, Yashar, et al.
Published: (2024)

REALM: Recursive Relevance Modeling for LLM-based Document Re-Ranking
by: Wang, Pinhuan, et al.
Published: (2025)

When LLM Judges Inflate Scores: Exploring Overrating in Relevance Assessment
by: Yu, Chuting, et al.
Published: (2026)

Joint Optimization of Relevance and Engagement in Multi-Task Ranking for E-Commerce with Efficient LLM Supervision
by: Chen, Luming, et al.
Published: (2026)

MA-DPR: Manifold-aware Distance Metrics for Dense Passage Retrieval
by: Liu, Yifan, et al.
Published: (2025)

Recommendation with Generative Models
by: Deldjoo, Yashar, et al.
Published: (2024)

LLM4PR: Improving Post-Ranking in Search Engine with Large Language Models
by: Yan, Yang, et al.
Published: (2024)

LLM-Assisted Pseudo-Relevance Feedback
by: Otero, David, et al.
Published: (2026)

HCMRM: A High-Consistency Multimodal Relevance Model for Search Ads
by: Gan, Guobing, et al.
Published: (2025)

Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labels
by: Zhuang, Honglei, et al.
Published: (2023)

LLM-based Relevance Assessment for Web-Scale Search Evaluation at Pinterest
by: Wang, Han, et al.
Published: (2025)

The Cranfield II Relevance Assessments: A Critical Evaluation
by: Harter, Stephen P.
Published: (1971)

Consolidating Ranking and Relevance Predictions of Large Language Models through Post-Processing
by: Yan, Le, et al.
Published: (2024)

SARA: A Collection of Sensitivity-Aware Relevance Assessments
by: McKechnie, Jack, et al.
Published: (2024)

JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
by: Rahmani, Hossein A., et al.
Published: (2024)

Rank-Preference Consistency as the Appropriate Metric for Recommender Systems
by: Nguyen, Tung, et al.
Published: (2024)

Rethinking Multi-objective Ranking Ensemble in Recommender System: From Score Fusion to Rank Consistency
by: Xia, Boyang, et al.
Published: (2026)

RankSteer: Activation Steering for Pointwise LLM Ranking
by: Wang, Yumeng, et al.
Published: (2026)

Judging with Personality and Confidence: A Study on Personality-Conditioned LLM Relevance Assessment
by: Chen, Nuo, et al.
Published: (2026)

OrdRankBen: A Novel Ranking Benchmark for Ordinal Relevance in NLP
by: Wang, Yan, et al.
Published: (2025)

Think When Needed: Model-Aware Reasoning Routing for LLM-based Ranking
by: Guo, Huizhong, et al.
Published: (2026)

Beyond Self-Consistency: Loss-Balanced Perturbation-Based Regularization Improves Industrial-Scale Ads Ranking
by: Ramazanli, Ilqar, et al.
Published: (2025)

Unbiased Learning to Rank with Query-Level Click Propensity Estimation: Beyond Pointwise Observation and Relevance
by: Yu, Lulu, et al.
Published: (2025)

Towards More Relevant Product Search Ranking Via Large Language Models: An Empirical Study
by: Liu, Qi, et al.
Published: (2024)

KuaiSearch: A Large-Scale E-Commerce Search Dataset for Recall, Ranking, and Relevance
by: Li, Yupeng, et al.
Published: (2026)

Contextual Relevance and Adaptive Sampling for LLM-Based Document Reranking
by: Huang, Jerry, et al.
Published: (2025)

Validating LLM-Generated Relevance Labels for Educational Resource Search
by: Sebastian, Ratan J., et al.
Published: (2025)

Judging the Judges: A Collection of LLM-Generated Relevance Judgements
by: Rahmani, Hossein A., et al.
Published: (2025)

Scaling Search Relevance: Augmenting App Store Ranking with LLM-Generated Judgments
by: Christakopoulou, Evangelia, et al.
Published: (2026)

Improving Topic Relevance Model by Mix-structured Summarization and LLM-based Data Augmentation
by: Liu, Yizhu, et al.
Published: (2024)

Beyond Relevance: Improving User Engagement by Personalization for Short-Video Search
by: Bao, Wentian, et al.
Published: (2024)

Generative Relevance Feedback and Convergence of Adaptive Re-Ranking: University of Glasgow Terrier Team at TREC DL 2023
by: Parry, Andrew, et al.
Published: (2024)

Limitations of Automatic Relevance Assessments with Large Language Models for Fair and Reliable Retrieval Evaluation
by: Otero, David, et al.
Published: (2024)