Copertina

Re-evaluating Automatic LLM System Ranking for Alignment with Human Preference

Fuente: arXiv

Salvato in:

Dettagli Bibliografici
Autori principali:	Gao, Mingqi, Liu, Yixin, Hu, Xinyu, Wan, Xiaojun, Bragg, Jonathan, Cohan, Arman
Natura:	Preprint
Pubblicazione:	2024
Soggetti:	Computation and Language Artificial Intelligence Machine Learning
Accesso online:	Acceder al recurso
Tags:	Aggiungi Tag Nessun Tag, puoi essere il primo ad aggiungerne!!

Documenti analoghi

On Evaluating LLM Alignment by Evaluating LLMs as Judges
di: Liu, Yixin, et al.
Pubblicazione: (2025)

References Improve LLM Alignment in Non-Verifiable Domains
di: Shi, Kejian, et al.
Pubblicazione: (2026)

ReIFE: Re-evaluating Instruction-Following Evaluation
di: Liu, Yixin, et al.
Pubblicazione: (2024)

COMAL: A Convergent Meta-Algorithm for Aligning LLMs with General Preferences
di: Liu, Yixin, et al.
Pubblicazione: (2024)

Calibrating Long-form Generations from Large Language Models
di: Huang, Yukun, et al.
Pubblicazione: (2024)

Understanding Reference Policies in Direct Preference Optimization
di: Liu, Yixin, et al.
Pubblicazione: (2024)

Re-Thinking the Automatic Evaluation of Image-Text Alignment in Text-to-Image Models
di: Zhang, Huixuan, et al.
Pubblicazione: (2025)

Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
di: Liu, Yixin, et al.
Pubblicazione: (2026)

Bayesian Calibration of Win Rate Estimation with LLM Evaluators
di: Gao, Yicheng, et al.
Pubblicazione: (2024)

Preference Ranking Optimization for Human Alignment
di: Song, Feifan, et al.
Pubblicazione: (2023)

Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences
di: Pattnaik, Pulkit, et al.
Pubblicazione: (2024)

Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment
di: Kim, Dongyoung, et al.
Pubblicazione: (2024)

Survey on Evaluation of LLM-based Agents
di: Yehudai, Asaf, et al.
Pubblicazione: (2025)

Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game
di: Cheng, Pengyu, et al.
Pubblicazione: (2023)

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models
di: Li, Chuhan, et al.
Pubblicazione: (2024)

MaxMin-RLHF: Alignment with Diverse Human Preferences
di: Chakraborty, Souradip, et al.
Pubblicazione: (2024)

PITA: Preference-Guided Inference-Time Alignment for LLM Post-Training
di: Bobbili, Sarat Chandra, et al.
Pubblicazione: (2025)

Less is More: Improving LLM Alignment via Preference Data Selection
di: Deng, Xun, et al.
Pubblicazione: (2025)

Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignment
di: Cheng, Ruoxi, et al.
Pubblicazione: (2025)

One-shot Optimized Steering Vectors Mediate Safety-relevant Behaviors in LLMs
di: Dunefsky, Jacob, et al.
Pubblicazione: (2025)

Preference Learning Algorithms Do Not Learn Preference Rankings
di: Chen, Angelica, et al.
Pubblicazione: (2024)

JuStRank: Benchmarking LLM Judges for System Ranking
di: Gera, Ariel, et al.
Pubblicazione: (2024)

On the Benefits of Fine-Grained Loss Truncation: A Case Study on Factuality in Summarization
di: Flores, Lorenzo Jaime Yu, et al.
Pubblicazione: (2024)

RLTHF: Targeted Human Feedback for LLM Alignment
di: Xu, Yifei, et al.
Pubblicazione: (2025)

Improving LLM General Preference Alignment via Optimistic Online Mirror Descent
di: Zhang, Yuheng, et al.
Pubblicazione: (2025)

Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
di: Dong, Guanting, et al.
Pubblicazione: (2024)

MINOS: A Multimodal Evaluation Model for Bidirectional Generation Between Image and Text
di: Zhang, Junzhe, et al.
Pubblicazione: (2025)

Larger or Smaller Reward Margins to Select Preferences for Alignment?
di: Huang, Kexin, et al.
Pubblicazione: (2025)

SUCEA: Reasoning-Intensive Retrieval for Adversarial Fact-checking through Claim Decomposition and Editing
di: Liu, Hongjun, et al.
Pubblicazione: (2025)

ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks
di: Yu, Xiaodong, et al.
Pubblicazione: (2023)

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets
di: Devine, Peter
Pubblicazione: (2024)

Course-Correction: Safety Alignment Using Synthetic Preferences
di: Xu, Rongwu, et al.
Pubblicazione: (2024)

Exploring the Multilingual NLG Evaluation Abilities of LLM-Based Evaluators
di: Chang, Jiayi, et al.
Pubblicazione: (2025)

Relative Preference Optimization: Enhancing LLM Alignment through Contrasting Responses across Identical and Diverse Prompts
di: Yin, Yueqin, et al.
Pubblicazione: (2024)

Learning from Reference Answers: Versatile Language Model Alignment without Binary Human Preference Data
di: Zhao, Shuai, et al.
Pubblicazione: (2025)

Towards Unified Alignment Between Agents, Humans, and Environment
di: Yang, Zonghan, et al.
Pubblicazione: (2024)

Quantifying and Mitigating Self-Preference Bias of LLM Judges
di: Yang, Jinming, et al.
Pubblicazione: (2026)

AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research
di: Zhao, Yilun, et al.
Pubblicazione: (2025)

Towards Robust Alignment of Language Models: Distributionally Robustifying Direct Preference Optimization
di: Wu, Junkang, et al.
Pubblicazione: (2024)

AlpsBench: An LLM Personalization Benchmark for Real-Dialogue Memorization and Preference Alignment
di: Xiao, Jianfei, et al.
Pubblicazione: (2026)