Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judge

Documenti analoghi

• ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models
  di: Elangovan, Aparna, et al.
  Pubblicazione: (2024)
• Human-Centered Design Recommendations for LLM-as-a-Judge
  di: Pan, Qian, et al.
  Pubblicazione: (2024)
• EvalAssist: A Human-Centered Tool for LLM-as-a-Judge
  di: Ashktorab, Zahra, et al.
  Pubblicazione: (2025)
• Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judges
  di: Do, Hyo Jin, et al.
  Pubblicazione: (2025)
• Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks
  di: Szymanski, Annalisa, et al.
  Pubblicazione: (2024)