Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2602.15778 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910271138693120 |
|---|---|
| author | Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien |
| author_facet | Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien |
| contents | Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over ``Yes/No'' answers without generating text. We introduce *-PLUIE, task specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15778 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | *-PLUIE: Personalisable metric with Llm Used for Improved Evaluation Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien Computation and Language Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over ``Yes/No'' answers without generating text. We introduce *-PLUIE, task specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost. |
| title | *-PLUIE: Personalisable metric with Llm Used for Improved Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.15778 |