*-PLUIE: Personalisable metric with Llm Used for Improved Evaluation
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866910271138693120 |
|---|---|
| author | Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien |
| author_facet | Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien |
| contents | Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over ``Yes/No'' answers without generating text. We introduce *-PLUIE, task specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_15778 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | *-PLUIE: Personalisable metric with Llm Used for Improved Evaluation Lemesle, Quentin Jourdan, Léane Munson, Daisy Alain, Pierre Chevelu, Jonathan Delhay, Arnaud Lolive, Damien Computation and Language Evaluating the quality of automatically generated text often relies on LLM-as-a-judge (LLM-judge) methods. While effective, these approaches are computationally expensive and require post-processing. To address these limitations, we build upon ParaPLUIE, a perplexity-based LLM-judge metric that estimates confidence over ``Yes/No'' answers without generating text. We introduce *-PLUIE, task specific prompting variants of ParaPLUIE and evaluate their alignment with human judgement. Our experiments show that personalised *-PLUIE achieves stronger correlations with human ratings while maintaining low computational cost. |
| title | *-PLUIE: Personalisable metric with Llm Used for Improved Evaluation |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2602.15778 |