SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics
Fuente:
arXiv
Salvato in:
| Autori principali: | , , , , |
|---|---|
| Natura: | Preprint |
| Pubblicazione: |
2026
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866917217409433600 |
|---|---|
| author | Casola, Silvia Shim, Ryan Soh-Eun Körner, Felicia Mao, Yuchen Plank, Barbara |
| author_facet | Casola, Silvia Shim, Ryan Soh-Eun Körner, Felicia Mao, Yuchen Plank, Barbara |
| contents | An increasing body of work has leveraged multilingual language models for Natural Language Generation tasks such as summarization. A major empirical bottleneck in this area is the shortage of accurate and robust evaluation metrics for many languages, which hinders progress. Recent studies suggest that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance. Motivated by the hypothesis that this mismatch could also apply to multilingual neural metrics, we ask whether steering their activations toward an English pivot can improve correlation with human judgments. We experiment with encoder- and decoder-based metrics and find that test-time intervention methods are effective across the board, increasing metric effectiveness for diverse languages. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2601_15809 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics Casola, Silvia Shim, Ryan Soh-Eun Körner, Felicia Mao, Yuchen Plank, Barbara Computation and Language An increasing body of work has leveraged multilingual language models for Natural Language Generation tasks such as summarization. A major empirical bottleneck in this area is the shortage of accurate and robust evaluation metrics for many languages, which hinders progress. Recent studies suggest that multilingual language models often use English as an internal pivot language, and that misalignment with this pivot can lead to degraded downstream performance. Motivated by the hypothesis that this mismatch could also apply to multilingual neural metrics, we ask whether steering their activations toward an English pivot can improve correlation with human judgments. We experiment with encoder- and decoder-based metrics and find that test-time intervention methods are effective across the board, increasing metric effectiveness for diverse languages. |
| title | SteerEval: Inference-time Interventions Strengthen Multilingual Generalization in Neural Summarization Metrics |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2601.15809 |