Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911190891888640 |
|---|---|
| author | Zhou, Ej Zhang, Caiqi Hu, Tiancheng Li, Chengzu Collier, Nigel Vulić, Ivan Korhonen, Anna |
| author_facet | Zhou, Ej Zhang, Caiqi Hu, Tiancheng Li, Chengzu Collier, Nigel Vulić, Ivan Korhonen, Anna |
| contents | Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, this critical property remains largely under-explored in multilingual contexts. In this work, we conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages, revealing that non-English languages suffer from systematically worse calibration. To diagnose this, we investigate the model's internal representations and find that the final layer, biased by English-centric training, provides a poor signal for multilingual confidence. In contrast, our layer-wise analysis uncovers a key insight that late-intermediate layers consistently offer a more reliable and better-calibrated signal. Building on this, we introduce a suite of training-free methods, including Language-Aware Confidence Ensemble (LACE), which adaptively selects an optimal ensemble of layers for each specific language. Our study highlights the hidden costs of English-centric alignment and offer a new path toward building more globally equitable and trustworthy LLMs by looking beyond the final layer. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2510_03136 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models Zhou, Ej Zhang, Caiqi Hu, Tiancheng Li, Chengzu Collier, Nigel Vulić, Ivan Korhonen, Anna Computation and Language Confidence calibration, the alignment of a model's predicted confidence with its actual accuracy, is crucial for the reliable deployment of Large Language Models (LLMs). However, this critical property remains largely under-explored in multilingual contexts. In this work, we conduct the first large-scale, systematic studies of multilingual calibration across six model families and over 100 languages, revealing that non-English languages suffer from systematically worse calibration. To diagnose this, we investigate the model's internal representations and find that the final layer, biased by English-centric training, provides a poor signal for multilingual confidence. In contrast, our layer-wise analysis uncovers a key insight that late-intermediate layers consistently offer a more reliable and better-calibrated signal. Building on this, we introduce a suite of training-free methods, including Language-Aware Confidence Ensemble (LACE), which adaptively selects an optimal ensemble of layers for each specific language. Our study highlights the hidden costs of English-centric alignment and offer a new path toward building more globally equitable and trustworthy LLMs by looking beyond the final layer. |
| title | Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models |
| topic | Computation and Language |
| url | https://arxiv.org/abs/2510.03136 |