From Internal Representations to Text Quality: A Geometric Approach to LLM Evaluation
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866909815192682496 |
|---|---|
| author | Yusupov, Viacheslav Maksimov, Danil Alaeva, Ameliia Vasileva, Anna Antipina, Anna Zaitseva, Tatyana Ermilova, Alina Burnaev, Evgeny Shvetsov, Egor |
| author_facet | Yusupov, Viacheslav Maksimov, Danil Alaeva, Ameliia Vasileva, Anna Antipina, Anna Zaitseva, Tatyana Ermilova, Alina Burnaev, Evgeny Shvetsov, Egor |
| contents | This paper bridges internal and external analysis approaches to large language models (LLMs) by demonstrating that geometric properties of internal model representations serve as reliable proxies for evaluating generated text quality. We validate a set of metrics including Maximum Explainable Variance, Effective Rank, Intrinsic Dimensionality, MAUVE score, and Schatten Norms measured across different layers of LLMs, demonstrating that Intrinsic Dimensionality and Effective Rank can serve as universal assessments of text naturalness and quality. Our key finding reveals that different models consistently rank text from various sources in the same order based on these geometric properties, indicating that these metrics reflect inherent text characteristics rather than model-specific artifacts. This allows a reference-free text quality evaluation that does not require human-annotated datasets, offering practical advantages for automated evaluation pipelines. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2509_25359 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | From Internal Representations to Text Quality: A Geometric Approach to LLM Evaluation Yusupov, Viacheslav Maksimov, Danil Alaeva, Ameliia Vasileva, Anna Antipina, Anna Zaitseva, Tatyana Ermilova, Alina Burnaev, Evgeny Shvetsov, Egor Computation and Language Artificial Intelligence This paper bridges internal and external analysis approaches to large language models (LLMs) by demonstrating that geometric properties of internal model representations serve as reliable proxies for evaluating generated text quality. We validate a set of metrics including Maximum Explainable Variance, Effective Rank, Intrinsic Dimensionality, MAUVE score, and Schatten Norms measured across different layers of LLMs, demonstrating that Intrinsic Dimensionality and Effective Rank can serve as universal assessments of text naturalness and quality. Our key finding reveals that different models consistently rank text from various sources in the same order based on these geometric properties, indicating that these metrics reflect inherent text characteristics rather than model-specific artifacts. This allows a reference-free text quality evaluation that does not require human-annotated datasets, offering practical advantages for automated evaluation pipelines. |
| title | From Internal Representations to Text Quality: A Geometric Approach to LLM Evaluation |
| topic | Computation and Language Artificial Intelligence |
| url | https://arxiv.org/abs/2509.25359 |