Recovering manifold structure in LLM responses through a joint Euclidean mirror
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2026
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910163526483968 |
|---|---|
| author | Baum, Maximilian Acharyya, Aranyak Chen, Tianyi Athreya, Avanti Park, Youngser Passino, Francesco Sanna Priebe, Carey E. Lubberts, Zachary |
| author_facet | Baum, Maximilian Acharyya, Aranyak Chen, Tianyi Athreya, Avanti Park, Youngser Passino, Francesco Sanna Priebe, Carey E. Lubberts, Zachary |
| contents | Understanding the behavior of black-box large language models and determining effective means of comparing their performance is a key task in modern machine learning. We consider how large language models respond to a specific query by analyzing how the distributions of responses vary over different values of tuning parameters. We frame this problem in a general mathematical setting, treating the mapping from model parameters to response distributions as a structured family of probability measures, endowed with a geometry via a dissimilarity measure. We show how dissimilarities between response distributions can be represented in low-dimensional Euclidean space through a joint Euclidean mirror surface encoding the underlying geometry, which permits both qualitative and quantitative analysis of large language models and provides insight into predicting response distributions for different values of tuning parameters. We propose an estimation procedure for the underlying joint Euclidean mirror based on observed samples from the response distributions, and we prove its asymptotic properties. Additionally, we propose a statistically consistent procedure to infer the value of an unknown model parameter based on samples from the corresponding response distribution and the estimated joint Euclidean mirror. In an experimental setting with large language models, we find that changes in different tuning parameter values correspond to distinct directions in the embedding space, making it possible to estimate the tuning parameters that were used to generate a given response. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2604_07011 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | Recovering manifold structure in LLM responses through a joint Euclidean mirror Baum, Maximilian Acharyya, Aranyak Chen, Tianyi Athreya, Avanti Park, Youngser Passino, Francesco Sanna Priebe, Carey E. Lubberts, Zachary Methodology Applications 62G05, 68T01 Understanding the behavior of black-box large language models and determining effective means of comparing their performance is a key task in modern machine learning. We consider how large language models respond to a specific query by analyzing how the distributions of responses vary over different values of tuning parameters. We frame this problem in a general mathematical setting, treating the mapping from model parameters to response distributions as a structured family of probability measures, endowed with a geometry via a dissimilarity measure. We show how dissimilarities between response distributions can be represented in low-dimensional Euclidean space through a joint Euclidean mirror surface encoding the underlying geometry, which permits both qualitative and quantitative analysis of large language models and provides insight into predicting response distributions for different values of tuning parameters. We propose an estimation procedure for the underlying joint Euclidean mirror based on observed samples from the response distributions, and we prove its asymptotic properties. Additionally, we propose a statistically consistent procedure to infer the value of an unknown model parameter based on samples from the corresponding response distribution and the estimated joint Euclidean mirror. In an experimental setting with large language models, we find that changes in different tuning parameter values correspond to distinct directions in the embedding space, making it possible to estimate the tuning parameters that were used to generate a given response. |
| title | Recovering manifold structure in LLM responses through a joint Euclidean mirror |
| topic | Methodology Applications 62G05, 68T01 |
| url | https://arxiv.org/abs/2604.07011 |