Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2026
|
| Materias: | |
| Acceso en línea: | https://arxiv.org/abs/2602.12143 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866912900864540672 |
|---|---|
| author | Wang, Xiaoxiao Li, Chunxiao Wang, Junying Guo, Yijin Chen, Zijian Li, Chunyi Liu, Xiaohong Zhang, Zicheng Zhai, Guangtao |
| author_facet | Wang, Xiaoxiao Li, Chunxiao Wang, Junying Guo, Yijin Chen, Zijian Li, Chunyi Liu, Xiaohong Zhang, Zicheng Zhai, Guangtao |
| contents | As comprehensive large model evaluation becomes prohibitively expensive, predicting model performance from limited observations has become essential. However, existing statistical methods struggle with pattern shifts, data sparsity, and lack of explanation, while pure LLM methods remain unreliable. We propose STAR, a framework that bridges data-driven STatistical expectations with knowledge-driven Agentic Reasoning. STAR leverages specialized retrievers to gather external knowledge and embeds semantic features into Constrained Probabilistic Matrix Factorization (CPMF) to generate statistical expectations with uncertainty. A reasoning module guided by Expectation Violation Theory (EVT) then refines predictions through intra-family analysis, cross-model comparison, and credibility-aware aggregation, producing adjustments with traceable explanations. Extensive experiments show that STAR consistently outperforms all baselines on both score-based and rank-based metrics, delivering a 14.46% gain in total score over the strongest statistical method under extreme sparsity, with only 1--2 observed scores per test model. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_12143 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction Wang, Xiaoxiao Li, Chunxiao Wang, Junying Guo, Yijin Chen, Zijian Li, Chunyi Liu, Xiaohong Zhang, Zicheng Zhai, Guangtao Artificial Intelligence Machine Learning I.2.0, I.2.6, H.3.3 As comprehensive large model evaluation becomes prohibitively expensive, predicting model performance from limited observations has become essential. However, existing statistical methods struggle with pattern shifts, data sparsity, and lack of explanation, while pure LLM methods remain unreliable. We propose STAR, a framework that bridges data-driven STatistical expectations with knowledge-driven Agentic Reasoning. STAR leverages specialized retrievers to gather external knowledge and embeds semantic features into Constrained Probabilistic Matrix Factorization (CPMF) to generate statistical expectations with uncertainty. A reasoning module guided by Expectation Violation Theory (EVT) then refines predictions through intra-family analysis, cross-model comparison, and credibility-aware aggregation, producing adjustments with traceable explanations. Extensive experiments show that STAR consistently outperforms all baselines on both score-based and rank-based metrics, delivering a 14.46% gain in total score over the strongest statistical method under extreme sparsity, with only 1--2 observed scores per test model. |
| title | STAR : Bridging Statistical and Agentic Reasoning for Large Model Performance Prediction |
| topic | Artificial Intelligence Machine Learning I.2.0, I.2.6, H.3.3 |
| url | https://arxiv.org/abs/2602.12143 |