Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2508.08277 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866909733350277120 |
|---|---|
| author | Du, Haoze Li, Richard Gehringer, Edward |
| author_facet | Du, Haoze Li, Richard Gehringer, Edward |
| contents | Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing well-defined benchmarks, factual datasets, and structured evaluation pipelines, the approach ensures consistent, reproducible, and bias-minimized measurements. The framework emphasizes automation and transparency in scoring, reducing reliance on human interpretation while ensuring alignment with real-world applications. This method addresses the limitations of subjective evaluation methods, providing a scalable solution for performance assessment in educational, scientific, and other high-stakes domains. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2508_08277 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Objective Metrics for Evaluating Large Language Models Using External Data Sources Du, Haoze Li, Richard Gehringer, Edward Computation and Language Machine Learning Evaluating the performance of Large Language Models (LLMs) is a critical yet challenging task, particularly when aiming to avoid subjective assessments. This paper proposes a framework for leveraging subjective metrics derived from the class textual materials across different semesters to assess LLM outputs across various tasks. By utilizing well-defined benchmarks, factual datasets, and structured evaluation pipelines, the approach ensures consistent, reproducible, and bias-minimized measurements. The framework emphasizes automation and transparency in scoring, reducing reliance on human interpretation while ensuring alignment with real-world applications. This method addresses the limitations of subjective evaluation methods, providing a scalable solution for performance assessment in educational, scientific, and other high-stakes domains. |
| title | Objective Metrics for Evaluating Large Language Models Using External Data Sources |
| topic | Computation and Language Machine Learning |
| url | https://arxiv.org/abs/2508.08277 |