Imagen de Portada

From Raw Corpora to Domain Benchmarks: Automated Evaluation of LLM Domain Expertise

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Sharma, Nitin, Wolfers, Thomas, Yıldız, Çağatay
Formato:	Preprint
Publicado:	2025
Materias:	Computation and Language
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

Question Answering under Temporal Conflict: Evaluating and Organizing Evolving Knowledge with LLMs
por: Özer, Atahan, et al.
Publicado: (2025)

Investigating Continual Pretraining in Large Language Models: Insights and Implications
por: Yıldız, Çağatay, et al.
Publicado: (2024)

Benchmarking GPT-4 against Human Translators: A Comprehensive Evaluation Across Languages, Domains, and Expertise Levels
por: Yan, Jianhao, et al.
Publicado: (2024)

A Reproduction Study: The Kernel PCA Interpretation of Self-Attention Fails Under Scrutiny
por: Sarıtaş, Karahan, et al.
Publicado: (2025)

Unearthing Large Scale Domain-Specific Knowledge from Public Corpora
por: Fei, Zhaoye, et al.
Publicado: (2024)

Building Corpora for Single-Channel Speech Separation Across Multiple Domains
por: Maciejewski, Matthew, et al.
Publicado: (2018)

ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertise
por: Guo, Xingang, et al.
Publicado: (2024)

From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models
por: Alsagheer, Dana, et al.
Publicado: (2025)

On the Benchmarking of LLMs for Open-Domain Dialogue Evaluation
por: Mendonça, John, et al.
Publicado: (2024)

GPT-4 vs. Human Translators: A Comprehensive Evaluation of Translation Quality Across Languages, Domains, and Expertise Levels
por: Yan, Jianhao, et al.
Publicado: (2024)

MIMIR: A Streamlined Platform for Personalized Agent Tuning in Domain Expertise
por: Deng, Chunyuan, et al.
Publicado: (2024)

DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicine
por: Seo, Jean, et al.
Publicado: (2024)

FActBench: A Benchmark for Fine-grained Automatic Evaluation of LLM-Generated Text in the Medical Domain
por: Afzal, Anum, et al.
Publicado: (2025)

Mathematical Entities: Corpora and Benchmarks
por: Collard, Jacob, et al.
Publicado: (2024)

Leveraging LLMs to Create Content Corpora for Niche Domains
por: Zhang, Franklin, et al.
Publicado: (2025)

GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agents
por: Diao, Lingxiao, et al.
Publicado: (2025)

Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation
por: Gu, Zhouhong, et al.
Publicado: (2023)

DomainRAG: A Chinese Benchmark for Evaluating Domain-specific Retrieval-Augmented Generation
por: Wang, Shuting, et al.
Publicado: (2024)

Advancing Conversational Psychotherapy: Integrating Privacy, Dual-Memory, and Domain Expertise with Large Language Models
por: Zhang, XiuYu, et al.
Publicado: (2024)

Automated Benchmark Generation from Domain Guidelines Informed by Bloom's Taxonomy
por: Chen, Si, et al.
Publicado: (2026)

From Physician Expertise to Clinical Agents: Preserving, Standardizing, and Scaling Physicians' Medical Expertise with Lightweight LLM
por: Luo, Chanyong, et al.
Publicado: (2026)

MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities
por: Chen, Jingxue, et al.
Publicado: (2025)

Fact Finder -- Enhancing Domain Expertise of Large Language Models by Incorporating Knowledge Graphs
por: Steinigen, Daniel, et al.
Publicado: (2024)

Fine-Tuning Medical Language Models for Enhanced Long-Contextual Understanding and Domain Expertise
por: Yang, Qimin, et al.
Publicado: (2024)

OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain
por: Wang, Shuting, et al.
Publicado: (2024)

Beyond English and Evasion: A Human-Annotated Multi-Domain Benchmark for High-Stakes LLM Safety Evaluation in Chinese
por: Zaghouani, Wajdi, et al.
Publicado: (2026)

Automated Collection of Evaluation Dataset for Semantic Search in Low-Resource Domain Language
por: Zhukova, Anastasia, et al.
Publicado: (2024)

Process-Supervised Reward Models for Verifying Clinical Note Generation: A Scalable Approach Guided by Domain Expertise
por: Wang, Hanyin, et al.
Publicado: (2024)

Evaluating LLM Reasoning in the Operations Research Domain with ORQA
por: Mostajabdaveh, Mahdi, et al.
Publicado: (2024)

EvalSense: A Framework for Domain-Specific LLM (Meta-)Evaluation
por: Dejl, Adam, et al.
Publicado: (2026)

MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
por: Mendonça, John, et al.
Publicado: (2025)

RAGalyst: Automated Human-Aligned Agentic Evaluation for Domain-Specific RAG
por: Gao, Joshua, et al.
Publicado: (2025)

Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning
por: Hu, Tianxiang, et al.
Publicado: (2024)

ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain
por: Zhao, Haochen, et al.
Publicado: (2024)

LegalAgentBench: Evaluating LLM Agents in Legal Domain
por: Li, Haitao, et al.
Publicado: (2024)

A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domains
por: Wang, Shirui, et al.
Publicado: (2025)

QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation
por: Hong, Mengze, et al.
Publicado: (2025)

Noor-Ghateh: A Benchmark Dataset for Evaluating Arabic Word Segmenters in Hadith Domain
por: AlShuhayeb, Huda, et al.
Publicado: (2023)

The Metacognitive Monitoring Battery: A Cross-Domain Benchmark for LLM Self-Monitoring
por: Cacioli, Jon-Paul
Publicado: (2026)

Novel Benchmark for NER in the Wastewater and Stormwater Domain
por: Cardillo, Franco Alberto, et al.
Publicado: (2025)