Cover Image

Evaluating Large Language Models for Real-World Engineering Tasks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Heesch, Rene, Eilermann, Sebastian, Windmann, Alexander, Diedrich, Alexander, Rosenthal, Philipp, Niggemann, Oliver
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence Computation and Language
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

A Continuous-Time Consistency Model for 3D Point Cloud Generation
by: Eilermann, Sebastian, et al.
Published: (2025)

AAAI Workshop on AI Planning for Cyber-Physical Systems -- CAIPI24
by: Niggemann, Oliver, et al.
Published: (2024)

Industrial AI Robustness Card for Time Series Models
by: Windmann, Alexander, et al.
Published: (2025)

MAWIFlow Benchmark: Realistic Flow-Based Evaluation for Network Intrusion Detection
by: Schraven, Joshua, et al.
Published: (2025)

Artificial Intelligence in Industry 4.0: A Review of Integration Challenges for Industrial Systems
by: Windmann, Alexander, et al.
Published: (2024)

Quantifying Robustness: A Benchmarking Framework for Deep Learning Forecasting in Cyber-Physical Systems
by: Windmann, Alexander, et al.
Published: (2025)

An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniques
by: Aly, Walid Mohamed, et al.
Published: (2025)

RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
by: Yang, Shuo, et al.
Published: (2025)

Data Driven Diagnosis for Large Cyber-Physical-Systems with Minimal Prior Information
by: Steude, Henrik Sebastian, et al.
Published: (2025)

Evaluating Ill-Defined Tasks in Large Language Models
by: Zhou, Yi, et al.
Published: (2026)

QuarkMedBench: A Real-World Scenario Driven Benchmark for Evaluating Large Language Models
by: Wu, Yao, et al.
Published: (2026)

Evaluating Large Language Models for Abstract Evaluation Tasks: An Empirical Study
by: Liu, Yinuo, et al.
Published: (2026)

Can Large Language Models Understand Real-World Complex Instructions?
by: He, Qianyu, et al.
Published: (2023)

Multimodal Large Language Models to Support Real-World Fact-Checking
by: Geng, Jiahui, et al.
Published: (2024)

Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines
by: Deng, Guifeng, et al.
Published: (2025)

MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
by: Zong, Xuanjun, et al.
Published: (2025)

SemEval-2026 Task 12: Abductive Event Reasoning: Towards Real-World Event Causal Inference for Large Language Models
by: Cao, Pengfei, et al.
Published: (2026)

Avionic Main Fuel Pump Simulation and Fault-Diagnosis Benchmark
by: Janzen, Felix Leonhard, et al.
Published: (2026)

Frontier-Eng: Benchmarking Self-Evolving Agents on Real-World Engineering Tasks with Generative Optimization
by: Chi, Yizhe, et al.
Published: (2026)

ConsintBench: Evaluating Language Models on Real-World Consumer Intent Understanding
by: Li, Xiaozhe, et al.
Published: (2025)

A Pragmatist Robot: Learning to Plan Tasks by Experiencing the Real World
by: Qu, Kaixian, et al.
Published: (2025)

Assessing Empathy in Large Language Models with Real-World Physician-Patient Interactions
by: Luo, Man, et al.
Published: (2024)

Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey
by: Mondorf, Philipp, et al.
Published: (2024)

ANLS* -- A Universal Document Processing Metric for Generative Large Language Models
by: Peer, David, et al.
Published: (2024)

A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks
by: Vatsal, Shubham, et al.
Published: (2024)

MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks
by: Bedi, Suhana, et al.
Published: (2025)

Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks
by: Anand, Avinash, et al.
Published: (2024)

MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
by: Luo, Ziyang, et al.
Published: (2025)

Scan-do Attitude: Towards Autonomous CT Protocol Management using a Large Language Model Agent
by: Kang, Xingjian, et al.
Published: (2025)

An Investigation of Large Language Models for Real-World Hate Speech Detection
by: Guo, Keyan, et al.
Published: (2024)

Large Language Models as User-Agents for Evaluating Task-Oriented-Dialogue Systems
by: Kazi, Taaha, et al.
Published: (2024)

Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Models
by: Lewis, Martha, et al.
Published: (2024)

SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks
by: Adamenko, Pavel, et al.
Published: (2025)

Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
by: Sahitaj, Premtim, et al.
Published: (2025)

Large Language Models as Generalizable Policies for Embodied Tasks
by: Szot, Andrew, et al.
Published: (2023)

AtomWorld: A Benchmark for Evaluating Spatial Reasoning in Large Language Models on Crystalline Materials
by: Lv, Taoyuze, et al.
Published: (2025)

Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Models
by: Han, Chen, et al.
Published: (2025)

TaskBench: Benchmarking Large Language Models for Task Automation
by: Shen, Yongliang, et al.
Published: (2023)

A Comprehensive Evaluation of Large Language Models on Benchmark Biomedical Text Processing Tasks
by: Jahan, Israt, et al.
Published: (2023)

Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review
by: Croxford, Emma, et al.
Published: (2024)