Cover Image

The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Models

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Herambourg, Claudia, Siuda, Dawid, Kopczyńska, Julia, Santos, Joao R. L., Sas, Wojciech, Śmietańska-Nowak, Joanna
Format:	Preprint
Published:	2025
Subjects:	Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

Similar Items

RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
by: Yang, Shuo, et al.
Published: (2025)

Modelling brain connectomes networks: Solv is a worthy competitor to hyperbolic geometry!
by: Celińska-Kopczyńska, Dorota, et al.
Published: (2024)

Editorial
by: Renata Siuda-Ambroziak
Published: (2017)

Por que vale a pena pesquisar e analisar as questões religiosas no contexto latino-americano?
by: Renata Siuda-Ambroziak
Published: (2017)

Açorianidade e brasilidade nas Festas do Divino: o caso de Viana (ES)
by: Renata Siuda-Ambroziak
Published: (2020)

Religião e religiosidade em transformações – dilemas metodológicos
by: Renata Siuda Ambroziak
Published: (2017)

Benchmarking ORCA PT-1 Boson Sampler in Simulation
by: Park, Jessica, et al.
Published: (2025)

RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
by: Jin, Zhuoran, et al.
Published: (2024)

QuarkMedBench: A Real-World Scenario Driven Benchmark for Evaluating Large Language Models
by: Wu, Yao, et al.
Published: (2026)

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
by: Hu, Ruida, et al.
Published: (2024)

Theoretical limits of electron and hole doping in single layer graphene from DFT calculations
by: Ciszewski, Dawid, et al.
Published: (2025)

Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcare
by: Yang, Yifan, et al.
Published: (2024)

angelinamary/ORCA_analysers: ORCA_5.0.4_analysers
by: Angelina Mary
Published: (2026)

Evaluating Large Language Models for Real-World Engineering Tasks
by: Heesch, Rene, et al.
Published: (2025)

Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks
by: Frank, Kevin, et al.
Published: (2025)

Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines
by: Deng, Guifeng, et al.
Published: (2025)

MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
by: Zong, Xuanjun, et al.
Published: (2025)

Thermal Generation of Spin Current in a Quantum Dot Coupled to Magnetic Insulators
by: Siuda, Emil, et al.
Published: (2024)

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
by: Fonseca, Joao, et al.
Published: (2025)

Numerical Aspects of Hyperbolic Geometry
by: Celinska-Kopczynska, Dorota, et al.
Published: (2024)

Non-Euclidean Self-Organizing Maps
by: Celińska-Kopczyńska, Dorota, et al.
Published: (2021)

Budgeting in an open system
by: Wojciech A. Nowak
Published: (2004)

MojoBench: Language Modeling and Benchmarks for Mojo
by: Raihan, Nishat, et al.
Published: (2024)

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding
by: Radwan, Ahmed Y., et al.
Published: (2026)

Exponential ergodicity of some Markov dynamical system with application to a Poisson driven stochastic differential equation
by: Czapla, Dawid, et al.
Published: (2018)

MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
by: Liang, Hao, et al.
Published: (2024)

MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
by: Luo, Ziyang, et al.
Published: (2025)

Modelling IRB Risk‐Weighted Assets: Looking Beyond Stress Tests
by: Josef Švéda, et al.
Published: (2025)

MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
by: Xue, Haochen, et al.
Published: (2025)

Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark
by: Shen, Xinjie, et al.
Published: (2025)

Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models
by: Moon, Hyeonseok, et al.
Published: (2025)

CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation
by: Yu, Guangya, et al.
Published: (2025)

ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models
by: Yu, Chung-En Johnny, et al.
Published: (2025)

ORCA: ORchestrating Causal Agent
by: Chung, Joanie Hayoun, et al.
Published: (2025)

A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models
by: Noda, Shiho, et al.
Published: (2025)

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management
by: Guan, Shengyue, et al.
Published: (2026)

Programa de remediação fonológica para alunos com Síndrome de Down: aplicabilidade do método JT na Educação Especial
by: Roberta Moreno Sás
Published: (2012)

Evaluating Computational Accuracy of Large Language Models in Numerical Reasoning Tasks for Healthcare Applications
by: Malghan, Arjun R.
Published: (2025)

A Benchmark for Language Models in Real-World System Building
by: Jin, Weilin, et al.
Published: (2026)

Loss of Response to Anti‐Tumor Necrosis Factor Alpha Therapy in Patients With Inflammatory Bowel Disease: A Real‐World Comparison of Combination Therapy Versus Monotherapy
by: Faris Chater, et al.
Published: (2026)