Imagen de Portada

The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Models

Fuente: arXiv

Guardado en:

Detalles Bibliográficos
Autores principales:	Herambourg, Claudia, Siuda, Dawid, Kopczyńska, Julia, Santos, Joao R. L., Sas, Wojciech, Śmietańska-Nowak, Joanna
Formato:	Preprint
Publicado:	2025
Materias:	Artificial Intelligence
Acceso en línea:	Acceder al recurso
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

Ejemplares similares

RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
por: Yang, Shuo, et al.
Publicado: (2025)

Modelling brain connectomes networks: Solv is a worthy competitor to hyperbolic geometry!
por: Celińska-Kopczyńska, Dorota, et al.
Publicado: (2024)

Editorial
por: Renata Siuda-Ambroziak
Publicado: (2017)

Por que vale a pena pesquisar e analisar as questões religiosas no contexto latino-americano?
por: Renata Siuda-Ambroziak
Publicado: (2017)

Açorianidade e brasilidade nas Festas do Divino: o caso de Viana (ES)
por: Renata Siuda-Ambroziak
Publicado: (2020)

Religião e religiosidade em transformações – dilemas metodológicos
por: Renata Siuda Ambroziak
Publicado: (2017)

Benchmarking ORCA PT-1 Boson Sampler in Simulation
por: Park, Jessica, et al.
Publicado: (2025)

RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
por: Jin, Zhuoran, et al.
Publicado: (2024)

QuarkMedBench: A Real-World Scenario Driven Benchmark for Evaluating Large Language Models
por: Wu, Yao, et al.
Publicado: (2026)

A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
por: Hu, Ruida, et al.
Publicado: (2024)

Theoretical limits of electron and hole doping in single layer graphene from DFT calculations
por: Ciszewski, Dawid, et al.
Publicado: (2025)

Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcare
por: Yang, Yifan, et al.
Publicado: (2024)

angelinamary/ORCA_analysers: ORCA_5.0.4_analysers
por: Angelina Mary
Publicado: (2026)

Evaluating Large Language Models for Real-World Engineering Tasks
por: Heesch, Rene, et al.
Publicado: (2025)

Jackal: A Real-World Execution-Based Benchmark Evaluating Large Language Models on Text-to-JQL Tasks
por: Frank, Kevin, et al.
Publicado: (2025)

Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines
por: Deng, Guifeng, et al.
Publicado: (2025)

MCP-SafetyBench: A Benchmark for Safety Evaluation of Large Language Models with Real-World MCP Servers
por: Zong, Xuanjun, et al.
Publicado: (2025)

Thermal Generation of Spin Current in a Quantum Dot Coupled to Magnetic Insulators
por: Siuda, Emil, et al.
Publicado: (2024)

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs
por: Fonseca, Joao, et al.
Publicado: (2025)

Numerical Aspects of Hyperbolic Geometry
por: Celinska-Kopczynska, Dorota, et al.
Publicado: (2024)

Non-Euclidean Self-Organizing Maps
por: Celińska-Kopczyńska, Dorota, et al.
Publicado: (2021)

Budgeting in an open system
por: Wojciech A. Nowak
Publicado: (2004)

MojoBench: Language Modeling and Benchmarks for Mojo
por: Raihan, Nishat, et al.
Publicado: (2024)

SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding
por: Radwan, Ahmed Y., et al.
Publicado: (2026)

Exponential ergodicity of some Markov dynamical system with application to a Poisson driven stochastic differential equation
por: Czapla, Dawid, et al.
Publicado: (2018)

MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
por: Liang, Hao, et al.
Publicado: (2024)

MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers
por: Luo, Ziyang, et al.
Publicado: (2025)

Modelling IRB Risk‐Weighted Assets: Looking Beyond Stress Tests
por: Josef Švéda, et al.
Publicado: (2025)

MMRC: A Large-Scale Benchmark for Understanding Multimodal Large Language Model in Real-World Conversation
por: Xue, Haochen, et al.
Publicado: (2025)

Measuring Physical-World Privacy Awareness of Large Language Models: An Evaluation Benchmark
por: Shen, Xinjie, et al.
Publicado: (2025)

Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models
por: Moon, Hyeonseok, et al.
Publicado: (2025)

CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation
por: Yu, Guangya, et al.
Publicado: (2025)

ORCA: An Agentic Reasoning Framework for Hallucination and Adversarial Robustness in Vision-Language Models
por: Yu, Chung-En Johnny, et al.
Publicado: (2025)

ORCA: ORchestrating Causal Agent
por: Chung, Joanie Hayoun, et al.
Publicado: (2025)

A Benchmark and Evaluation for Real-World Out-of-Distribution Detection Using Vision-Language Models
por: Noda, Shiho, et al.
Publicado: (2025)

SupChain-Bench: Benchmarking Large Language Models for Real-World Supply Chain Management
por: Guan, Shengyue, et al.
Publicado: (2026)

Programa de remediação fonológica para alunos com Síndrome de Down: aplicabilidade do método JT na Educação Especial
por: Roberta Moreno Sás
Publicado: (2012)

Evaluating Computational Accuracy of Large Language Models in Numerical Reasoning Tasks for Healthcare Applications
por: Malghan, Arjun R.
Publicado: (2025)

A Benchmark for Language Models in Real-World System Building
por: Jin, Weilin, et al.
Publicado: (2026)

Loss of Response to Anti‐Tumor Necrosis Factor Alpha Therapy in Patients With Inflammatory Bowel Disease: A Real‐World Comparison of Combination Therapy Versus Monotherapy
por: Faris Chater, et al.
Publicado: (2026)