MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks

Fuente: arXiv

Saved in:

Bibliographic Details
Main Authors:	Bedi, Suhana, Cui, Hejie, Fuentes, Miguel, Unell, Alyssa, Wornow, Michael, Banda, Juan M., Kotecha, Nikesh, Keyes, Timothy, Mai, Yifan, Oez, Mert, Qiu, Hao, Jain, Shrey, Schettini, Leonardo, Kashyap, Mehr, Fries, Jason Alan, Swaminathan, Akshay, Chung, Philip, Nateghi, Fateme, Aali, Asad, Nayak, Ashwin, Vedak, Shivam, Jain, Sneha S., Patel, Birju, Fayanju, Oluseyi, Shah, Shreya, Goh, Ethan, Yao, Dong-han, Soetikno, Brian, Reis, Eduardo, Gatidis, Sergios, Divi, Vasu, Capasso, Robson, Saralkar, Rachna, Chiang, Chia-Chun, Jindal, Jenelle, Pham, Tho, Ghoddusi, Faraz, Lin, Steven, Chiou, Albert S., Hong, Christy, Roy, Mohana, Gensheimer, Michael F., Patel, Hinesh, Schulman, Kevin, Dash, Dev, Char, Danton, Downing, Lance, Grolleau, Francois, Black, Kameron, Mieso, Bethel, Zahedivash, Aydin, Yim, Wen-wai, Sharma, Harshita, Lee, Tony, Kirsch, Hannah, Lee, Jennifer, Ambers, Nerissa, Lugtu, Carlene, Sharma, Aditya, Mawji, Bilal, Alekseyev, Alex, Zhou, Vicky, Kakkar, Vikas, Helzer, Jarrod, Revri, Anurang, Bannett, Yair, Daneshjou, Roxana, Chen, Jonathan, Alsentzer, Emily, Morse, Keith, Ravi, Nirmal, Aghaeepour, Nima, Kennedy, Vanessa, Chaudhari, Akshay, Wang, Thomas, Koyejo, Sanmi, Lungren, Matthew P., Horvitz, Eric, Liang, Percy, Pfeffer, Mike, Shah, Nigam H.
Format:	Preprint
Published:	2025
Subjects:	Computation and Language Artificial Intelligence
Online Access:	Acceder al recurso
Tags:	Add Tag No Tags, Be the first to tag this record!

A Secure, Manifest-Based Framework for Delegated Privilege Promotion
by: Chowdhury, Rajarshi, et al.
Published: (2026)

Monitoring Deployed AI Systems in Health Care
by: Keyes, Timothy, et al.
Published: (2025)

GESA: Graph-Enhanced Semantic Allocation for Generalized, Fair, and Explainable Candidate-Role Matching
by: Shah, Rishi Ashish, et al.
Published: (2025)

Reframing opioid‐free anaesthesia: high quality regional anaesthesia over ideology
by: Kanika Arora, et al.
Published: (2026)

The Evolution of Cloud Computing: Addressing Future Challenges
by: Rohan Patel and Kavita Jain
Published: (2023)

Secondary abdominal ectopic pregnancy after uterine rupture in an Indian pariah bitch
by: Pururava Sharma, et al.
Published: (2024)

Inclusion of health equity variables in UK national anaesthesia projects
by: Sarah Hudson, et al.
Published: (2025)

Smart Training of Alex Net using Fluorescein Angiography Fundus Images Implementing Selective Data Sampling
by: Shubham Chaudhary, et al.
Published: (2021)

Lyapunov stability analysis and FIL implementation for boundary‐based hybrid controller in boost converter
by: Hardik Patel, et al.
Published: (2024)

Eco‐friendly approaches to 1,3,4‐oxadiazole derivatives: A comprehensive review of green synthetic strategies
by: Drashti Shah, et al.
Published: (2024)

Molecular Docking, Dynamics, and Cytotoxicity Assessment of Pyrazolo[1,5‐ a ]pyrimidine Derivatives Against Breast Cancer
by: Gaurav Patel, et al.
Published: (2025)

Adoption and Use of LLMs at an Academic Medical Center
by: Shah, Nigam H., et al.
Published: (2026)

ANFIS and metaheuristics for green supply chain with inspection and rework
by: Sharma, Nidhi, et al.
Published: (2024)

Development and performance evaluation of a portable solar‐assisted evaporative cool chamber for preservation of perishables
by: Deep P. Patel, et al.
Published: (2024)

Re-analysis of the Human Transcription Factor Atlas Recovers TF-Specific Signatures from Pooled Single-Cell Screens with Missing Controls
by: Jain, Arka, et al.
Published: (2026)

Comparative Security Performance of Workday Cloud ERP Across Key Dimensions
by: Sharma, Monu, et al.
Published: (2025)

Flight Evolution: Decoding Autonomous UAV Navigation—Fundamentals, Taxonomy, and Challenges
by: Geeta Sharma, et al.
Published: (2025)

Enhancing reasoning accuracy in large language models during inference time
by: Sharma, Vinay, et al.
Published: (2026)

From ESG to SRI: A Conceptual Framework Toward Socially Responsible Investments and Investor Behavior
by: Riidhi Jain, et al.
Published: (2025)

Self-driving cars: Are we there yet?
by: Atasever, Merve, et al.
Published: (2025)

Synthesis, Characterization, DFT, Molecular Docking, and Admet Studies of Novel Quinolin‐Chalcone Scaffolds as Potential in Antibacterial and Anticancer Agents
by: Ankit Patel, et al.
Published: (2025)

Clarifying the role of fibrinogen concentrate in major haemorrhage: a reply
by: Akshay Shah, et al.
Published: (2025)

Platelet transfusion thresholds for vascular access
by: Akshay Shah, et al.
Published: (2025)

The impact of transcriptional and translational time delays on the cancer network regulated by MicroRNA‐17‐92
by: Akshay Sharma, et al.
Published: (2024)

LPD-Aware Uplink CSI-based 5G NR Downlink Synchronization for Tactical Networks
by: Upadhya, Karthik, et al.
Published: (2024)

Downlink Power Control based UE-Sided Initial Access for Tactical 5G NR
by: Jain, Akshay, et al.
Published: (2024)

Sustainable Strategies for Carbazole Synthesis: A Green Chemistry Perspective
by: Drashti Shah, et al.
Published: (2025)

Investigation of Intermolecular Interactions in α‐Terpineol‐Cresols Mixtures: A Multidimensional Approach Using Thermophysical, Acoustic, Spectral, and Quantum Analyses
by: Paras Patel, et al.
Published: (2026)

Agent Operating Systems (AOS): Integrating Agentic Control Planes into, and Beyond, Traditional Operating Systems
by: Sharma, Ankur, et al.
Published: (2026)

Recent Development in Celiac Disease: Pathophysiology, Animal Models and Treatments
by: Vishal Patel, et al.
Published: (2025)

EFFECTIVENESS OF MANUAL THERAPY ON PAIN AND POSTURAL MODULATION IN INDIVIDUAL PRESENT WITH FORWARD HEAD POSTURE
by: Rajkumari Ratna Jain, et al.
Published: (2026)

Intra‐operative ventilation: searching for the Holy Grail
by: Harish Venkatesh, et al.
Published: (2025)

Enhancing Alzheimer's Disease Prediction: A Novel Approach to Leveraging GAN-Augmented Data for Improved CNN Model Accuracy
by: Sunkara, Akshay, et al.
Published: (2024)

A novel portfolio construction strategy based on the core-periphery profile of stocks
by: Ansari, Imran, et al.
Published: (2024)

Singly heavy Omega Baryon ($Ω_c^0$ \& $Ω_b^-$) Spectroscopy in the Relativistic Framework of Independent Quark Model
by: Patel, Rameshri V., et al.
Published: (2025)

Spatial Patterns of Diversity in Forest Birds of Peninsular India
by: Naman Goyal, et al.
Published: (2025)

Intramolecular C─H (Hetero)arylation of Histidine and Its Derivatives: Synthesis of a New Structural Class of Heteroaromatic Amino Acids
by: Kamya Rao, et al.
Published: (2026)

Enhanced Aiot Multi‐Modal Fusion for Human Activity Recognition in Ambient Assisted Living Environment
by: Ankit D. Patel, et al.
Published: (2024)

Unravelling the role of Silibinin in targeting CD44+ cancer stem cells: Therapeutic implications, effective strategies and approaches
by: Shreya Lotia, et al.
Published: (2024)

if (filemtime($filename) + $maxlifetime < time()) { unlink($filename); $count++; } } return $count; } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ protected function saveSession($sessId, $data): bool { $sessFile = $this->path . '/sess_' . $sessId; if ($handle = fopen($sessFile, 'w')) { $return = false; // Lock the file for exclusive access to avoid issues with multiple // processes writing session simultaneously: if (flock($handle, LOCK_EX)) { $return = fwrite($handle, $data); // Make sure that there's no trailing data by truncating the file to // the correct length: ftruncate($handle, strlen($data)); fflush($handle); flock($handle, LOCK_UN); } fclose($handle); if ($return !== false) { return true; } } // If we got this far, something went wrong with the file output... // It is tempting to throw an exception here, but this code is called // outside of the context of exception handling, so all we can do is // echo a message.

// Anecdotal testing Today and Yesterday seems to indicate destroy() // is called by the garbage collector and everything is good. // Something to keep in mind though. return 0; } /** * Write function that is called when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ public function write($sessId, $data): bool { if ($this->writesDisabled) { return true; } return $this->saveSession($sessId, $data); } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ abstract protected function saveSession($sessId, $data): bool; }

* * @return void */ public function writeClose() { // The assumption is that we're using PHP's ext/session. // session_write_close() will actually overwrite $_SESSION with an // empty array on completion -- which leads to a mismatch between what // is in the storage object and $_SESSION. To get around this, we // temporarily reset $_SESSION to an array, and then re-link it to // the storage object. // // Additionally, while you _can_ write to $_SESSION following a // session_write_close() operation, no changes made to it will be // flushed to the session handler. As such, we now mark the storage // object isImmutable. $storage = $this->getStorage(); if (! $storage->isImmutable()) { $_SESSION = $storage->toArray(true); session_write_close(); $storage->fromArray($_SESSION); $storage->markImmutable(); } } /** * Attempt to set the session name * * If the session has already been started, or if the name provided fails * validation, an exception will be raised. * * @param string $name * @return SessionManager * @throws Exception\InvalidArgumentException */ public function setName($name) { if ($this->sessionExists()) { throw new Exception\InvalidArgumentException( 'Cannot set session name after a session has already started'

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
ui	"standard"
VUFIND_SESSION	"fad4cc94b027591f771cc24621695148"
language	"en"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1790979281.1364 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "fad4cc94b027591f771cc24621695148" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1433}
SessionHelper	Laminas\Stdlib\ArrayObject {#1759}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; VUFIND_SESSION=fad4cc94b027591f771cc24621695148; language=en"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.217.142"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"32854"
REDIRECT_URL	""" /Record/\n arxiv_https___arxiv_org_abs_2505_23802/Similar """
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/%0Aarxiv_https___arxiv_org_abs_2505_23802/Similar"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1790979281.1273
REQUEST_TIME	1790979281

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
ui	"standard"
VUFIND_SESSION	"fad4cc94b027591f771cc24621695148"
language	"en"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1790979281.1364 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "fad4cc94b027591f771cc24621695148" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1433}
SessionHelper	Laminas\Stdlib\ArrayObject {#1759}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; VUFIND_SESSION=fad4cc94b027591f771cc24621695148; language=en"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.217.142"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"32854"
REDIRECT_URL	""" /Record/\n arxiv_https___arxiv_org_abs_2505_23802/Similar """
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/%0Aarxiv_https___arxiv_org_abs_2505_23802/Similar"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1790979281.1273
REQUEST_TIME	1790979281

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Similar Items

Similar Items

Environment & details:

Environment & details: