TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866929744531947520 |
|---|---|
| author | Cui, Hejie Unell, Alyssa Chen, Bowen Fries, Jason Alan Alsentzer, Emily Koyejo, Sanmi Shah, Nigam |
| author_facet | Cui, Hejie Unell, Alyssa Chen, Bowen Fries, Jason Alan Alsentzer, Emily Koyejo, Sanmi Shah, Nigam |
| contents | Large language models (LLMs) have emerged as promising tools for assisting in medical tasks, yet processing Electronic Health Records (EHRs) presents unique challenges due to their longitudinal nature. While LLMs' capabilities to perform medical tasks continue to improve, their ability to reason over temporal dependencies across multiple patient visits and time frames remains unexplored. We introduce TIMER (Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records), a framework that incorporate instruction-response pairs grounding to different parts of a patient's record as a critical dimension in both instruction evaluation and tuning for longitudinal clinical records. We develop TIMER-Bench, the first time-aware benchmark that evaluates temporal reasoning capabilities over longitudinal EHRs, as well as TIMER-Instruct, an instruction-tuning methodology for LLMs to learn reasoning over time. We demonstrate that models fine-tuned with TIMER-Instruct improve performance by 7.3% on human-generated benchmarks and 9.2% on TIMER-Bench, indicating that temporal instruction-tuning improves model performance for reasoning over EHR. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2503_04176 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records Cui, Hejie Unell, Alyssa Chen, Bowen Fries, Jason Alan Alsentzer, Emily Koyejo, Sanmi Shah, Nigam Artificial Intelligence Computational Engineering, Finance, and Science Computation and Language Machine Learning 68T50, 68T37 I.2.7; J.3 Large language models (LLMs) have emerged as promising tools for assisting in medical tasks, yet processing Electronic Health Records (EHRs) presents unique challenges due to their longitudinal nature. While LLMs' capabilities to perform medical tasks continue to improve, their ability to reason over temporal dependencies across multiple patient visits and time frames remains unexplored. We introduce TIMER (Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records), a framework that incorporate instruction-response pairs grounding to different parts of a patient's record as a critical dimension in both instruction evaluation and tuning for longitudinal clinical records. We develop TIMER-Bench, the first time-aware benchmark that evaluates temporal reasoning capabilities over longitudinal EHRs, as well as TIMER-Instruct, an instruction-tuning methodology for LLMs to learn reasoning over time. We demonstrate that models fine-tuned with TIMER-Instruct improve performance by 7.3% on human-generated benchmarks and 9.2% on TIMER-Bench, indicating that temporal instruction-tuning improves model performance for reasoning over EHR. |
| title | TIMER: Temporal Instruction Modeling and Evaluation for Longitudinal Clinical Records |
| topic | Artificial Intelligence Computational Engineering, Finance, and Science Computation and Language Machine Learning 68T50, 68T37 I.2.7; J.3 |
| url | https://arxiv.org/abs/2503.04176 |