One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866918317170622464 |
|---|---|
| author | Jing, Zilin Jeanselme, Vincent Kobayashi, Yuta Lee, Simon A. Pang, Chao Kashyap, Aparajita Li, Yanwei Jiang, Xinzhuo Joshi, Shalmali |
| author_facet | Jing, Zilin Jeanselme, Vincent Kobayashi, Yuta Lee, Simon A. Pang, Chao Kashyap, Aparajita Li, Yanwei Jiang, Xinzhuo Joshi, Shalmali |
| contents | Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages. The sequential nature of EHR, analogous to natural language, has motivated the use of next-token prediction to train prior EHR Foundation Models (FMs) over events. However, this training fails to capture the full structure of EHR. We propose ORA, a marked time-to-event pretraining objective that jointly models event timing and associated measurements. Across multiple datasets, downstream tasks, and model architectures, this objective consistently yields more generalizable representations than next-token prediction and pretraining losses that ignore continuous measurements. Importantly, the proposed objective yields improvements beyond traditional classification evaluation, including better regression and time-to-event prediction. Beyond introducing a new family of FMs, our results suggest a broader takeaway: pretraining objectives that account for EHR structure are critical for expanding downstream capabilities and generalizability |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_00541 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models Jing, Zilin Jeanselme, Vincent Kobayashi, Yuta Lee, Simon A. Pang, Chao Kashyap, Aparajita Li, Yanwei Jiang, Xinzhuo Joshi, Shalmali Machine Learning Clinical events captured in Electronic Health Records (EHR) are irregularly sampled and may consist of a mixture of discrete events and numerical measurements, such as laboratory values or treatment dosages. The sequential nature of EHR, analogous to natural language, has motivated the use of next-token prediction to train prior EHR Foundation Models (FMs) over events. However, this training fails to capture the full structure of EHR. We propose ORA, a marked time-to-event pretraining objective that jointly models event timing and associated measurements. Across multiple datasets, downstream tasks, and model architectures, this objective consistently yields more generalizable representations than next-token prediction and pretraining losses that ignore continuous measurements. Importantly, the proposed objective yields improvements beyond traditional classification evaluation, including better regression and time-to-event prediction. Beyond introducing a new family of FMs, our results suggest a broader takeaway: pretraining objectives that account for EHR structure are critical for expanding downstream capabilities and generalizability |
| title | One Loss to Rule Them All: Marked Time-to-Event for Structured EHR Foundation Models |
| topic | Machine Learning |
| url | https://arxiv.org/abs/2602.00541 |