Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2512.19097 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913158718816256 |
|---|---|
| author | Han, Danny Dongyeop Gwon, Yonghyeon Lee, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Choi, Jubin Lee, Sebin Bang, Jihyun Lee, Seungju Park, David Keetae Yoo, Shinjae Chung, Chun Kee Cha, Jiook |
| author_facet | Han, Danny Dongyeop Gwon, Yonghyeon Lee, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Choi, Jubin Lee, Sebin Bang, Jihyun Lee, Seungju Park, David Keetae Yoo, Shinjae Chung, Chun Kee Cha, Jiook |
| contents | Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time attention, spatio-temporal resampling, input-conditioned positional embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_19097 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations Han, Danny Dongyeop Gwon, Yonghyeon Lee, Ahhyun Lucy Lee, Taeyang Lee, Seong Jin Choi, Jubin Lee, Sebin Bang, Jihyun Lee, Seungju Park, David Keetae Yoo, Shinjae Chung, Chun Kee Cha, Jiook Machine Learning Artificial Intelligence Intracranial EEG (iEEG) provides direct, millisecond-scale recordings of human neural activity, but reusable representation learning is difficult because electrode layouts, anatomical coverage, referencing schemes, and recording conditions vary across patients and centers. We introduce DIVER-1, a self-supervised iEEG foundation model for variable-input recordings that combines any-variate electrode-time attention, spatio-temporal resampling, input-conditioned positional embeddings, and multi-domain masked reconstruction without assuming a fixed electrode montage. We pretrain two variants, DIVER-1-0.1s and DIVER-1-1s, on 5,310 hours of ECoG and SEEG spanning 352k channel-hours, roughly 54x the BrainTreeBank-based pretraining volume. We evaluate DIVER-1 on two held-out benchmarks: Neuroprobe for naturalistic cognitive decoding and MAYO for seizure detection. On leakage-aware Neuroprobe, DIVER-1-0.1s outperforms prior evaluated iEEG foundation models despite using no BrainTreeBank recordings, the corpus underlying Neuroprobe, during pretraining; it also exceeds the linear spectrogram decoder in mean AUROC and remains competitive with stronger nonlinear baselines, a level prior evaluated iEEG foundation models did not reach. DIVER-1-1s also achieves the top AUROC on MAYO seizure detection. Finally, we conduct, to our knowledge, the first controlled compute-aware scaling study for self-supervised iEEG pretraining, sweeping data scale, subject count, training duration, and model size up to 1.8B parameters. Our results indicate a data-constrained regime: expanding unique recordings and training sufficiently long are more reliable scaling axes than increasing parameter count alone. Code is available at link. |
| title | DIVER-1: Scaling Intracranial EEG Foundation Models for Transferable Representations |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2512.19097 |