JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2026
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866913161681043456 |
|---|---|
| author | Larey, Ariel Dahan, Elay Bleiweiss, Amit Kellerman, Raizy Leib, Guy Nayshool, Omri Ofer, Dan Zinger, Tal Dominissini, Dan Rechavi, Gideon Bussola, Nicole Lee, Simon O'Connell, Shane Hoang, Dung Wirth, Marissa Charney, Alexander W. Daniel, Nati Shavit, Yoli |
| author_facet | Larey, Ariel Dahan, Elay Bleiweiss, Amit Kellerman, Raizy Leib, Guy Nayshool, Omri Ofer, Dan Zinger, Tal Dominissini, Dan Rechavi, Gideon Bussola, Nicole Lee, Simon O'Connell, Shane Hoang, Dung Wirth, Marissa Charney, Alexander W. Daniel, Nati Shavit, Yoli |
| contents | Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective. Our framework establishes a new state-of-the-art for GFMs, surpassing the best existing models by bridging generative precision with latent semantic grounding. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is publicly available at https://github.com/NVIDIA-Digital-Bio/JEPA-DNA. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2602_17162 |
| institution | arXiv |
| publishDate | 2026 |
| record_format | arxiv |
| spellingShingle | JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures Larey, Ariel Dahan, Elay Bleiweiss, Amit Kellerman, Raizy Leib, Guy Nayshool, Omri Ofer, Dan Zinger, Tal Dominissini, Dan Rechavi, Gideon Bussola, Nicole Lee, Simon O'Connell, Shane Hoang, Dung Wirth, Marissa Charney, Alexander W. Daniel, Nati Shavit, Yoli Artificial Intelligence Genomics Genomic Foundation Models (GFMs) typically rely on Masked Language Modeling (MLM) or Next-Token Prediction (NTP) to learn the "Laws of Nature". While effective at capturing local syntax, these generative paradigms prioritize token-level reconstruction over high-level functional context. We introduce JEPA-DNA, a model-agnostic continual training framework that integrates a Joint-Embedding Predictive Architecture (JEPA) with traditional generative objectives. By supervising global sequence embeddings in a latent space, JEPA-DNA forces models to predict the functional representations of masked genomic segments, shifting the learning signal from token recovery to semantic alignment. We evaluate JEPA-DNA on 17 diverse genomic benchmark tasks, demonstrating consistent gains in linear probing and zero-shot performance regardless of the underlying GFM architecture or generative objective. Our framework establishes a new state-of-the-art for GFMs, surpassing the best existing models by bridging generative precision with latent semantic grounding. Through extensive ablation studies, we further characterize the synergistic interplay between generative and latent objectives. Our code is publicly available at https://github.com/NVIDIA-Digital-Bio/JEPA-DNA. |
| title | JEPA-DNA: Grounding Genomic Foundation Models through Joint-Embedding Predictive Architectures |
| topic | Artificial Intelligence Genomics |
| url | https://arxiv.org/abs/2602.17162 |