GENERator: A Long-Context Generative Genomic Foundation Model
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2025
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866911412102627328 |
|---|---|
| author | Wu, Wei Li, Qiuyi Zhang, Yuanyuan Zhan, Zhihao Chen, Ruipu Li, Mingyang Fu, Kun Qi, Junyan Bao, Yongzhou Wang, Chao Zhu, Yiheng Zhang, Zhiyun Tang, Jian Feng, Fuli Ye, Jieping Liu, Yuwen Xiong, Hui Wang, Zheng |
| author_facet | Wu, Wei Li, Qiuyi Zhang, Yuanyuan Zhan, Zhihao Chen, Ruipu Li, Mingyang Fu, Kun Qi, Junyan Bao, Yongzhou Wang, Chao Zhu, Yiheng Zhang, Zhiyun Tang, Jian Feng, Fuli Ye, Jieping Liu, Yuwen Xiong, Hui Wang, Zheng |
| contents | The rapid advancement of DNA sequencing has produced vast genomic datasets, yet interpreting and engineering genomic function remain fundamental challenges. Recent large language models have opened new avenues for genomic analysis, but existing approaches are often limited by restricted training scope, constrained generative capability, or prohibitive computational cost. We introduce GENErator, a generative genomic foundation model for long-context DNA modeling, with a context length of 98k nucleotides, pre-trained on 386 billion nucleotides of eukaryotic DNA. Without task-specific fine-tuning, GENERator exhibits strong intrinsic capabilities: unsupervised embedding analyses reveal phylogenetically coherent structure, and sequence recovery benchmarks demonstrate generative accuracy comparable to or exceeding state-of-the-art models with substantially improved computational efficiency. In a zero-shot setting, GENERator achieves competitive variant effect prediction performance relative to alignment-based methods, while remaining fully alignment-free and broadly applicable across species. With task-specific fine-tuning, the model attains leading performance on established genomic benchmarks. We further demonstrate practical generative applications. GENERator can generate protein-coding DNA sequences that translate into structurally plausible proteins and, through a prompt-guided design framework, design cis-regulatory elements with targeted activity profiles, including synthetic super-enhancers validated by high-throughput UMI-STARR-seq assays. Together, these results establish GENERator as an efficient and biologically grounded framework for genomic interpretation and programmable sequence design. Code and supplementary resources are available at https://github.com/GenerTeam/GENERator. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2502_07272 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | GENERator: A Long-Context Generative Genomic Foundation Model Wu, Wei Li, Qiuyi Zhang, Yuanyuan Zhan, Zhihao Chen, Ruipu Li, Mingyang Fu, Kun Qi, Junyan Bao, Yongzhou Wang, Chao Zhu, Yiheng Zhang, Zhiyun Tang, Jian Feng, Fuli Ye, Jieping Liu, Yuwen Xiong, Hui Wang, Zheng Computation and Language Genomics The rapid advancement of DNA sequencing has produced vast genomic datasets, yet interpreting and engineering genomic function remain fundamental challenges. Recent large language models have opened new avenues for genomic analysis, but existing approaches are often limited by restricted training scope, constrained generative capability, or prohibitive computational cost. We introduce GENErator, a generative genomic foundation model for long-context DNA modeling, with a context length of 98k nucleotides, pre-trained on 386 billion nucleotides of eukaryotic DNA. Without task-specific fine-tuning, GENERator exhibits strong intrinsic capabilities: unsupervised embedding analyses reveal phylogenetically coherent structure, and sequence recovery benchmarks demonstrate generative accuracy comparable to or exceeding state-of-the-art models with substantially improved computational efficiency. In a zero-shot setting, GENERator achieves competitive variant effect prediction performance relative to alignment-based methods, while remaining fully alignment-free and broadly applicable across species. With task-specific fine-tuning, the model attains leading performance on established genomic benchmarks. We further demonstrate practical generative applications. GENERator can generate protein-coding DNA sequences that translate into structurally plausible proteins and, through a prompt-guided design framework, design cis-regulatory elements with targeted activity profiles, including synthetic super-enhancers validated by high-throughput UMI-STARR-seq assays. Together, these results establish GENERator as an efficient and biologically grounded framework for genomic interpretation and programmable sequence design. Code and supplementary resources are available at https://github.com/GenerTeam/GENERator. |
| title | GENERator: A Long-Context Generative Genomic Foundation Model |
| topic | Computation and Language Genomics |
| url | https://arxiv.org/abs/2502.07272 |