FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics
Fuente:
arXiv
Enregistré dans:
| Auteurs principaux: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Publié: |
2024
|
| Sujets: | |
| Accès en ligne: | |
| Tags: |
Ajouter un tag
Pas de tags, Soyez le premier à ajouter un tag!
|
| _version_ | 1866909440587857920 |
|---|---|
| author | Duan, ChenRui Zang, Zelin Xu, Yongjie He, Hang Liu, Zihan Li, Siyuan Song, Zijia Zheng, Ju-Sheng Li, Stan Z. |
| author_facet | Duan, ChenRui Zang, Zelin Xu, Yongjie He, Hang Liu, Zihan Li, Siyuan Song, Zijia Zheng, Ju-Sheng Li, Stan Z. |
| contents | Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer, which limits the capture of structurally and functionally relevant gene contexts. Moreover, these approaches struggle with encoding biologically meaningful genes and fail to address the One-to-Many and Many-to-One relationships inherent in metagenomic data. To overcome these challenges, we introduce FGBERT, a novel metagenomic pre-trained model that employs a protein-based gene representation as a context-aware and structure-relevant tokenizer. FGBERT incorporates Masked Gene Modeling (MGM) to enhance the understanding of inter-gene contextual relationships and Triplet Enhanced Metagenomic Contrastive Learning (TMC) to elucidate gene sequence-function relationships. Pre-trained on over 100 million metagenomic sequences, FGBERT demonstrates superior performance on metagenomic datasets at four levels, spanning gene, functional, bacterial, and environmental levels and ranging from 1k to 213k input sequences. Case studies of ATP Synthase and Gene Operons highlight FGBERT's capability for functional recognition and its biological relevance in metagenomic research. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2402_16901 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics Duan, ChenRui Zang, Zelin Xu, Yongjie He, Hang Liu, Zihan Li, Siyuan Song, Zijia Zheng, Ju-Sheng Li, Stan Z. Genomics Artificial Intelligence Machine Learning Metagenomic data, comprising mixed multi-species genomes, are prevalent in diverse environments like oceans and soils, significantly impacting human health and ecological functions. However, current research relies on K-mer, which limits the capture of structurally and functionally relevant gene contexts. Moreover, these approaches struggle with encoding biologically meaningful genes and fail to address the One-to-Many and Many-to-One relationships inherent in metagenomic data. To overcome these challenges, we introduce FGBERT, a novel metagenomic pre-trained model that employs a protein-based gene representation as a context-aware and structure-relevant tokenizer. FGBERT incorporates Masked Gene Modeling (MGM) to enhance the understanding of inter-gene contextual relationships and Triplet Enhanced Metagenomic Contrastive Learning (TMC) to elucidate gene sequence-function relationships. Pre-trained on over 100 million metagenomic sequences, FGBERT demonstrates superior performance on metagenomic datasets at four levels, spanning gene, functional, bacterial, and environmental levels and ranging from 1k to 213k input sequences. Case studies of ATP Synthase and Gene Operons highlight FGBERT's capability for functional recognition and its biological relevance in metagenomic research. |
| title | FGBERT: Function-Driven Pre-trained Gene Language Model for Metagenomics |
| topic | Genomics Artificial Intelligence Machine Learning |
| url | https://arxiv.org/abs/2402.16901 |