Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization
Fuente:
arXiv
Gespeichert in:
| Hauptverfasser: | , , , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2024
|
| Schlagworte: | |
| Online-Zugang: | |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866910586872266752 |
|---|---|
| author | Chen, Dingshuo Li, Zhixun Ni, Yuyan Zhang, Guibin Wang, Ding Liu, Qiang Wu, Shu Yu, Jeffrey Xu Wang, Liang |
| author_facet | Chen, Dingshuo Li, Zhixun Ni, Yuyan Zhang, Guibin Wang, Ding Liu, Qiang Wu, Shu Yu, Jeffrey Xu Wang, Liang |
| contents | With the emergence of various molecular tasks and massive datasets, how to perform efficient training has become an urgent yet under-explored issue in the area. Data pruning (DP), as an oft-stated approach to saving training burdens, filters out less influential samples to form a coreset for training. However, the increasing reliance on pretrained models for molecular tasks renders traditional in-domain DP methods incompatible. Therefore, we propose a Molecular data Pruning framework for enhanced Generalization (MolPeg), which focuses on the source-free data pruning scenario, where data pruning is applied with pretrained models. By maintaining two models with different updating paces during training, we introduce a novel scoring function to measure the informativeness of samples based on the loss discrepancy. As a plug-and-play framework, MolPeg realizes the perception of both source and target domain and consistently outperforms existing DP methods across four downstream tasks. Remarkably, it can surpass the performance obtained from full-dataset training, even when pruning up to 60-70% of the data on HIV and PCBA dataset. Our work suggests that the discovery of effective data-pruning metrics could provide a viable path to both enhanced efficiency and superior generalization in transfer learning. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_01081 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization Chen, Dingshuo Li, Zhixun Ni, Yuyan Zhang, Guibin Wang, Ding Liu, Qiang Wu, Shu Yu, Jeffrey Xu Wang, Liang Machine Learning Artificial Intelligence Biomolecules With the emergence of various molecular tasks and massive datasets, how to perform efficient training has become an urgent yet under-explored issue in the area. Data pruning (DP), as an oft-stated approach to saving training burdens, filters out less influential samples to form a coreset for training. However, the increasing reliance on pretrained models for molecular tasks renders traditional in-domain DP methods incompatible. Therefore, we propose a Molecular data Pruning framework for enhanced Generalization (MolPeg), which focuses on the source-free data pruning scenario, where data pruning is applied with pretrained models. By maintaining two models with different updating paces during training, we introduce a novel scoring function to measure the informativeness of samples based on the loss discrepancy. As a plug-and-play framework, MolPeg realizes the perception of both source and target domain and consistently outperforms existing DP methods across four downstream tasks. Remarkably, it can surpass the performance obtained from full-dataset training, even when pruning up to 60-70% of the data on HIV and PCBA dataset. Our work suggests that the discovery of effective data-pruning metrics could provide a viable path to both enhanced efficiency and superior generalization in transfer learning. |
| title | Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization |
| topic | Machine Learning Artificial Intelligence Biomolecules |
| url | https://arxiv.org/abs/2409.01081 |