Cliqueformer: Model-Based Optimization with Structured Transformers
Fuente:
arXiv
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2024
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866911526887096320 |
|---|---|
| author | Kuba, Jakub Grudzien Abbeel, Pieter Levine, Sergey |
| author_facet | Kuba, Jakub Grudzien Abbeel, Pieter Levine, Sergey |
| contents | Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based optimization (MBO) problems. While predictive models may not directly translate to effective design, recent MBO algorithms incorporate reinforcement learning and generative modeling approaches. Meanwhile, theoretical work suggests that exploiting the target function's structure can enhance MBO performance. We present Cliqueformer, a transformer-based architecture that learns the black-box function's structure through functional graphical models (FGM), addressing distribution shift without relying on explicit conservative approaches. Across various domains, including chemical and genetic design tasks, Cliqueformer demonstrates superior performance compared to existing methods. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_13106 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Cliqueformer: Model-Based Optimization with Structured Transformers Kuba, Jakub Grudzien Abbeel, Pieter Levine, Sergey Machine Learning Artificial Intelligence Large neural networks excel at prediction tasks, but their application to design problems, such as protein engineering or materials discovery, requires solving offline model-based optimization (MBO) problems. While predictive models may not directly translate to effective design, recent MBO algorithms incorporate reinforcement learning and generative modeling approaches. Meanwhile, theoretical work suggests that exploiting the target function's structure can enhance MBO performance. We present Cliqueformer, a transformer-based architecture that learns the black-box function's structure through functional graphical models (FGM), addressing distribution shift without relying on explicit conservative approaches. Across various domains, including chemical and genetic design tasks, Cliqueformer demonstrates superior performance compared to existing methods. |
| title | Cliqueformer: Model-Based Optimization with Structured Transformers |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2410.13106 |