Gespeichert in:
| Hauptverfasser: | , , , , , , |
|---|---|
| Format: | Preprint |
| Veröffentlicht: |
2025
|
| Schlagworte: | |
| Online-Zugang: | https://arxiv.org/abs/2505.11580 |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| _version_ | 1866912380804399104 |
|---|---|
| author | Liu, Andrew Elaldi, Axel Franklin, Nicholas T Russell, Nathan Atwal, Gurinder S Ban, Yih-En A Viessmann, Olivia |
| author_facet | Liu, Andrew Elaldi, Axel Franklin, Nicholas T Russell, Nathan Atwal, Gurinder S Ban, Yih-En A Viessmann, Olivia |
| contents | Invariant Point Attention (IPA) is a key algorithm for geometry-aware modeling in structural biology, central to many protein and RNA models. However, its quadratic complexity limits the input sequence length. We introduce FlashIPA, a factorized reformulation of IPA that leverages hardware-efficient FlashAttention to achieve linear scaling in GPU memory and wall-clock time with sequence length. FlashIPA matches or exceeds standard IPA performance while substantially reducing computational costs. FlashIPA extends training to previously unattainable lengths, and we demonstrate this by re-training generative models without length restrictions and generating structures of thousands of residues. FlashIPA is available at https://github.com/flagshippioneering/flash_ipa. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2505_11580 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Flash Invariant Point Attention Liu, Andrew Elaldi, Axel Franklin, Nicholas T Russell, Nathan Atwal, Gurinder S Ban, Yih-En A Viessmann, Olivia Machine Learning Artificial Intelligence Biomolecules Invariant Point Attention (IPA) is a key algorithm for geometry-aware modeling in structural biology, central to many protein and RNA models. However, its quadratic complexity limits the input sequence length. We introduce FlashIPA, a factorized reformulation of IPA that leverages hardware-efficient FlashAttention to achieve linear scaling in GPU memory and wall-clock time with sequence length. FlashIPA matches or exceeds standard IPA performance while substantially reducing computational costs. FlashIPA extends training to previously unattainable lengths, and we demonstrate this by re-training generative models without length restrictions and generating structures of thousands of residues. FlashIPA is available at https://github.com/flagshippioneering/flash_ipa. |
| title | Flash Invariant Point Attention |
| topic | Machine Learning Artificial Intelligence Biomolecules |
| url | https://arxiv.org/abs/2505.11580 |