EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866912093894082560 |
|---|---|
| author | Pan, Yi Jiang, Hanqi Chen, Junhao Li, Yiwei Zhao, Huaqin Zhou, Yifan Shu, Peng Wu, Zihao Liu, Zhengliang Zhu, Dajiang Li, Xiang Abate, Yohannes Liu, Tianming |
| author_facet | Pan, Yi Jiang, Hanqi Chen, Junhao Li, Yiwei Zhao, Huaqin Zhou, Yifan Shu, Peng Wu, Zihao Liu, Zhengliang Zhu, Dajiang Li, Xiang Abate, Yohannes Liu, Tianming |
| contents | Neuromorphic computing has emerged as a promising energy-efficient alternative to traditional artificial intelligence, predominantly utilizing spiking neural networks (SNNs) implemented on neuromorphic hardware. Significant advancements have been made in SNN-based convolutional neural networks (CNNs) and Transformer architectures. However, neuromorphic computing for the medical imaging domain remains underexplored. In this study, we introduce EG-SpikeFormer, an SNN architecture tailored for clinical tasks that incorporates eye-gaze data to guide the model's attention to the diagnostically relevant regions in medical images. Our developed approach effectively addresses shortcut learning issues commonly observed in conventional models, especially in scenarios with limited clinical data and high demands for model reliability, generalizability, and transparency. Our EG-SpikeFormer not only demonstrates superior energy efficiency and performance in medical image prediction tasks but also enhances clinical relevance through multi-modal information alignment. By incorporating eye-gaze data, the model improves interpretability and generalization, opening new directions for applying neuromorphic computing in healthcare. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2410_09674 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis Pan, Yi Jiang, Hanqi Chen, Junhao Li, Yiwei Zhao, Huaqin Zhou, Yifan Shu, Peng Wu, Zihao Liu, Zhengliang Zhu, Dajiang Li, Xiang Abate, Yohannes Liu, Tianming Image and Video Processing Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing Neuromorphic computing has emerged as a promising energy-efficient alternative to traditional artificial intelligence, predominantly utilizing spiking neural networks (SNNs) implemented on neuromorphic hardware. Significant advancements have been made in SNN-based convolutional neural networks (CNNs) and Transformer architectures. However, neuromorphic computing for the medical imaging domain remains underexplored. In this study, we introduce EG-SpikeFormer, an SNN architecture tailored for clinical tasks that incorporates eye-gaze data to guide the model's attention to the diagnostically relevant regions in medical images. Our developed approach effectively addresses shortcut learning issues commonly observed in conventional models, especially in scenarios with limited clinical data and high demands for model reliability, generalizability, and transparency. Our EG-SpikeFormer not only demonstrates superior energy efficiency and performance in medical image prediction tasks but also enhances clinical relevance through multi-modal information alignment. By incorporating eye-gaze data, the model improves interpretability and generalization, opening new directions for applying neuromorphic computing in healthcare. |
| title | EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis |
| topic | Image and Video Processing Computer Vision and Pattern Recognition Machine Learning Neural and Evolutionary Computing |
| url | https://arxiv.org/abs/2410.09674 |