Intraoperative Registration by Cross-Modal Inverse Neural Rendering
Fuente:
arXiv
Saved in:
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2024
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866913506402500608 |
|---|---|
| author | Fehrentz, Maximilian Azampour, Mohammad Farid Dorent, Reuben Rasheed, Hassan Galvin, Colin Golby, Alexandra Wells, William M. Frisken, Sarah Navab, Nassir Haouchine, Nazim |
| author_facet | Fehrentz, Maximilian Azampour, Mohammad Farid Dorent, Reuben Rasheed, Hassan Galvin, Colin Golby, Alexandra Wells, William M. Frisken, Sarah Navab, Nassir Haouchine, Nazim |
| contents | We present in this paper a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. This disentanglement is achieved by controlling a Neural Radiance Field's appearance with a multi-style hypernetwork. Once trained, the implicit neural representation serves as a differentiable rendering engine, which can be used to estimate the surgical camera pose by minimizing the dissimilarity between its rendered images and the target intraoperative image. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration. Code and additional resources can be found at https://maxfehrentz.github.io/style-ngp/. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2409_11983 |
| institution | arXiv |
| publishDate | 2024 |
| record_format | arxiv |
| spellingShingle | Intraoperative Registration by Cross-Modal Inverse Neural Rendering Fehrentz, Maximilian Azampour, Mohammad Farid Dorent, Reuben Rasheed, Hassan Galvin, Colin Golby, Alexandra Wells, William M. Frisken, Sarah Navab, Nassir Haouchine, Nazim Computer Vision and Pattern Recognition We present in this paper a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. This disentanglement is achieved by controlling a Neural Radiance Field's appearance with a multi-style hypernetwork. Once trained, the implicit neural representation serves as a differentiable rendering engine, which can be used to estimate the surgical camera pose by minimizing the dissimilarity between its rendered images and the target intraoperative image. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration. Code and additional resources can be found at https://maxfehrentz.github.io/style-ngp/. |
| title | Intraoperative Registration by Cross-Modal Inverse Neural Rendering |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2409.11983 |