Optimizing Rank for High-Fidelity Implicit Neural Representations
Fuente:
arXiv
Guardado en:
| Autores principales: | , , , , , , , , |
|---|---|
| Formato: | Preprint |
| Publicado: |
2025
|
| Materias: | |
| Acceso en línea: | |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| _version_ | 1866917545729064960 |
|---|---|
| author | McGinnis, Julian Hölzl, Florian A. Shit, Suprosanna Bieder, Florentin Friedrich, Paul Mühlau, Mark Menze, Björn Rueckert, Daniel Wiestler, Benedikt |
| author_facet | McGinnis, Julian Hölzl, Florian A. Shit, Suprosanna Bieder, Florentin Friedrich, Paul Mühlau, Mark Menze, Björn Rueckert, Daniel Wiestler, Benedikt |
| contents | Implicit Neural Representations (INRs) based on vanilla Multi-Layer Perceptrons (MLPs) are widely believed to be incapable of representing high-frequency content. This has directed research efforts towards architectural interventions, such as coordinate embeddings or specialized activation functions, to represent high-frequency signals. In this paper, we challenge the notion that the low-frequency bias of vanilla MLPs is an intrinsic, architectural limitation to learn high-frequency content, but instead a symptom of stable rank degradation during training. We empirically demonstrate that regulating the network's rank during training substantially improves the fidelity of the learned signal, rendering even simple MLP architectures expressive. Extensive experiments show that using optimizers like Muon, with high-rank, near-orthogonal updates, consistently enhances INR architectures even beyond simple ReLU MLPs. These substantial improvements hold across a diverse range of domains, including natural and medical images and novel view synthesis, with up to +9 dB PSNR over the same architecture. Code is available at (https://rank-inrs.github.io). |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2512_14366 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Optimizing Rank for High-Fidelity Implicit Neural Representations McGinnis, Julian Hölzl, Florian A. Shit, Suprosanna Bieder, Florentin Friedrich, Paul Mühlau, Mark Menze, Björn Rueckert, Daniel Wiestler, Benedikt Computer Vision and Pattern Recognition Implicit Neural Representations (INRs) based on vanilla Multi-Layer Perceptrons (MLPs) are widely believed to be incapable of representing high-frequency content. This has directed research efforts towards architectural interventions, such as coordinate embeddings or specialized activation functions, to represent high-frequency signals. In this paper, we challenge the notion that the low-frequency bias of vanilla MLPs is an intrinsic, architectural limitation to learn high-frequency content, but instead a symptom of stable rank degradation during training. We empirically demonstrate that regulating the network's rank during training substantially improves the fidelity of the learned signal, rendering even simple MLP architectures expressive. Extensive experiments show that using optimizers like Muon, with high-rank, near-orthogonal updates, consistently enhances INR architectures even beyond simple ReLU MLPs. These substantial improvements hold across a diverse range of domains, including natural and medical images and novel view synthesis, with up to +9 dB PSNR over the same architecture. Code is available at (https://rank-inrs.github.io). |
| title | Optimizing Rank for High-Fidelity Implicit Neural Representations |
| topic | Computer Vision and Pattern Recognition |
| url | https://arxiv.org/abs/2512.14366 |