Critical dynamics governs deep learning
Fuente:
arXiv
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866910028137496576 |
|---|---|
| author | Vock, Simon Meisel, Christian |
| author_facet | Vock, Simon Meisel, Christian |
| contents | The rapid advances in artificial intelligence (AI) have largely been driven by scaling deep neural networks (DNNs) - increasing model size, data, and computational resources. Yet performance is ultimately governed by network dynamics. The lack of a principled understanding of DNN dynamics beyond heuristic design has contributed to challenges in robustness, suboptimal performance, high energy consumption, and pathologies in continual and AI-generated content learning. In contrast, the human brain appears largely resilient to these problems, and converging evidence suggests this advantage arises from dynamics poised at a critical phase transition. Inspired by this principle, we propose that criticality provides a unifying framework linking structure, dynamics, and function in DNNs. First, analyzing more than 80 state-of-the-art models, we show that a decade of AI progress has implicitly driven successful networks toward criticality - explaining why some architectures succeeded while others failed. Second, we demonstrate that explicitly incorporating criticality into training improves robustness and accuracy while mitigating key limitations of current models. Third, we show that major AI pathologies - including performance degradation in continual learning and model collapse during training on AI-generated data - reflect a loss of critical dynamics. By maintaining networks near criticality, we provide a principled solution to these failures, demonstrating that criticality-based optimization prevents degradation and collapse. Our results establish criticality as a substrate-independent principle of intelligence, connecting AI progress with fundamental principles of brain function, and offering both theoretical insight and practical strategies to ensure long-term DNN performance and resilience as models scale. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2507_08527 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Critical dynamics governs deep learning Vock, Simon Meisel, Christian Neurons and Cognition The rapid advances in artificial intelligence (AI) have largely been driven by scaling deep neural networks (DNNs) - increasing model size, data, and computational resources. Yet performance is ultimately governed by network dynamics. The lack of a principled understanding of DNN dynamics beyond heuristic design has contributed to challenges in robustness, suboptimal performance, high energy consumption, and pathologies in continual and AI-generated content learning. In contrast, the human brain appears largely resilient to these problems, and converging evidence suggests this advantage arises from dynamics poised at a critical phase transition. Inspired by this principle, we propose that criticality provides a unifying framework linking structure, dynamics, and function in DNNs. First, analyzing more than 80 state-of-the-art models, we show that a decade of AI progress has implicitly driven successful networks toward criticality - explaining why some architectures succeeded while others failed. Second, we demonstrate that explicitly incorporating criticality into training improves robustness and accuracy while mitigating key limitations of current models. Third, we show that major AI pathologies - including performance degradation in continual learning and model collapse during training on AI-generated data - reflect a loss of critical dynamics. By maintaining networks near criticality, we provide a principled solution to these failures, demonstrating that criticality-based optimization prevents degradation and collapse. Our results establish criticality as a substrate-independent principle of intelligence, connecting AI progress with fundamental principles of brain function, and offering both theoretical insight and practical strategies to ensure long-term DNN performance and resilience as models scale. |
| title | Critical dynamics governs deep learning |
| topic | Neurons and Cognition |
| url | https://arxiv.org/abs/2507.08527 |