Algorithmic Free Energy Minimization: A New Foundation for Parsimonious Artificial Intelligence
Fuente:
Zenodo
Salvato in:
| Autore principale: | |
|---|---|
| Natura: | Recurso digital |
| Lingua: | inglese |
| Pubblicazione: |
Zenodo
2025
|
| Soggetti: | |
| Accesso online: | |
| Tags: |
Aggiungi Tag
Nessun Tag, puoi essere il primo ad aggiungerne!!
|
| _version_ | 1866901678578466816 |
|---|---|
| author | Fathi, Kevin |
| author_facet | Fathi, Kevin |
| contents | <p>The dominant paradigm in modern Artificial Intelligence relies on maximizing performance metrics using massively overparameterized models trained via brute-force computation. This approach suffers from fundamental limitations in generalization, efficiency, and interpretability. We propose a new foundation based on the minimization of Algorithmic Free Energy (AFE). Derived from Algorithmic Information Theory, AFE provides a rigorous formalization of Occam's razor by unifying model complexity (Kolmogorov complexity) and data fit within a single objective: the expected total description length. We derive novel generalization bounds using the PAC-Bayesian framework under the normalized universal prior. We rigorously define this prior and the corresponding likelihood via STOP-symbol completion, ensuring uniform $O(1)$ constants. We prove two key results: (1) for bounded losses, the generalization gap is rigorously controlled by the expected algorithmic complexity of the hypotheses; (2) for the KL risk (log loss), the expected generalization error is tightly bounded by the normalized AFE (minus posterior entropy). These results demonstrate that minimizing AFE directly tightens provable upper bounds on generalization error. Although AFE is uncomputable, we prove that computable surrogates (Computable Free Energy, CFE) provide practical objectives that yield valid (possibly looser) generalization bounds. AFE minimization provides a principled path toward AI that is inherently parsimonious, efficient, and robust.</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_17560581 |
| institution | Zenodo |
| language | eng |
| publishDate | 2025 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | Algorithmic Free Energy Minimization: A New Foundation for Parsimonious Artificial Intelligence Fathi, Kevin Algorithmic free energy Kolmogorov complexity Algorithmic information theory Minimum description length PAC-Bayesian bounds Generalization error Universal prior STOP completion Variational free energy Occam's razor Parsimonious artificial intelligence Model complexity Description length Sample complexity Computable free energy Compression-based learning Bayesian inference Deep learning theory <p>The dominant paradigm in modern Artificial Intelligence relies on maximizing performance metrics using massively overparameterized models trained via brute-force computation. This approach suffers from fundamental limitations in generalization, efficiency, and interpretability. We propose a new foundation based on the minimization of Algorithmic Free Energy (AFE). Derived from Algorithmic Information Theory, AFE provides a rigorous formalization of Occam's razor by unifying model complexity (Kolmogorov complexity) and data fit within a single objective: the expected total description length. We derive novel generalization bounds using the PAC-Bayesian framework under the normalized universal prior. We rigorously define this prior and the corresponding likelihood via STOP-symbol completion, ensuring uniform $O(1)$ constants. We prove two key results: (1) for bounded losses, the generalization gap is rigorously controlled by the expected algorithmic complexity of the hypotheses; (2) for the KL risk (log loss), the expected generalization error is tightly bounded by the normalized AFE (minus posterior entropy). These results demonstrate that minimizing AFE directly tightens provable upper bounds on generalization error. Although AFE is uncomputable, we prove that computable surrogates (Computable Free Energy, CFE) provide practical objectives that yield valid (possibly looser) generalization bounds. AFE minimization provides a principled path toward AI that is inherently parsimonious, efficient, and robust.</p> |
| title | Algorithmic Free Energy Minimization: A New Foundation for Parsimonious Artificial Intelligence |
| topic | Algorithmic free energy Kolmogorov complexity Algorithmic information theory Minimum description length PAC-Bayesian bounds Generalization error Universal prior STOP completion Variational free energy Occam's razor Parsimonious artificial intelligence Model complexity Description length Sample complexity Computable free energy Compression-based learning Bayesian inference Deep learning theory |
| url | https://doi.org/10.5281/zenodo.17560581 |