| _version_ | 1866901280130072576 |
|---|---|
| author | Rados, David |
| author_facet | Rados, David |
| contents | <p class="p2">Why does memorising past-paper questions not guarantee success in exams? Why does ChatGPT not memorise the content of the entire internet? Both questions have the same answer: overfitting—the trap of adapting to existing data so precisely, that the advantages of generalisation are lost. </p> <p class="p2">This article introduces overfitting through an example based on predicting daily temperatures. It then develops the underlying mathematics—mean squared error, the Lagrange interpolation theorem, and the bias–variance tradeoff framework. Next, it links overfitting to the training of large language models (LLMs) and argues that algorithmic overfitting in social media can produce “certainty” without scientific truth.</p> <p class="p2">Finally, it connects overfitting to the ideas of Popper, Kolmogorov, Kahneman, Descartes, and Aristotle through the core concepts of the ‘first principles’ and generalisation.</p> <p class="p2">This is an expository article rather than a research paper. It is aimed primarily at high school students interested in mathematics. The approach is interdisciplinary, synthesising ideas from mathematics, artificial intelligence (AI), and epistemology (theory of knowledge).</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_19643001 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | MSE = 0: The Perfect Error of Overfitting Rados, David overfitting Test Dataset Training Dataset Machine learning Generalisation <p class="p2">Why does memorising past-paper questions not guarantee success in exams? Why does ChatGPT not memorise the content of the entire internet? Both questions have the same answer: overfitting—the trap of adapting to existing data so precisely, that the advantages of generalisation are lost. </p> <p class="p2">This article introduces overfitting through an example based on predicting daily temperatures. It then develops the underlying mathematics—mean squared error, the Lagrange interpolation theorem, and the bias–variance tradeoff framework. Next, it links overfitting to the training of large language models (LLMs) and argues that algorithmic overfitting in social media can produce “certainty” without scientific truth.</p> <p class="p2">Finally, it connects overfitting to the ideas of Popper, Kolmogorov, Kahneman, Descartes, and Aristotle through the core concepts of the ‘first principles’ and generalisation.</p> <p class="p2">This is an expository article rather than a research paper. It is aimed primarily at high school students interested in mathematics. The approach is interdisciplinary, synthesising ideas from mathematics, artificial intelligence (AI), and epistemology (theory of knowledge).</p> |
| title | MSE = 0: The Perfect Error of Overfitting |
| topic | overfitting Test Dataset Training Dataset Machine learning Generalisation |
| url | https://doi.org/10.5281/zenodo.19643001 |