NN3: Energy Separation: How the Primes Teach Us to Build Modular Neural Networks — A Principled Approach to Communication-Free Distributed Training
Fuente:
Zenodo
Saved in:
| Main Author: | |
|---|---|
| Format: | Recurso digital |
| Language: | English |
| Published: |
Zenodo
2026
|
| Online Access: | |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866902107746992128 |
|---|---|
| author | Ong, Edwin |
| author_facet | Ong, Edwin |
| contents | <p>Training large neural networks is constrained by the need for synchronized gradient computation across all parameters. Even "embarrassingly parallel" architectures communicate through shared loss functions, creating bottlenecks that limit scalability. The loss function couples what should be independent. It is the bottleneck.</p> <p>We show that the Energy Separation Theorem from the Canvas Model's proof of the Riemann Hypothesis provides a principled path to true modularity. On the prime lattice, the spectral energy separates exactly across primes: E = E_0 + \sum_p E_p, with no cross-terms. Each prime independently minimizes its own energy. The Riemann zeta function—one of the most complex objects in mathematics—is computed by an ensemble of modules that do not communicate, do not share parameters, and do not coordinate their optimization. Each prime consults only its own accumulated history of past phase deformations (Feed-backwards mode), with no cross-prime communication.</p> <p>What this paper provides:</p> <p>· A rigorous condition for loss separation. A loss function is separable when it can be written as \mathcal{L}(W_1, \ldots, W_K) = \mathcal{L}_0 + \sum_i \mathcal{L}_i(W_i) with vanishing cross-derivatives \partial^2 \mathcal{L} / \partial W_i \partial W_j = 0 for i \neq j. When the loss separates, the gradient for module i depends only on \mathcal{L}_i. No gradient communication is required. No synchronization is needed.<br>· Sufficient conditions for architectural separation: Additive output (y = \sum_i f_i(x; W_i)), additive loss (\mathcal{L} = \sum_i \ell(y_i, y_i^*)), disjoint parameters, and known targets for each module. These conditions are architectural choices that guarantee exact separation.<br>· A concrete separable architecture (Sum-of-Experts). The network output is the sum of independent module outputs. Each module trains on its own device with no communication. The training loop is embarrassingly parallel at the loss-function level. The primes are the existence proof that such separation is possible; the architecture is the blueprint for achieving it.<br>· The Feed-Mode distinction. The Energy Separation Theorem applies to the Feed-backwards mode—each prime independently performs gradient descent on its local energy E_p(\theta_p), consulting only its own accumulated history, with no cross-prime communication. This mirrors the Feed-backwards (Steering) mode in neural networks, where each module updates based on its own loss without sharing gradients with other modules.<br>· Empirical validation on a synthetic separable task. The target is the sum of three independent functions. Separable modules achieve better accuracy than a single network with the same total capacity, train in half the time (parallel execution), and the cross-derivatives are identically zero throughout training—verified by explicit computation.<br>· Validation on multi-task MNIST. Three tasks (digit classification, even/odd, greater than 4) are assigned to independent modules. Separable modules match or exceed the accuracy of shared-backbone and separate-network baselines while training each module independently. Training time for K modules on K devices is constant as K grows.<br>· Comparison to existing modular architectures. Mixture of Experts requires a gating network (high communication). Pathways uses shared layers (communication). Ensemble methods achieve loss separation but cannot specialize modules. Separable modules achieve both specialization and independence.<br>· The blind separation problem (addressed in a companion paper). The experiments assume known targets for each module. In real-world tasks without natural decompositions, the decomposition must be discovered from data. The Energy Separation Theorem guarantees that a decomposition exists; it does not provide an algorithm for finding it.</p> <p>Why this matters:</p> <p>The primes train independently. Their energies separate. Their contributions to the Euler product do not interfere. The Riemann zeta function—encoding the distribution of primes, connected to the zeros on the critical line, the subject of the greatest unsolved problem in mathematics for 165 years—is computed by an ensemble of modules that do not communicate, do not share parameters, and do not coordinate their optimization. This is not a metaphor. It is a mathematical fact, proved in the Canvas Model's resolution of the Riemann Hypothesis.</p> <p>If the primes can do it, neural network modules can do it. The architectural challenge now is to solve the blind separation problem—to discover the decomposition when it is not given—and to scale separable architectures to the tasks where communication bottlenecks currently limit what can be built.</p> <p>Keywords: Energy Separation Theorem, modular neural networks, separable loss, Riemann zeta function, prime lattice, distributed training, communication-free optimization, sum-of-experts, blind separation, mixture of experts, Canvas Model, Feed-backwards mode</p> |
| format | Recurso digital |
| id | zenodo_https___doi_org_10_5281_zenodo_20383068 |
| institution | Zenodo |
| language | eng |
| publishDate | 2026 |
| publisher | Zenodo |
| record_format | zenodo |
| spellingShingle | NN3: Energy Separation: How the Primes Teach Us to Build Modular Neural Networks — A Principled Approach to Communication-Free Distributed Training Ong, Edwin <p>Training large neural networks is constrained by the need for synchronized gradient computation across all parameters. Even "embarrassingly parallel" architectures communicate through shared loss functions, creating bottlenecks that limit scalability. The loss function couples what should be independent. It is the bottleneck.</p> <p>We show that the Energy Separation Theorem from the Canvas Model's proof of the Riemann Hypothesis provides a principled path to true modularity. On the prime lattice, the spectral energy separates exactly across primes: E = E_0 + \sum_p E_p, with no cross-terms. Each prime independently minimizes its own energy. The Riemann zeta function—one of the most complex objects in mathematics—is computed by an ensemble of modules that do not communicate, do not share parameters, and do not coordinate their optimization. Each prime consults only its own accumulated history of past phase deformations (Feed-backwards mode), with no cross-prime communication.</p> <p>What this paper provides:</p> <p>· A rigorous condition for loss separation. A loss function is separable when it can be written as \mathcal{L}(W_1, \ldots, W_K) = \mathcal{L}_0 + \sum_i \mathcal{L}_i(W_i) with vanishing cross-derivatives \partial^2 \mathcal{L} / \partial W_i \partial W_j = 0 for i \neq j. When the loss separates, the gradient for module i depends only on \mathcal{L}_i. No gradient communication is required. No synchronization is needed.<br>· Sufficient conditions for architectural separation: Additive output (y = \sum_i f_i(x; W_i)), additive loss (\mathcal{L} = \sum_i \ell(y_i, y_i^*)), disjoint parameters, and known targets for each module. These conditions are architectural choices that guarantee exact separation.<br>· A concrete separable architecture (Sum-of-Experts). The network output is the sum of independent module outputs. Each module trains on its own device with no communication. The training loop is embarrassingly parallel at the loss-function level. The primes are the existence proof that such separation is possible; the architecture is the blueprint for achieving it.<br>· The Feed-Mode distinction. The Energy Separation Theorem applies to the Feed-backwards mode—each prime independently performs gradient descent on its local energy E_p(\theta_p), consulting only its own accumulated history, with no cross-prime communication. This mirrors the Feed-backwards (Steering) mode in neural networks, where each module updates based on its own loss without sharing gradients with other modules.<br>· Empirical validation on a synthetic separable task. The target is the sum of three independent functions. Separable modules achieve better accuracy than a single network with the same total capacity, train in half the time (parallel execution), and the cross-derivatives are identically zero throughout training—verified by explicit computation.<br>· Validation on multi-task MNIST. Three tasks (digit classification, even/odd, greater than 4) are assigned to independent modules. Separable modules match or exceed the accuracy of shared-backbone and separate-network baselines while training each module independently. Training time for K modules on K devices is constant as K grows.<br>· Comparison to existing modular architectures. Mixture of Experts requires a gating network (high communication). Pathways uses shared layers (communication). Ensemble methods achieve loss separation but cannot specialize modules. Separable modules achieve both specialization and independence.<br>· The blind separation problem (addressed in a companion paper). The experiments assume known targets for each module. In real-world tasks without natural decompositions, the decomposition must be discovered from data. The Energy Separation Theorem guarantees that a decomposition exists; it does not provide an algorithm for finding it.</p> <p>Why this matters:</p> <p>The primes train independently. Their energies separate. Their contributions to the Euler product do not interfere. The Riemann zeta function—encoding the distribution of primes, connected to the zeros on the critical line, the subject of the greatest unsolved problem in mathematics for 165 years—is computed by an ensemble of modules that do not communicate, do not share parameters, and do not coordinate their optimization. This is not a metaphor. It is a mathematical fact, proved in the Canvas Model's resolution of the Riemann Hypothesis.</p> <p>If the primes can do it, neural network modules can do it. The architectural challenge now is to solve the blind separation problem—to discover the decomposition when it is not given—and to scale separable architectures to the tasks where communication bottlenecks currently limit what can be built.</p> <p>Keywords: Energy Separation Theorem, modular neural networks, separable loss, Riemann zeta function, prime lattice, distributed training, communication-free optimization, sum-of-experts, blind separation, mixture of experts, Canvas Model, Feed-backwards mode</p> |
| title | NN3: Energy Separation: How the Primes Teach Us to Build Modular Neural Networks — A Principled Approach to Communication-Free Distributed Training |
| url | https://doi.org/10.5281/zenodo.20383068 |