Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Preprint |
| Published: |
2025
|
| Subjects: | |
| Online Access: | https://arxiv.org/abs/2506.11030 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1866916792436260864 |
|---|---|
| author | As-Saquib, Nazmus Saadat Abeer, A N M Nafiz Chien, Hung-Ta Yoon, Byung-Jun Kumar, Suhas Yi, Su-in |
| author_facet | As-Saquib, Nazmus Saadat Abeer, A N M Nafiz Chien, Hung-Ta Yoon, Byung-Jun Kumar, Suhas Yi, Su-in |
| contents | Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that, despite its widespread success, suffers from key limitations in both biological and hardware perspectives. These include backward error propagation by symmetric weights, non-local credit assignment, and frozen activity during backward passes. We propose Forward Target Propagation (FTP), a biologically plausible and computationally efficient alternative that replaces the backward pass with a second forward pass. FTP estimates layerwise targets using only feedforward computations, eliminating the need for symmetric feedback weights or learnable inverse functions, hence enabling modular and local learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating accuracies competitive with BP on MNIST, CIFAR10, and CIFAR100, as well as effective modeling of long-term dependencies in sequential tasks. Moreover, FTP outperforms BP under quantized low-precision and emerging hardware constraints while also demonstrating substantial efficiency gains over other biologically inspired methods such as target propagation variants and forward-only learning algorithms. With its minimal computational overhead, forward-only nature, and hardware compatibility, FTP provides a promising direction for energy-efficient on-device learning and neuromorphic computing. |
| format | Preprint |
| id |
arxiv_https___arxiv_org_abs_2506_11030 |
| institution | arXiv |
| publishDate | 2025 |
| record_format | arxiv |
| spellingShingle | Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses As-Saquib, Nazmus Saadat Abeer, A N M Nafiz Chien, Hung-Ta Yoon, Byung-Jun Kumar, Suhas Yi, Su-in Machine Learning Artificial Intelligence Training neural networks has traditionally relied on backpropagation (BP), a gradient-based algorithm that, despite its widespread success, suffers from key limitations in both biological and hardware perspectives. These include backward error propagation by symmetric weights, non-local credit assignment, and frozen activity during backward passes. We propose Forward Target Propagation (FTP), a biologically plausible and computationally efficient alternative that replaces the backward pass with a second forward pass. FTP estimates layerwise targets using only feedforward computations, eliminating the need for symmetric feedback weights or learnable inverse functions, hence enabling modular and local learning. We evaluate FTP on fully connected networks, CNNs, and RNNs, demonstrating accuracies competitive with BP on MNIST, CIFAR10, and CIFAR100, as well as effective modeling of long-term dependencies in sequential tasks. Moreover, FTP outperforms BP under quantized low-precision and emerging hardware constraints while also demonstrating substantial efficiency gains over other biologically inspired methods such as target propagation variants and forward-only learning algorithms. With its minimal computational overhead, forward-only nature, and hardware compatibility, FTP provides a promising direction for energy-efficient on-device learning and neuromorphic computing. |
| title | Forward Target Propagation: A Forward-Only Approach to Global Error Credit Assignment via Local Losses |
| topic | Machine Learning Artificial Intelligence |
| url | https://arxiv.org/abs/2506.11030 |