Table of Contents: :: Library Catalog

Saved in:

Bibliographic Details
Main Author:	Heckel, Kade
Format:	Recurso digital
Language:
Published:	Zenodo 2023
Online Access:	https://doi.org/10.5281/zenodo.10620442
Tags:	Add Tag No Tags, Be the first to tag this record!

Table of Contents:

<div> <div>While incredible achievements have been attained using 2nd generation artificial neural networks (ANN) in the past few years, the computational and financial resources required to train and deploy models at scale pose a significant challenge. While endeavors like knowledge distillation, weight pruning, and quantization attempt to reduce computational demand in ANN architectures, they often lead to significant quality loss for smaller models in resource-constrained setups; in contrast, the emerging field of neuromorphic computing addresses these issues by adapting hardware and net- work structures using energy-efficient principles inspired by biology. Using the brain as a blueprint, neuromorphic systems depart from ANN architectures by limiting neurons to only communicate via binary spikes; such a construction allows for the elimination of multiplication of floating point val- ues and opens the door for asynchronous and event-triggered computation, vastly reducing the complexity and power demands of hardware. These benefits are not reaped for free however as the all-or-nothing firing in spiking neural networks (SNN) introduces new challenges in the training process as their discontinuous dynamics yield an ill-defined gradient when performing optimization. Hailed as the 3rd generation of neural computation, recent SNN optimization has been facilitated by surrogate gradient methods which perform Back Propagation Through Time (BPTT) with a smoothed approximation of the spiking activation function’s gradient. However, while surrogate gradients ad- dress the concern that exact gradient methods are ignorant of spike generation or deletion, they still face the issue of dead neurons which once completely silent cannot trigger weight updates. Neuroevolution, a gradient-free optimization approach that had fallen out of favor until recent years due computational constraints, provides a resolution to these challenges by circumventing the non- differentiability of spiking activation functions. This thesis brings two distinct contributions: firstly, the introduction of a new framework called Spyx, which enhances SNN training speed by an or- der of magnitude, and secondly, an exploration of modern neuroevolution’s effectiveness for SNNs, demonstrating competitive performance with gradient-based methods across multiple benchmark datasets</div> </div>

Similar Items