Training ultra-low-power spiking neural networks for neuromorphic IoT vision sensing and recognition

Computer vision and machine learning benefit from a growing interest in solving complex problems where computers are expected to automatically learn from data [1]. Despite impressive performances [2, 3], these algorithms are often energy-intensive [2, 4, 5] and fall far short of the robustness and versatility of biological systems [6, 7, 8, 9, 10, 11, 12, 13]. Over the last decades, computers have become more powerful [14]. However, this exponential growth in technology scaling seems to come to an end [14, 15], calling for new computing systems and processing algorithms. Indeed, current von Neumann architectures [16] have inherent hardware limitations [10] preventing them from efficiently implementing general-purpose real-time autonomous and energy-efficient systems, unlike biological brains. This drives a paradigm shift towards alternative architectures that could bring silicon systems closer to the efficiency of biological ones. To address these challenges, an ongoing research field called neuromorphic engineering aims at developing practical solutions by drawing inspiration from the brain [17, 18]. Although neuromorphic sensors and processors already exist [7, 18, 19, 20, 21, 22, 23, 24], conventional imaging algorithms cannot be directly mapped to these hardware platforms as their architecture is fundamentally different from von Neumann computers [3]. Consequently, new algorithms are needed to process data efficiently on neuromorphic platforms. This thesis aims at investigating several ways of training spiking neural networks (SNNs) in order to classify accurately samples from real-world neuromorphic datasets. At the same time, neuromorphic processor(s) developed at UCLouvain come as a support hardware framework, which defines chip related constraints. The main achievements of this thesis range from the development of a Python spiking simulator for inference in the spiking domain, the generation of SSP-MNIST which is a synthetic spiking version of MNIST, to an adapted integrate-and-fire neuron model thought to improve the classification accuracy of SNNs. To the best of our knowledge, this was the first time that the proposed approach was applied to the real-world neuromorphic dataset N-CARS. Finally, even though the proposed approach did not outperform state-of-the-art techniques, it achieved good classification accuracy despite being subject to hardware constraints (e.g. 98.19% on SSP-MNIST, 96.83% on N-MNIST and 81.97% on N-CARS).