A Sparse and Spike-timing-based Adaptive Photo Encoder for Augmenting Machine Vision for Spiking Neural Networks

Representation of external stimuli in the form of action potentials or spikes constitute the basis of energy efficient neural computation that emerging spiking neural networks (SNNs) aspire to imitate. With recent evidence suggesting that information in the brain is more often represented by explicit firing times of the neurons rather than mean firing rates, it is imperative to develop novel hardware that can accelerate sparse and spike-timing-based encoding. Here we introduce a medium scale integrated (MSI) circuit comprised of two cascaded three-stage inverters and one XOR logic gate fabricated using a total of 21 memtransistors based on photosensitive two-dimensional (2D) monolayer MoS for spike-timing-based encoding of visual information. We show that different illumination intensities can be encoded into sparse spiking with time-to-first-spike representing the illumination information, i.e., higher intensities invoke earlier spikes and vice versa. In addition, non-volatile and analog programmability in our photo encoder is exploited for adaptive photo encoding that allows expedited spiking under scotopic (low-light) and deferred spiking under photopic (bright-light) conditions, respectively. Finally, low energy expenditure of less than 1 μJ by the 2D memtransistor-based photo encoder highlights the benefits in-sensor and bio-inspired design that can be transformative for the acceleration of SNNs. This article is protected by copyright. All rights reserved.