Hetero‐Integrated InGaAs Photodiode and Oxide Memristor‐Based Artificial Optical Nerve for In‐Sensor NIR Image Processing

In‐sensor computing is an emerging architectural paradigm that fuses data acquisition and processing within a sensory domain. The integration of multiple functions into a single domain reduces the system footprint while it minimizes the energy and time for data transfer between sensory and computing units. However, it is challenging for a simple and compact image sensor array to achieve both sensing and computing in each pixel. Here, this work demonstrates a focal plane array with a heterogeneously integrated one‐photodiode one‐resistor (1P‐1R)‐based artificial optical neuron that emulates the sensing, computing, and memorization of a biological retina system. This work employs an InGaAs photodiode featuring a high responsivity and a broad spectrum that covers near‐infrared (NIR) signals and employs an HfO2 memristor as the artificial synapse to achieve the computing/memorization in an analog domain. Using the fabricated focal plane array integrated with an artificial neural network, this work performs in‐sensor image identification of finger veins driven by NIR light illumination (≈84 % accuracy). The proposed in‐sensor image computing architecture that broadly covers the NIR spectrum offers widespread application of focal plane array for computer vision, neuromorphic computing, biomedical engineering, etc. In‐sensor computing is realized by hetero‐integrated one‐photodiode one‐resistor (1P‐1R) structure. This work employs a HfO2 memristor capable of storing information via resistive switching. The 1P‐1R array is fabricated and the optical programming and computing of the 1P‐1R array for machine learning applications is demonstrated, including Modified National Institute of Standards and Technology (MNIST) digit classification and vein identification.