Single Pixel Event Tensor: A New Representation Method of Event Stream for Image Reconstruction

Event-based cameras are novel sensors that capture asynchronous changes in brightness with high temporal resolution. The output of an event camera is called an event. Events are a completely different form of data from images, and event streams are difficult to use directly. Reconstructing the event stream into an image is an efficient way to handle events. However, most current image reconstruction methods first transform the event stream into grid-based data, losing a significant amount of information in the temporal dimension in the transformation process. To fully exploit the temporal dimension of the event stream in the reconstruction process, we propose the Single Pixel Event Tensor (SPET), a new representation of the event stream. SPET counts the number and polarity of events for a single pixel within a given time interval, effectively preserving information about the temporal dimension. We built an image reconstruction network called SPETNet, which is based on one-dimensional convolution with SPET as input. SPETNet allows fast reconstruction of images. We trained SPETNet with simulated event streams and verified the effectiveness of SPETNet with real event streams. The validation results show that SPETNet is able to reconstruct greyscale images quickly. In terms of reconstruction quality, SPETNet achieves results comparable to state-of-the-art methods, and in terms of reconstruction speed, the average image reconstruction time for SPETNet is about half that of state-of-the-art method E2VID, SPETNet is significantly faster.