Augmented reality display device with deep learning sensors

The present invention provides a system comprising non-transitory computer-readable memory storing executable instructions and one or more processors programmed by the executable instructions to at least train a neural network based on a training set comprising a plurality of sensor data as input data and a plurality of corresponding types of events as output data, wherein the neural network comprises an input layer for receiving input of the neural network, a plurality of intermediate layers including a first intermediate layer of the plurality of intermediate layers coupled to the input layer, and a plurality of head components including a head output node coupled to a last intermediate layer of the plurality of intermediate layers through a plurality of head component layers. Fig. 29 WO 2018/039269 PCT/US2017/048068 "Heads" (264) Face Visual Gesture I.D. Semantic Object Lighting SLAM Relocalization Depth Recognition Search Segmentation Detection Estimation Middle Layers (266) Lower Layers (268) Ou tw ard -f ac ing IMU ("world") Depth Sensor Sound, Voice cameras Inputs (270) Hydra Architecture (272) FIG 29