Fast nearest neighbor search for output generation of convolutional neural networks

In one embodiment, a method includes receiving an input vector corresponding to a query at a neural network model comprising a plurality of layers, wherein the plurality of layers comprise a last layer associated with a mapping matrix, generating a binary matrix based on the mapping matrix, an identity matrix, and one or more Gaussian vectors, generating an integer vector based on the binary matrix and a binary vector associated with the input vector, identifying a plurality of indices corresponding to a plurality of top values of the integer vector for the integer vector, generating an output vector based on the input vector and a plurality of rows of the mapping matrix, wherein the plurality of rows is associated with the plurality of identified indices, respectively, and determining the query is associated with one or more classes based on the output vector.