Systems and methods for content-based indexing of videos at web-scale

Techniques for content-based indexing of videos at web-scale are described. As one example, a computer-implemented method includes receiving a video file, splitting the video file into video frames and audio for the video frames, determining audial features for the audio, clustering each of a plurality of subsets of the audial features into a respective audio centroid for a shared set of bases, determining a first adjacency matrix of distances between the respective audio centroids, determining visual features for the video frames, clustering each of a plurality of subsets of the visual features into a respective video centroid, and determining a second adjacency matrix of distances between the respective video centroids.