Context-based utterance prediction for assistant systems

In one embodiment, a method includes, by one or more computing systems, receiving, from a client system associated with a user, an initial portion of a user input, wherein the initial portion comprises a partial request, and wherein the initial portion is received while the user is continuing to provide further input, generating, responsive to receiving the initial portion of the user input, one or more speculative queries based on the partial request and a machine-learning predictive model, wherein each speculative query is a predicted complete request based on the partial request, calculating a confidence score for each speculative query based on the predictive model, ranking the one or more speculative queries based on their respective confidence scores and associated costs, executing one or more of the speculative queries based on their ranks, and caching one or more results of the executed one or more speculative queries.