Attention aware virtual assistant clearance

Systems and processes for operating an intelligent automated assistant are provided. An exemplary process includes initiating a virtual assistant session in response to receiving a user input. In accordance with initiating the virtual assistant session, the process includes determining, based on data obtained using one or more sensors of the electronic device, whether one or more criteria representing that the expressed user is not of interest are met. In accordance with a determination that the one or more criteria representing that the expressed user is not interested are satisfied prior to a first time, the process includes automatically deactivating the virtual assistant session prior to the first time. The first time is defined by a setting of the electronic device. In accordance with a determination that the one or more criteria representing that the expressed user is not of interest are not satisfied prior to the first time, the process includes automatically deactivating the virtual assistant session