Architecture for context-augmented word embedding

Features are disclosed for training and using a word embedding model configured to receive textual and context data associated with an utterance of a user. A word embedding model can be trained with text data and context data to account for context associated with the text data. The word embedding model can receive an input vector including text data and one or more sets of context data associated with the text data and perform word embedding based on the input vector. In some embodiments, the input vector can include an automatic speech recognition ("ASR") confidence score generated by an ASR model and one or more labels generated by an NLU model. In some embodiments, the input vector can include user information associated with the user.