Framework for building, orchestrating and deploying large-scale machine learning applications

Techniques for a framework for building, orchestrating, and deploying complex, large-scale Machine Learning (ML) or deep learning (DL) inference applications is described. A ML application orchestration service is disclosed that enables the construction, orchestration, and deployment of complex ML inference applications in a provider network. The disclosed service provides customers with the ability to define machine learning (ML) models and define transformation operations on data before and/or after being provided to the ML models to construct a complex ML inference application. The service provides a framework for the orchestration (co-ordination) of the workflow logic (e.g., of the request and/or response flows) involved in building and deploying a complex ML inference application in the provider network.