USING MACHINE-LEARNED MODELS TO THROTTLE CONTENT

Techniques for using machine-learned models to throttle content are provided. In one technique, based on multiple selection events, a distribution of relevance measures is computed, where the relevance measures are associated with the content item selection events. The relevance measures may be generated by one or more machine-learned models. Based on the computed distribution, a threshold relevance measure is computed. Thereafter, a request for content is received over a computer network. In response, a computer system performs, in real-time, multiple steps. For example, an identity of an entity that is associated with the request is identified and, based on that identity, multiple content delivery groups are identified. A relevance measure of one of the content delivery groups relative to the entity is determined and compared to the threshold relevance measure. The content delivery group is selected only after determining that the relevance measure is above the threshold relevance measure.