Measuring the diversity of recommendations: a preference-aware approach for evaluating and adjusting diversity

While recent research has highlighted the importance of identifying diverse aspects of user preferences in terms of the quality of recommendations, most of the widely used performance measures tend to consider the accuracy of the recommendations and ignore other important aspects such as preference for diversity in recommendations. This is despite the emerging consensus that improving the diversity in recommendations allows users to discover a wider variety of items and encourage them to extend their range of interests in domains such as books, movies, and music. By proposing a novel diversity evaluation metric, this paper aims to address the problem of measuring the diversity with respect to the distribution of preferences for diversity among recommendation system users. We perform several experiments in order to provide a better understanding of the diversity preferences of users and present the results of diversity evaluations of several recommendation methods. These experiments highlight the accuracy–diversity trade-off and show that higher accuracy does not lead to higher performance in terms of the diversity of the recommendations and that the users’ preferred level of diversity should be considered when designing and evaluating recommender systems. This paper also proposes our Diversity Adjustment algorithm that modifies the diversity of recommendations to suit each user’s preferences while preserving the accuracy. Our experiments suggest that diversifying the recommendations without considering the user’s preferences can lead to a dramatic decline in accuracy, while adjusting the diversity based on users’ diversity needs can support recommender systems in maintaining overall accuracy.