Understanding and overcoming biases in online review systems

This study addresses the issues of social influence and selection biases in the context of online review systems. We propose that one way to reduce these biases is to send email invitations to write a review to a random sample of buyers, and not exposing them to existing reviews while they write their reviews. We provide empirical evidence showing how such a simple intervention from the retailer mitigates the biases by analyzing data from four diverse online retailers over multiple years. The data include both self-motivated reviews, where the reviewer sees other reviews at the time of writing, and retailer-prompted reviews generated by an email invitation to verified buyers, where the reviewer does not see existing reviews. Consistent with previous research on the social influence bias, we find that the star ratings of self-motivated reviews decrease over time (i.e., downward trend), while the star ratings of retailer-prompted reviews remain constant. As predicted by theories on motivation, the self-motivated reviews are shown to be more negative (lower valence), longer, and more helpful, which suggests that the nature of self-motivated and retailer-prompted reviews is distinctively different and the influx of retailer-prompted reviews would enhance diversity in the overall review system. Regarding the selection bias, we found that email invitations can improve the representativeness of reviews by adding a new segment of verified buyers. In sum, implementing appropriate design and policy in online review systems will improve the quality and validity of online reviews and help practitioners provide more credible and representative ratings to their customers. •This study addresses the issues of social influence and selection biases in online review systems.•Email invitations to write a review to a random sample of verified buyers can be one way to reduce these biases.•Analyses of online reviews data from four online retailers show the effects of email invitations on reducing the biases.•Retailer-prompted reviews do not show a sign of social bias, whereas self-motivated reviews exhibit a sign of social bias.•This difference is due to the nature of review submission process (i.e., whether social signals are present or not).•The introduction of email invitations brings in a new group of reviewer population, which improves the representativeness.