Face to purchase: Predicting consumer choices with structured facial and behavioral traits embedding

Predicting consumers’ purchasing behaviors is critical for targeted advertisement and sales promotion in e-commerce. Human faces are an invaluable source of information for gaining insights into consumer personality and behavioral traits. However, consumer’s faces are largely unexplored in previous research, and the existing face-related studies focus on high-level features such as personality traits while neglecting the business significance of learning from facial data. We propose to predict consumers’ purchases based on their facial features and purchasing histories. We design a semi-supervised model based on a hierarchical embedding network to extract high-level features of consumers and to predict the top-N purchase destinations of a consumer. Our experimental results on a real-world dataset demonstrate the positive effect of incorporating facial information in predicting consumers’ purchasing behaviors. [Display omitted] •A hierarchical model that combines face embedding with structured behavioral traits embedding for purchase prediction.•Feature engineering of structural behavioral traits and multi-faceted face features for generating a graph structure of images.•Selective graph convolution based on Graph Convolutional Network and light Inception for leveraging graph information effectively.