Trust and deception with high stakes: Evidence from the friend or foe dataset

Many social interactions rely on the premise of mutual trust, but deception violates trust and poses risk. Empirically examining trust and deception, particularly in high-stakes situations, is challenging but essential for improving the research realism and generalizability. To address this difficulty, we study trusting and deceptive behaviors in a high-stakes situation by using a novel dataset created from an American game show, Friend or Foe (FoF). In the show, a contestant's reward was determined through a trust game modified from the prisoner's dilemma. We explore how numerous human behaviors including facial expressions, gaze, head pose, body motion, language, and socio-demographic attributes, were related to a contestant's trusting or deceptive decision. Using a data-driven approach, we find that the deceivers' (contestants who chose Foe) behavior featured a neutralized face, negative facial emotions, enhanced upper body motion, and language with a lower sense of immediacy and agreeableness. The contestants who chose to trust (chose Friend) exhibited opposite behavioral patterns. Socio-demographic factors such as age, height, and facial attractiveness were also associated with a contestant's choice. Combining multimodal information, machine learning classifiers could predict the contestant's choice with an accuracy about 25% greater than earlier reported human accuracy. We contribute to both trust and deception literature by examining the generalizability of trusting and deceptive behaviors to a new high-stakes scenario. We also add to the decision support literature by showing the superior predictive performances of combining behavioral and socio-demographic features. Furthermore, we contribute to the academic community by introducing the FoF dataset. •Lab studies on trust and deception have realism and generalizability concerns.•Trusting and deceptive behaviors in a high-stakes prisoner's dilemma are studied.•Facial, kinesic and verbal cues to trusting and deceptive decisions are identified.•Machine learning classifiers can predict decisions more accurately than humans.