AI as a moral crumple zone: The effects of AI-mediated communication on attribution and trust

AI-mediated communication (AI-MC) represents a new paradigm where communication is augmented or generated by an intelligent system. As AI-MC becomes more prevalent, it is important to understand the effects that it has on human interactions and interpersonal relationships. Previous work tells us that in human interactions with intelligent systems, misattribution is common and trust is developed and handled differently than in interactions between humans. This study uses a 2 (successful vs. unsuccessful conversation) x 2 (standard vs. AI-mediated messaging app) between subjects design to explore whether AI mediation has any effects on attribution and trust. We show that the presence of AI-generated smart replies serves to increase perceived trust between human communicators and that, when things go awry, the AI seems to be perceived as a coercive agent, allowing it to function like a moral crumple zone and lessen the responsibility assigned to the other human communicator. These findings suggest that smart replies could be used to improve relationships and perceptions of conversational outcomes between interlocutors. Our findings also add to existing literature regarding perceived agency in smart agents by illustrating that in this type of AI-MC, the AI is considered to have agency only when communication goes awry. [Display omitted] •AI-generated smart replies in messaging affect interpersonal relationships.•Increased levels of trust in the human communicator are seen in the presence of smart replies.•When conversations go awry, the AI is designated some responsibility that would have been assigned to the human communicator.•In AI-mediated messaging, the AI is considered to have agency only when things go awry.•Smart replies could be used to improve relationships between communicators.