Automatic identification of storytelling responses to past‐behavior interview questions via machine learning

Structured interviews often feature past‐behavior questions, where applicants are asked to tell a story about past work experience. Applicants often experience difficulties producing such stories. Automatic analyses of applicant behavior in responding to past‐behavior questions may constitute a basis for delivering feedback and thus helping them improve their performance. We used machine learning algorithms to predict storytelling in transcribed speech of participants responding to past‐behavior questions in a simulated selection interview. Responses were coded as to whether they featured a story or not. For each story, utterances were also manually coded as to whether they described the situation, the task/action performed, or results obtained. The algorithms predicted whether a response features a story or not (best accuracy: 78%), as well as the count of situation, task/action, and response utterances. These findings contribute to better automatic identification of verbal responses to past‐behavior questions and may support automatic provision of feedback to applicants about their interview performance. Practitioner points Past‐behavior questions constitute a best practice in selection interviews. Past‐behavior questions invite applicants to tell a story about what they did in a past work‐related situation. Applicants often fail to produce stories, and when they do, they tend to focus on describing the situation rather than what they did and what results they obtained. Coaching may help them improve their responses but is costly. Using machine learning, we accurately predict storytelling responses to past‐behavior questions and their narrative content from transcripts of applicant responses. It is feasible to design systems for automatic delivery of feedback to applicants to improve their responses to past‐behavior questions.