Towards a reliable repeated-measures beads task for assessing the jumping to conclusions bias

•We tested the reliability and repeatability of the standard beads task used to identify jumping to conclusions.•The standard single-sequence task was not reliable or repeatable over repeated measures.•Using additional distractor sequences improved the repeatability and reliability of the task.•When distractor sequences were used, participants were more likely to believe sequences were random.•Discarding first-trial data (a “silent” practice trial) also improved reliability and repeatability. The jumping to conclusions bias (JTC), in which some people gather less information than others before making a decision, has been linked to delusions in psychosis. JTC is usually identified via the beads task, in which a sequence of beads (the “target” sequence) is used to measure the amount of evidence participants require before making a decision. Yet, despite its common use, the reliability of the task has never been properly investigated. We investigated its reliability, and tested an alternate version which used distractor sequences to obfuscate the target sequence. Healthy participants (N = 212) were randomised into two groups. One group completed ten trials using the target sequence, while the other completed ten trials of the target sequence and three distractor sequences. Our data indicated the standard task may not be reliable over repeated measures, but that by including distractor sequences, the task becomes more believable, repeatable, and reliable. Additionally, excluding first-trial data (a “silent” practice trial) also improves repeatability. These improvements to the task are relevant to single trial studies, and will be especially useful to repeated-measures longitudinal, experimental, and treatment studies. Such repeated-measures studies are important for investigating the causal link between JTC and delusions.