Normative responding on cognitive bias tasks: Some evidence for a weak rationality factor that is mostly explained by numeracy and actively open-minded thinking

We conducted two studies with two goals in mind. First, we investigated the dimensionality of several prominent cognitive bias tasks to see whether a single rationality factor can explain a performance on these tasks. Second, we validated this factor by correlating it with a number of constructs from its nomological network (fluid intelligence, numeracy, actively open-minded thinking, conspiracy and superstitious thinking, personality traits) and several real-life outcomes (decision-outcome inventory, job and career satisfaction, peer-rated decision-making quality). Although in both studies one-factor solution was the most appropriate for our tasks, this factor (i.e., “rationality factor”) was weak and only able to account for modest portion of variance among the tasks. Across both studies, the two strongest correlates of this rationality factor were numeracy and actively open-minded thinking. We conclude that cognitive bias tasks are highly heterogeneous, having very little in common. What they had in common, however, was largely underpinned by abilities and dispositions assessed with numeracy and actively open-minded thinking. We discuss how our findings relate to the dual-process theories and offer our view on the place of rationality in a broader model of human intelligence. •A weak rationality factor underpins performance on several prominent cognitive bias tasks.•This factor exhibits relatively high correlations with numeracy and actively open-minded thinking.•Conversely, its correlations with real-life outcomes are mostly low or non-existent.•Both abilities and dispositions are crucial for rationality, as predicted by tripartite theories.•We argue for inclusion of rationality in the CHC model of intelligence.