Exploring the Relation between Time on Task and Ability in Complex Problem Solving

Complex problem solving (CPS) as an individual skill has gained importance in educational and psychological research, since it requires skills beyond routine reasoning, assessed by computer-based means. However, most of the CPS research has been concerned with the validity of CPS assessments based on students’ responses (i.e., their CPS ability) rather than their response times (i.e., CPS time on task). Therefore, the present investigation was aimed at examining the empirical relation between CPS time on task and CPS ability. On the basis of the data obtained from a representative sample of N =2,000 Finnish ninth graders, we specified Bayesian response time and structural equation models in order to study the relations between CPS time on task, CPS ability, cognitive and motivational constructs as well as school achievement. Our results suggest that (a) Long time on task in CPS and CPS ability are positively related (ρ=.40); (b) reasoning and students’ performance and mastery orientations have differential effects on CPS time on task (βREASON=.25, βPER-GOALS=−.06, βMAST-GOALS=.13) and CPS ability (βREASON=.54, βPER-GOALS=.03, βMAST-GOALS=.03); and (c) CPS time on task predicts school achievement (β=.12–.22), but does not explain variance beyond CPS ability. We conclude that CPS time on task provides a more differentiated perspective on CPS performance, yielding implications for research on and the assessment of CPS. •CPS ability and CPS time on task are positively related (ρ=.40).•Long response times are associated with higher performance in CPS.•Reasoning predicts CPS ability (β=.54) and CPS time on task (β=.25).•Performance and mastery orientations predict CPS time on task only (β=−.06, .13).•CPS time on task predicts school achievement.