From Clicks to Constructs: An Examination of Validity Evidence of Game-Based Indicators Derived from Theory

A critical issue in using fine-grained gameplay data to measure learning processes is the development of indicators and the algorithms used to derive such indicators. Successful development--that is, developing traceable, interpretable, and sensitive-to-learning indicators--requires understanding the underlying theory, how the theory is instantiated under different game conditions, and programming that considers these exigencies using fine-grained gameplay data. This study examines preliminary validity evidence on various game-based indicators (GBIs) developed from theoretical frameworks and not from unsupervised feature extraction methods. Examples of indicators are presented that represent three types of indicators: Common indicators, distance-based indicators, and games-specific indicators. For each example, the theoretical background is presented briefly, followed by a description of how the indicator design flows from the theory. Correlational analyses show how the GBIs relate to external criterion measures. Limitations and next steps are discussed.