Abstract
(1) Background: A learner's cognitive load in a learning system should be effectively addressed to provide optimal learning processing because the cognitive load explains individual learning differences. However, little empirical research has been conducted into the validation of a cognitive load measurement tool (cognitive load scale, i.e., CLS) suited to online learning systems within higher education. The purpose of this study was to evaluate the psychometric properties of the CLS in an online learning system within higher education through the framework suggested by the Standards for Educational and Psychological Testing. (2) Methods: Data from 800 learners were collected from a cyber-university in South Korea. The age of students ranged from 20 to 64. The CLS was developed, including three components: extraneous cognitive load, intrinsic cognitive load, and germane cognitive load. Then, psychometric properties of the CLS were evaluated including reliability and validity. Evidence relating to content validity, construct validity, and criterion validity were collected. The response pattern of each item was evaluated on the basis of item response theory (IRT). Cronbach's α was computed for reliability. (3) Results: The CLS presented high internal consistency. A three-factor model with extraneous cognitive load, intrinsic cognitive load, and germane cognitive load was suggested by exploratory and confirmatory factor analysis. This three-factor model is consistent with the previous research into the cognitive load in an offline learning environment. Higher levels of the extraneous cognitive load and intrinsic cognitive load were related to lower levels of academic achievement in an online learning environment, but the germane cognitive load was not significantly positively associated with midterm exam scores, though it was significantly related to the final exam scores. IRT analysis showed that the item-fit statistics for all items were acceptable. Lastly, the measurement invariance was examined through differential item functioning analysis (DIF), with the results suggesting that the items did not contain measurement variance in terms of gender. (4) Conclusions: This validation study of the CLS in an online learning environment within higher education assesses psychometric properties and suggests that the CLS is valid and reliable with a three-factor model. There is a need for an evaluation tool to take into account the cognitive load among learners in online learning system because the characteristics of learners within higher education were varied. This CLS will help instructional/curriculum designers and educational instructors to provide more effective instructions and identify individual learning differences in an online learning environment within higher education.