Abstract
Despite the widespread availability and increasing use of cyberlearning environments, there remains a need for more research about their usefulness in undergraduate education, particularly in STEM education. The process of evaluating the usefulness of a cyberlearning environment is an essential measure of its success and is useful in assisting the design process and ensuring user satisfaction. Unfortunately, there are relatively few empirical studies that provide a comprehensive test of the usefulness of cyberlearning in education. Additionally, there is a lack of standards upon whose usefulness evaluators agree. In this research, we present multiple user studies that can be used to assess the usefulness of a cyberlearning environment used in Computer Science and Software Engineering courses through testing its usability and measuring its utility using user interface and user experience evaluations. Based on these assessments, we propose an evaluation framework to evaluate cyberlearning environments. To help illustrate the framework utility and usability evaluations, we explain them through an example SEP-CyLE (Software Engineering and Programming Cyberlearning Environment). The evaluation techniques used are cognitive walkthroughs with a think-aloud protocol and a heuristic evaluation survey. We further use a network-based analysis to find the statistically significant correlated responses in the heuristic evaluation survey with regard to the students' perceptions of using SEP-CyLE. Our goal is to improve cyberlearning practice and to emphasize the need for evaluating cyberlearning environments with respect to its designated tasks and its users using UI/UX evaluations. Our experiments demonstrated participants were able to utilize SEP-CyLE efficiently to accomplish the tasks we posed to them and to enhance their software development concepts, specifically, software testing. We discovered areas of improvement in the visibility and navigation of SEP-CyLE's current design. We provide our recommendations for improving SEP-CyLE and provide guidance and possible directions for future research on designing cyberlearning environments for computer education.