Designing a recommender system to promote self-regulated learning in online contexts : a two-year design-based study

Abstract

With the outbreak of the COVID-19 pandemic, many institutions transferred to online teaching in response to urgent demands to continue school education. In the absence of face-to-face support from instructors, online learning presents unique challenges for learners who lack self-regulation skills. Self-regulated learning (SRL) refers to a learner’s ability to self-monitor and adjust their learning behaviors to achieve learning goals. However, previous studies have reported that many students struggle with self-regulation in online learning, indicating the need to provide students with additional support for conducting SRL.

The present study developed a recommender system to promote students’ SRL skills. This study adopted a design-based research methodology to iteratively design, implement, and evaluate different SRL recommendation designs. The main study consisted of three cycles, which were conducted in a fully online graduate course. Students enrolled in this course in three semesters were involved in this study (i.e., n = 29 in the 2021 spring semester, n = 25 in the 2022 spring semester, and n = 27 in the 2022 fall semester). Each cycle adopted mixed methods for data collection. The qualitative results generated from student interviews were assessed immediately after each cycle to refine the next cycle. The quantitative results were analyzed after the last cycle of the study to examine the effect of different SRL recommendation designs on students’ SRL skills and learning performance.

In the first cycle of the study, a recommendation approach was proposed to enable the students to self-report their use of SRL strategies and receive recommendations on how to improve for future learning. During the interviews, students affirmed that the recommendation approach raised their awareness of how to conduct SRL, but they stated that the format and rules for providing recommendations could be improved. Therefore, in the second cycle, a prototype recommender system was developed, which enabled students to learn SRL while receiving recommendations on what SRL strategies they should improve. The students found the system helpful for them learn and practice SRL skills. Their major concerns were related to the criteria used to make personalized recommendations and connecting with the course content. To refine the system, in the third cycle, a course section was added to the system and provided recommendations by tracking students’ online learning behaviors in real time. The students considered the system to be comprehensive and easy to use. Future research could advance this system by exploring ways to sustain students’ interest in persistently engaging in SRL activities.

The results of SRL pre- and post-tests across the three cycles showed that different recommendation designs could significantly affect students’ SRL skills. The refined system adopted in the last cycle displayed the greatest potential to promote students’ SRL skills, although no significant effect was found on students’ learning performance. Overall, this study explored the potential of using a recommender system to promote SRL with sufficient evaluations and empirical evidence. The findings and implications should be valuable for future research.