Analytics-supported virtual reality content creation for cultural heritage education

Abstract

The act of making is fundamental to our everyday lives. As an innovative pedagogy evolving in recent years, maker activities aim to not only promote students’ acquisition of content knowledge and development of 21st century skills, but also empower students to create public-facing, personally meaningful products. Such creation often requires an iterative and recursive process of ideation, experimentation, and self-evaluation, which aligns with self-regulated learning (SRL) phases and strategies. As a promising form of maker activity, virtual reality (VR) content creation can improve students’ digital content creation competences and can be applied to a variety of STEM and Arts and Humanities disciplines. Research on assessing maker activities, including VR content creation, has focused on their multidimensional learning outcomes, although it is also important for researchers and educators to understand and improve the complex and dynamic learning processes during maker activities. Learning analytics (LA), which can be seen as a marriage between learning science and data science, has the affordances of producing moment-to-moment, fine-grained feedback through student-facing dashboards, thus holding potential in facilitating students’ SRL processes and improving their learning outcomes.

Contextualized in the interdisciplinary domain of cultural heritage education, this study aimed (1) to design and implement a LA-supported, maker-based pedagogy of VR content creation, and (2) to evaluate the effectiveness of LA-enabled support and feedback on students’ learning process and outcomes during the maker activity of VR creation. Based on students’ needs, instructors’ input, and recommendations in the literature, a student-facing LA dashboard was developed to prompt and support students’ self-regulated learning during VR creation on a VR authoring platform. To examine the effects of such LA-enabled support and feedback, we adopted the design-based research (DBR) design and conducted two iterations of classroom quasi-experiments in a university-wide undergraduate general education course in which half of the students were given access to the LA dashboard. 102 and 106 students from diverse demographics and academic backgrounds participated in each of the two iterations, respectively. Multiple sources of data were collected, including surveys, system logs, performance scores, student-created artefacts, learning journal entries, and interviews. We employed a variety of statistical (e.g., Mann–Whitney U tests, Linear Regression), data mining (e.g., process mining, sequential pattern mining, text mining), and qualitative (e.g., thematic coding) methods.

The results show that LA-enabled support and feedback helped students achieve some of the target learning outcomes in the cultural heritage domain, which was in line with and supplemented findings from recent evaluations of LA interventions on cognitive learning outcomes. In terms of students’ SRL process, the LA dashboard helped students plan better at the beginning of the maker activity and motivated them to reflect on their peer-referenced and norm-referenced progress midway and towards the end of the maker activity. These findings demonstrated the efficacy of student-facing LA dashboards in promoting students’ SRL strategies, and showcased the feasibility of leveraging rich qualitative data to offer contextualized explanations of quantitative results from LA evaluations. Overall, this study makes theoretical, methodological, and practical contributions to maker education, learning analytics, and cultural heritage education.