Reflective assessment in knowledge building using the knowledge connections analyzer

Abstract

This dissertation investigated the design, process and effects of reflective assessment of students’ knowledge building, using the Knowledge Connections Analyzer (KCA) for Knowledge Forum® (van Aalst et al., 2012). The dissertation is a dissertation by articles, comprised of four studies.

Study 1 is a systematic review of the literature on formative assessment interventions in technology-enhanced environments in K-14 education; 39 studies that satisfied the search criteria were analyzed. The findings indicate that assessment interventions with three orientations—continuous formative assessment, embedded formative assessment, and reflective assessment—have been be used to scaffold students’ learning, and that the most effective assessment designs provide students with agency, and help students to actively learn both disciplinary ideas and reasoning, and metacognitive skills. This review positions the current dissertation within a broader theoretical framework and informs the choices that were made when investigating classroom designs.

Study 2 examined nine recent Knowledge Forum databases to understand the state of students’ online discourse, and to identify the challenges in knowledge building that may be addressed by reflective assessment using the KCA. This study found that students contributed many notes and they were seemingly interactive; however, the practices of collaboration, of synthesis and rise-above notes, and of analyzing conceptual progress of ideas were not well-developed in many databases.

Study 3 is a case study that investigated whether and how students with low achievement can benefit from reflective assessment using the KCA in knowledge building. Participants were a class of 20 Grade 11 students taking a visual art course from an experienced teacher. Analysis of students’ online discourse indicated that students were able to take responsibility for advancing collective knowledge. Analysis of qualitative data indicated that reflective assessment may have helped students focus on goals of knowledge building. However, due to the lack of collaborative reflection on the KCA data and the framing of data-driven discourse improvement as collective responsibility, some students continued to experience difficulties such as focusing on their own performance rather than that of the whole community.

Study 4 is a more ethnographic case study and aimed to address the problems identified in Study 3 by implementing a design in which use of the KCA was integrated with other aspects of a pedagogical design for knowledge building. The design had several components, including (a) establishing a collaborative culture and norms; (b) periodic tasks promoting development of cognitive and metacognitive skills; (c) regular and opportunistic use of knowledge-building talks promoting collaborative reflection and thoughtful use of the KCA; and (d) framing the datadriven discourse improvement from a community perspective. 37 students with low achievement from a 9th-grade Visual Arts course participated in the study. Analysis of students’ online discourse showed that students in this class were able to advance the community’s discourse. Analysis of classroom events indicated that the pedagogical design may have influenced the improvement of ideas on Knowledge Forum.

The dissertation’s findings have important implications for the design of technologyrich environments to support learners, and shed light on how teachers can use them to help learners to engage in productive collaborative inquiry. The dissertation also has theoretical value, as it offers insights into the relationships between reflective assessment, collaborative inquiry, and instructional practice, and the potential affordances of knowledge building for students with low achievement.