Data mining and dialog act modeling in educational dialog

Abstract

Educational dialogue provides an essential perspective on the learning process, encapsulating the intricate dynamics between teachers and students. Traditional methods of analyzing educational dialogue, however, are labor-intensive and challenging for large-scale analysis. This constraint limits our capacity to fully utilize the growing volumes of educational dialogue data, particularly from online learning platforms. This thesis presents a comprehensive solution to this issue through three pivotal studies, aiming to expedite and enhance the data analysis process to inform and improve educational practices.

The initial study focuses on an exploratory data analysis of tutor engagement within large-scale online tutoring, emphasizing superficial features like the duration of tutoring sessions. While this approach discloses intriguing trends, the lack of dialogue act coding restricts our capability to extract pedagogical insights, highlighting the necessity for more nuanced methods of analysis. In the subsequent study, we employ transformer-based pre-trained language models as automatic dialogue act annotators within two key educational dialogue settings: online tutoring and classroom interactions. Despite achieving high accuracy, these models face challenges in online settings due to the imbalanced distribution of dialogue acts. We conduct a comparison with the state-of-the-art language model, ChatGPT, revealing limitations due to its input constraints. To address these shortcomings, we introduce a data augmentation pipeline, utilizing GPT3.5’s generative capabilities to enhance the automatic annotator’s ability to manage less frequently observed dialogue acts. The third study employs the improved automatic annotator in two practical contexts. Initially, we interpret the classifications made by the automatic annotator into keyword importance, illustrating its potential as an instructional tool for educators to refine their discursive techniques, leading to more effective and enriched pedagogical practice. Secondly, we execute a data mining analysis of the dialogic pattern in online tutoring data, uncovering more profound, pedagogically relevant patterns. These insights validate existing educational theories and contribute to the development of pedagogical thought, thereby enhancing the teaching-learning dynamic.

This thesis underscores the potency of data mining and dialogue act modeling in deriving significant insights from vast educational dialogue data. By facilitating the analysis process, the research not only broadens our understanding of effective teaching and learning strategies but also provides innovative tools to advance educational practices in a digitally enhanced learning landscape.