Designing and Evaluating Three Chatbot-Enhanced Activities for a Flipped Graduate Course

Abstract

The purpose of this study is to discuss the implementation and evaluation of chatbot in a flipped graduate course run by the University of Hong Kong. Using the IBM Watson system, three chatbot activities were designed: (a) the first chatbot as a multiple-choice guide for learners’ knowledge exploration; (b) the second chatbot as a case study facilitator for new information elaboration; and (c) the third chatbot as a bibliographic tutor to answer learners’ FAQs related to the learning contents. All three chatbot activities were implemented as pre-class activities, combining with video lectures and online quizzes. Participants interacted with the three chatbots via the course web for an average of 20 minutes. After that, the participants completed a questionnaire and interview to yield insights related to their perceived social presence and interpersonal attraction about the chatbot. The findings of this study will help instructors gain valuable insights about students’ attitude toward the three types of chatbot activities, the content of students’ chatbot interactions, as well as recommendations for improving chatbot use in a flipped course.