Does video content facilitate or impair comprehension of documentaries? The effect of cognitive abilities and eye movement strategy

Abstract

It remains unclear whether multimedia facilitates or impairs knowledge acquisition. Here we examined whether subtitles and video content facilitate comprehension of documentaries consisting of statements of facts and whether the comprehension depends on participants’ cognitive abilities and eye movement strategies during video watching. We found that subtitles facilitated comprehension regardless of participants’ cognitive abilities or eye movement strategies for video watching. In contrast, with video content but not subtitles, comprehension depended on participants’ auditory working memory, task switching ability, and eye movement strategy. Through the Eye Movement analysis with Hidden Markov Models (EMHMM) method, we found that a centralized (looking mainly at the screen center) eye movement strategy predicted better comprehension as opposed to a distributed strategy (with distributed regions of interest) after contributions from cognitive abilities were controlled. Thus, whether video content facilitates comprehension of documentaries depends on the viewers’ eye movement strategy in addition to cognitive abilities.