Understanding eye movement patterns in face recognition using hidden Markov models

Abstract

Recent research has reported substantial individual differences in eye movement patterns in cognitive tasks. Thus, it is important to take these individual differences into account in eye movement data analysis. In this talk, I will present a hidden Markov model (HMM) based approach for eye movement data analysis and how this approach leads to new discoveries in face recognition thus far not revealed by existing methods. In this approach, each individual’s eye movements are modeled with an HMM, including both person-specific regions of interests (ROIs) and transitions among the ROIs. Individual HMMs can be clustered to discover common patterns among individuals, and similarities between individual eye movement patterns can be quantitatively assessed. Through this clustering, we discovered three common patterns in both Asian and Caucasian participants: holistic (looking mostly at the face center), analytic (looking mostly at the two eyes in addition to the face center), and hybrid patterns. The frequency of participants adopting the three patterns did not differ significantly between Asians and Caucasians, suggesting little modulation from culture. Significantly more participants showed similar eye movement patterns when viewing own- and other-race faces than different patterns. In contrast to the scan path theory, which posits that eye movements during learning have to be recapitulated during recognition for the recognition to be successful, participants who used the same or different patterns during learning and recognition did not differ in recognition performance. Interestingly, analytic patterns were associated with better face recognition performance and higher activation in brain regions important for top-down control of visual attention, whereas holistic patterns were associated with aging and lower cognitive status in older adults. This result suggests the possibility of using eye movements as an easily deployable screening assessment for cognitive decline or deficits.