Understanding eye movement patterns in face recognition using hidden Markov models

Abstract

Recent research has reported substantial individual differences in eye movement patterns in visual tasks. Here we present a hidden Markov model (HMM) based approach for eye movement data analysis that takes individual differences into account. In this approach, each individual’s eye movements are modeled with an HMM, including both person-specific regions of interests (ROIs) and transitions among the ROIs. Individual HMMs can be clustered to discover common patterns, and similarities between individual patterns can be quantitatively assessed. Through this approach, we discovered two common patterns for viewing faces: holistic (looking mostly at the face center) and analytic (looking mostly at the two eyes). Most participants used holistic patterns for face learning and analytic patterns for face recognition. Participants who used the same or different patterns during learning and recognition did not differ in recognition performance, in contrast to the scan path theory. Interestingly, analytic patterns were associated with better face recognition performance and higher activation in brain regions important for top-down control of visual attention, whereas holistic patterns were associated with ageing and lower cognitive status in older adults. This result suggests the possibility of using eye movements as an easily deployable screening assessment for cognitive decline or deficits.