Infant cross-modal learning

Abstract

Cross-modal information facilitates adult learning, but we know little about whether infants enjoy the same benefit during their first year of life when their brain development, being most prolific, is critically influenced by sensory experience. We investigate whether and how infants’ acquisition of an abstract rule is enhanced or limited by simultaneously presented audio-visual (A-V) stimuli. We habituated 8–10 month old infants with a sequential rule (i.e. AAB). Then we showed them rule-consistent (i.e. AAB) and rule-inconsistent (i.e. ABA or ABB) dis-habituating trials. Looking time differences between the two types of trials indicated infants’ successful discrimination and rule acquisition. We found A-V bimodal stimuli do not always facilitate learning—against the prediction of “intersensory redundancy hypothesis”. Instead, A-V congruency and correspondence were critical. For example, congruent emotional faces and emotional sounds, syllable-speaking faces and syllable sounds, upward moving shapes and rising pitch sounds all led to successful rule learning. But rule learning disappeared when we altered the cross-modal relevance and correspondence relationship between the A-V pair. Our studies suggested that during infancy, highlevel integration across A-V information put a prior constraint upon rule acquisition and learning. The sensory inputs for infant learning are not always the more, the better. Supported by grants from the Hong Kong Grant Research Council and the University of Hong Kong Seed Funding Programme for Basic Research to Chia-huei Tseng.