Does learning a cross-modal association alter within-modal sensitivities?

Abstract

Previous findings showing that synesthetes with innate and acquired synesthesia have differing capacities in discriminating visual hues have led to the suggestion of a mental resource equilibrium model, which proposes that a shared resource pool exists across sensory modalities and that a reallocation of resources across the modalities could be triggered through establishment of cross-modal associations (Ward, 2009). In this study, we examined this proposition by testing whether training that promoted the formation of cross-modal associations (between visual brightness and auditory pitch) would affect subjects’ within-modal discrimination abilities for judging brightness and pitch. Participants were trained on either intuitive (i.e., positive correlation between the brightness of a uniform patch and pitch of a simultaneously presented sound stimulus), counterintuitive (i.e., negative correlation), or random brightness-pitch relationships, and tested before and after training on independent pitch and brightness discrimination tasks. We found that while training was successful in promoting the acquisition of the audiovisual correlations, it did not alter within-modal discrimination capacity for either brightness or pitch.Our data suggest that learned multimodal associations are not formed at a cost of altered within-modal sensitivities. We speculate that the strategies learned by subjects are likely more general in nature, and unlikely to be stimulus- or even modality-specific.