Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour

Research on audiovisual speech integration has reported high levels of individual variability, especially among young infants. In the present study we tested the hypothesis that this variability results from individual differences in the maturation of audiovisual speech processing during infancy. A developmental shift in selective attention to audiovisual speech has been demonstrated between 6 and 9 months with an increase in the time spent looking to articulating mouths as compared to eyes (Lewkowicz & Hansen-Tift. (2012) Proc. Natl Acad. Sci. USA, 109, 1431–1436; Tomalski et al. (2012) Eur. J. Dev. Psychol., 1–14). In the present study we tested whether these changes in behavioural maturational level are associated with differences in brain responses to audiovisual speech across this age range. We measured high-density event-related potentials (ERPs) in response to videos of audiovisually matching and mismatched syllables /ba/ and /ga/, and subsequently examined visual scanning of the same stimuli with eye-tracking. There were no clear age-specific changes in ERPs, but the amplitude of audiovisual mismatch response (AVMMR) to the combination of visual /ba/ and auditory /ga/ was strongly negatively associated with looking time to the mouth in the same condition. These results have significant implications for our understanding of individual differences in neural signatures of audiovisual speech processing in infants, suggesting that they are not strictly related to chronological age but instead associated with the maturation of looking behaviour, and develop at individual rates in the second half of the first year of life.

Beyond visual imagery: How modality-specific is enhanced mental imagery in synesthesia?

Synesthesia based in visual modalities has been associated with reports of vivid visual imagery. We extend this finding to consider whether other forms of synesthesia are also associated with enhanced imagery, and whether this enhancement reflects the modality of synesthesia. We used self‐report imagery measures across multiple sensory modalities, comparing synesthetes’ responses (with a variety of forms of synesthesia) to those of nonsynesthete matched controls. Synesthetes reported higher levels of visual, auditory, gustatory, olfactory and tactile imagery and a greater level of imagery use. Furthermore, their reported enhanced imagery is restricted to the modalities involved in the individual’s synesthesia. There was also a relationship between the number of forms of synesthesia an individual has, and the reported vividness of their imagery, highlighting the need for future research to consider the impact of multiple forms of synesthesia. We also recommend the use of behavioral measures to validate these self‐report findings.