Brain responses to audiovisual speech mismatch in infants are associated with individual differences in looking behaviour

Research on audiovisual speech integration has reported high levels of individual variability, especially among young infants. In the present study we tested the hypothesis that this variability results from individual differences in the maturation of audiovisual speech processing during infancy. A developmental shift in selective attention to audiovisual speech has been demonstrated between 6 and 9 months with an increase in the time spent looking to articulating mouths as compared to eyes (Lewkowicz & Hansen-Tift. (2012) Proc. Natl Acad. Sci. USA, 109, 1431–1436; Tomalski et al. (2012) Eur. J. Dev. Psychol., 1–14). In the present study we tested whether these changes in behavioural maturational level are associated with differences in brain responses to audiovisual speech across this age range. We measured high-density event-related potentials (ERPs) in response to videos of audiovisually matching and mismatched syllables /ba/ and /ga/, and subsequently examined visual scanning of the same stimuli with eye-tracking. There were no clear age-specific changes in ERPs, but the amplitude of audiovisual mismatch response (AVMMR) to the combination of visual /ba/ and auditory /ga/ was strongly negatively associated with looking time to the mouth in the same condition. These results have significant implications for our understanding of individual differences in neural signatures of audiovisual speech processing in infants, suggesting that they are not strictly related to chronological age but instead associated with the maturation of looking behaviour, and develop at individual rates in the second half of the first year of life.

Looking behaviour and preference for artworks: The role of emotional valence and location

The position of an item influences its evaluation, with research consistently finding that items occupying central locations are preferred and have a higher subjective value. The current study investigated whether this centre-stage effect (CSE) is a result of bottom-up gaze allocation to the central item, and whether it is affected by item valence. Participants (n=50) were presented with three images of artistic paintings in a row and asked to choose the image they preferred. Eye movements were recorded for a subset of participants (n=22). On each trial the three artworks were either similar but different, or were identical and with positive valence, or were identical and with negative valence. The results showed a centre-stage effect, with artworks in the centre of the row preferred, but only when they were identical and of positive valence. Significantly greater gaze allocation to the central and left artwork was not mirrored by equivalent increases in preference choices. Regression analyses showed that when the artworks were positive and identical the participants’ last fixation predicted preference for the central art-work, whereas the fixation duration predicted preference if the images were different. Overall the result showed that item valence, rather than level of gaze allocation, influences the CSE, which is incompatible with the bottom-up gaze explanation. We propose that the centre stage heuristic, which specifies that the best items are in the middle, is able to explain these findings and the centre-stage effect.