Decoding semantics across fMRI sessions with different stimulus modalities:a practical MVPA study

Both embodied and symbolic accounts of conceptual organization would predict partial sharing and partial differentiation between the neural activations seen for concepts activated via different stimulus modalities. But cross-participant and cross-session variability in BOLD activity patterns makes analyses of such patterns with MVPA methods challenging. Here, we examine the effect of cross-modal and individual variation on the machine learning analysis of fMRI data recorded during a word property generation task. We present the same set of living and non-living concepts (land-mammals, or work tools) to a cohort of Japanese participants in two sessions: the first using auditory presentation of spoken words; the second using visual presentation of words written in Japanese characters. Classification accuracies confirmed that these semantic categories could be detected in single trials, with within-session predictive accuracies of 80-90%. However cross-session prediction (learning from auditory-task data to classify data from the written-word-task, or vice versa) suffered from a performance penalty, achieving 65-75% (still individually significant at p « 0.05). We carried out several follow-on analyses to investigate the reason for this shortfall, concluding that distributional differences in neither time nor space alone could account for it. Rather, combined spatio-temporal patterns of activity need to be identified for successful cross-session learning, and this suggests that feature selection strategies could be modified to take advantage of this.

Can illusory deviant stimuli be used as attentional distractors to record vMMN in a passive three stimulus oddball paradigm?

A passive three stimulus oddball paradigm was used to investigate Visual Mismatch Negativity (vMMN) a component of the Event Related Potential (ERP) believed to represent a central pre-attentive change mechanism. Responses to a change in orientation were recorded to monochrome stimuli presented to subjects on a computer screen. One of the infrequent stimuli formed an illusory figure (Kanizsa Square) aimed to capture spatial attention in the absence of an active task. Nineteen electrodes (10-20 system) were used to record the electroencephalogram in fourteen subjects (ten females) mean age 34.5 years. ERPs to all stimuli consisted of a positive negative positive complex recorded maximally over lateral occipital areas. The negative component was greater for deviant and illusory deviant compared to standard stimuli in a time window of 170-190 ms. A P3a component over frontal/central electrodes to the illusory deviant but not to the deviant stimulus suggests the illusory figure was able to capture attention and orientate subjects to the recording. Subtraction waveforms revealed visual discrimination responses at occipital electrodes, which may represent vMMN. In a control study with 13 subjects (11 females; mean age 29.23 years), using an embedded active attention task, we confirmed the existence of an earlier (150-170 ms) and attenuated vMMN. Recordings from an intracranial case study confirmed separation of N1 and discrimination components to posterior and anterior occipital areas, respectively. We conclude that although the illusory figure captured spatial attention in its own right it did not draw sufficient attentional resources from the standard-deviant comparison as revealed when using a concurrent active task.