Mathematical models of cognitive processes

The research activity carried out during the PhD course was focused on the development of mathematical models of some cognitive processes and their validation by means of data present in literature, with a double aim: i) to achieve a better interpretation and explanation of the great amount of data obtained on these processes from different methodologies (electrophysiological recordings on animals, neuropsychological, psychophysical and neuroimaging studies in humans), ii) to exploit model predictions and results to guide future research and experiments. In particular, the research activity has been focused on two different projects: 1) the first one concerns the development of neural oscillators networks, in order to investigate the mechanisms of synchronization of the neural oscillatory activity during cognitive processes, such as object recognition, memory, language, attention; 2) the second one concerns the mathematical modelling of multisensory integration processes (e.g. visual-acoustic), which occur in several cortical and subcortical regions (in particular in a subcortical structure named Superior Colliculus (SC)), and which are fundamental for orienting motor and attentive responses to external world stimuli. This activity has been realized in collaboration with the Center for Studies and Researches in Cognitive Neuroscience of the University of Bologna (in Cesena) and the Department of Neurobiology and Anatomy of the Wake Forest University School of Medicine (NC, USA). PART 1. Objects representation in a number of cognitive functions, like perception and recognition, foresees distribute processes in different cortical areas. One of the main neurophysiological question concerns how the correlation between these disparate areas is realized, in order to succeed in grouping together the characteristics of the same object (binding problem) and in maintaining segregated the properties belonging to different objects simultaneously present (segmentation problem). Different theories have been proposed to address these questions (Barlow, 1972). One of the most influential theory is the so called “assembly coding”, postulated by Singer (2003), according to which 1) an object is well described by a few fundamental properties, processing in different and distributed cortical areas; 2) the recognition of the object would be realized by means of the simultaneously activation of the cortical areas representing its different features; 3) groups of properties belonging to different objects would be kept separated in the time domain. In Chapter 1.1 and in Chapter 1.2 we present two neural network models for object recognition, based on the “assembly coding” hypothesis. These models are networks of Wilson-Cowan oscillators which exploit: i) two high-level “Gestalt Rules” (the similarity and previous knowledge rules), to realize the functional link between elements of different cortical areas representing properties of the same object (binding problem); 2) the synchronization of the neural oscillatory activity in the γ-band (30-100Hz), to segregate in time the representations of different objects simultaneously present (segmentation problem). These models are able to recognize and reconstruct multiple simultaneous external objects, even in difficult case (some wrong or lacking features, shared features, superimposed noise). In Chapter 1.3 the previous models are extended to realize a semantic memory, in which sensory-motor representations of objects are linked with words. To this aim, the network, previously developed, devoted to the representation of objects as a collection of sensory-motor features, is reciprocally linked with a second network devoted to the representation of words (lexical network) Synapses linking the two networks are trained via a time-dependent Hebbian rule, during a training period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from linguistic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with some shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits). PART 2. The ability of the brain to integrate information from different sensory channels is fundamental to perception of the external world (Stein et al, 1993). It is well documented that a number of extraprimary areas have neurons capable of such a task; one of the best known of these is the superior colliculus (SC). This midbrain structure receives auditory, visual and somatosensory inputs from different subcortical and cortical areas, and is involved in the control of orientation to external events (Wallace et al, 1993). SC neurons respond to each of these sensory inputs separately, but is also capable of integrating them (Stein et al, 1993) so that the response to the combined multisensory stimuli is greater than that to the individual component stimuli (enhancement). This enhancement is proportionately greater if the modality-specific paired stimuli are weaker (the principle of inverse effectiveness). Several studies have shown that the capability of SC neurons to engage in multisensory integration requires inputs from cortex; primarily the anterior ectosylvian sulcus (AES), but also the rostral lateral suprasylvian sulcus (rLS). If these cortical inputs are deactivated the response of SC neurons to cross-modal stimulation is no different from that evoked by the most effective of its individual component stimuli (Jiang et al 2001). This phenomenon can be better understood through mathematical models. The use of mathematical models and neural networks can place the mass of data that has been accumulated about this phenomenon and its underlying circuitry into a coherent theoretical structure. In Chapter 2.1 a simple neural network model of this structure is presented; this model is able to reproduce a large number of SC behaviours like multisensory enhancement, multisensory and unisensory depression, inverse effectiveness. In Chapter 2.2 this model was improved by incorporating more neurophysiological knowledge about the neural circuitry underlying SC multisensory integration, in order to suggest possible physiological mechanisms through which it is effected. This endeavour was realized in collaboration with Professor B.E. Stein and Doctor B. Rowland during the 6 months-period spent at the Department of Neurobiology and Anatomy of the Wake Forest University School of Medicine (NC, USA), within the Marco Polo Project. The model includes four distinct unisensory areas that are devoted to a topological representation of external stimuli. Two of them represent subregions of the AES (i.e., FAES, an auditory area, and AEV, a visual area) and send descending inputs to the ipsilateral SC; the other two represent subcortical areas (one auditory and one visual) projecting ascending inputs to the same SC. Different competitive mechanisms, realized by means of population of interneurons, are used in the model to reproduce the different behaviour of SC neurons in conditions of cortical activation and deactivation. The model, with a single set of parameters, is able to mimic the behaviour of SC multisensory neurons in response to very different stimulus conditions (multisensory enhancement, inverse effectiveness, within- and cross-modal suppression of spatially disparate stimuli), with cortex functional and cortex deactivated, and with a particular type of membrane receptors (NMDA receptors) active or inhibited. All these results agree with the data reported in Jiang et al. (2001) and in Binns and Salt (1996). The model suggests that non-linearities in neural responses and synaptic (excitatory and inhibitory) connections can explain the fundamental aspects of multisensory integration, and provides a biologically plausible hypothesis about the underlying circuitry.

Deficit in the Emotional Embodiment in Alexithymia

Alexithymia refers to difficulties in recognizing one’s own emotions and others emotions. Theories of emotional embodiment suggest that, in order to understand other peoples’ feelings, observers re-experience, or simulate, the relevant component (i.e. somatic, motor, visceral) of emotion’s expressed by others in one’s self. In this way, the emotions are “embodied”. Critically, to date, there are no studies investigating the ability of alexithymic individuals in embodying the emotions conveyed by faces. In the present dissertation different implicit paradigms and techniques falling within the field of affective neuroscience have been employed in order to test a possible deficit in the embodiment of emotions in alexithymia while subjects were requested to observe faces manifesting different expression: fear, disgust, happiness and neutral. The level of the perceptual encoding of emotional faces and the embodiment of emotions in the somato-sensory and sensory-motor system have been investigated. Moreover, non-communicative motor reaction to emotional stimuli (i.e. visceral reactions) and interoceptive abilities of alexithymic subjects have been explored. The present dissertation provided convergent evidences in support of a deficit in the processing of fearful expression in subjects with high alexithymic personality traits. Indeed, the pattern of fear induced changes in the perceptual encoding, in the somato-sensory and in the somato-motor system (both the communicative and non communicative one) is widely and consistently altered in alexithymia. This support the hypothesis of a diminished responses to fearful stimuli in alexithymia. In addition, the overall results on happiness and disgust, although preliminary, provided interesting results. Indeed, the results on happiness revealed a defective perceptual encoding, coupled with a slight difficulty (i.e. delayed responses) at the level of the communicative somato-motor system, and the emotion of disgust has been found to be abnormally embodied at the level of the somato-sensory system.

Visual-somatosensory interactions in mental representations of the body and the face

The body is represented in the brain at levels that incorporate multisensory information. This thesis focused on interactions between vision and cutaneous sensations (i.e., touch and pain). Experiment 1 revealed that there are partially dissociable pathways for visual enhancement of touch (VET) depending upon whether one sees one’s own body or the body of another person. This indicates that VET, a seeming low-level effect on spatial tactile acuity, is actually sensitive to body identity. Experiments 2-4 explored the effect of viewing one’s own body on pain perception. They demonstrated that viewing the body biases pain intensity judgments irrespective of actual stimulus intensity, and, more importantly, reduces the discriminative capacities of the nociceptive pathway encoding noxious stimulus intensity. The latter effect only occurs if the pain-inducing event itself is not visible, suggesting that viewing the body alone and viewing a stimulus event on the body have distinct effects on cutaneous sensations. Experiment 5 replicated an enhancement of visual remapping of touch (VRT) when viewing fearful human faces being touched, and further demonstrated that VRT does not occur for observed touch on non-human faces, even fearful ones. This suggests that the facial expressions of non-human animals may not be simulated within the somatosensory system of the human observer in the same way that the facial expressions of other humans are. Finally, Experiment 6 examined the enfacement illusion, in which synchronous visuo-tactile inputs cause another’s face to be assimilated into the mental self-face representation. The strength of enfacement was not affected by the other’s facial expression, supporting an asymmetric relationship between processing of facial identity and facial expressions. Together, these studies indicate that multisensory representations of the body in the brain link low-level perceptual processes with the perception of emotional cues and body/face identity, and interact in complex ways depending upon contextual factors.

Residual function, spontaneous reorganisation and treatment plasticity in homonymous visual field defects

This thesis will focus on the residual function and visual and attentional deficits in human patients, which accompany damage to the visual cortex or its thalamic afferents, and plastic changes, which follow it. In particular, I will focus on homonymous visual field defects, which comprise a broad set of central disorders of vision. I will present experimental evidence that when the primary visual pathway is completely damaged, the only signal that can be implicitly processed via subcortical visual networks is fear. I will also present data showing that in a patient with relative deafferentation of visual cortex, changes in the spatial tuning and response gain of the contralesional and ipsilesional cortex are observed, which are accompanied by changes in functional connectivity with regions belonging to the dorsal attentional network and the default mode network. I will also discuss how cortical plasticity might be harnessed to improve recovery through novel treatments. Moreover, I will show how treatment interventions aimed at recruiting spared subcortical pathway supporting multisensory orienting can drive network level change.

Adaptive and Maladaptive Implications of Reinforcement Learning Processes: Fronto-Striatal Loops and Behavioural Correlates

That humans and animals learn from interaction with the environment is a foundational idea underlying nearly all theories of learning and intelligence. Learning that certain outcomes are associated with specific actions or stimuli (both internal and external), is at the very core of the capacity to adapt behaviour to environmental changes. In the present work, appetitive and aversive reinforcement learning paradigms have been used to investigate the fronto-striatal loops and behavioural correlates of adaptive and maladaptive reinforcement learning processes, aiming to a deeper understanding of how cortical and subcortical substrates interacts between them and with other brain systems to support learning. By combining a large variety of neuroscientific approaches, including behavioral and psychophysiological methods, EEG and neuroimaging techniques, these studies aim at clarifying and advancing the knowledge of the neural bases and computational mechanisms of reinforcement learning, both in normal and neurologically impaired population.

Integration of Multiple Simultaneous Stimuli within and between Somatosensory sub-Modalities

Psychological characterisation of the somatosensory system often focusses on minimal units of perception, such as detection, localisation, and magnitude estimation of single events. Research on how multiple simultaneous stimuli are aggregated to create integrated, synthetic experiences is rarer. This thesis aims to shed a light on the mechanisms underlying the integration of multiple simultaneous stimuli, within and between different sub-modalities of the somatosensory system. First, we investigated the ability of healthy individuals to perceive the total intensity of composite somatosensory patterns. We found that the overall intensity of tactile, cold, or warm patterns was systematically overestimated when the multiple simultaneous stimuli had different intensities. Perception of somatosensory totals was biased towards the most salient element in the pattern. Furthermore, we demonstrated that peak-biased aggregation is a genuine perceptual phenomenon which does not rely on the discrimination of the parts, but is rather based on the salience of each stimulus. Next, we studied a classical thermal illusion to assess participants’ ability to localise thermal stimuli delivered on the fingers either in isolation, or in uniform and non-uniform patterns. We found that despite a surprisingly high accuracy in reporting the location of a single stimulus, when participants were presented with non-uniform patterns, their ability to identify the thermal state of a specific finger was completely abolished. Lastly, we investigated the perceptual and neural correlates of thermo-nociceptive interaction during the presentation of multiple thermal stimuli. We found that inhibition of pain by warmth was independent from both the position and the number of thermal stimuli administered. Our results suggest that nonlinear integration of multiple stimuli, within and between somatosensory sub-modalities, may be an efficient way by which the somatosensory system synthesises the complexity of reality, providing an extended and coherent perception of the world, in spite of its deep bandwidth limitations.

Models for the study of neuronal activity during cognitive tasks

Assessment of brain connectivity among different brain areas during cognitive or motor tasks is a crucial problem in neuroscience today. Aim of this research study is to use neural mass models to assess the effect of various connectivity patterns in cortical EEG power spectral density (PSD), and investigate the possibility to derive connectivity circuits from EEG data. To this end, two different models have been built. In the first model an individual region of interest (ROI) has been built as the parallel arrangement of three populations, each one exhibiting a unimodal spectrum, at low, medium or high frequency. Connectivity among ROIs includes three parameters, which specify the strength of connection in the different frequency bands. Subsequent studies demonstrated that a single population can exhibit many different simultaneous rhythms, provided that some of these come from external sources (for instance, from remote regions). For this reason in the second model an individual ROI is simulated only with a single population. Both models have been validated by comparing the simulated power spectral density with that computed in some cortical regions during cognitive and motor tasks. Another research study is focused on multisensory integration of tactile and visual stimuli in the representation of the near space around the body (peripersonal space). This work describes an original neural network to simulate representation of the peripersonal space around the hands, in basal conditions and after training with a tool used to reach the far space. The model is composed of three areas for each hand, two unimodal areas (visual and tactile) connected to a third bimodal area (visual-tactile), which is activated only when a stimulus falls within the peripersonal space. Results show that the peripersonal space, which includes just a small visual space around the hand in normal conditions, becomes elongated in the direction of the tool after training, thanks to a reinforcement of synapses.

Testo e memoria tra semiotica e scienze cognitive: una situazione sperimentale intorno al ricordo di un film

The research project presented in this dissertation is about text and memory. The title of the work is "Text and memory between Semiotics and Cognitive Science: an experimental setting about remembering a movie". The object of the research is the relationship between texts or "textuality" - using a more general semiotic term - and memory. The goal is to analyze the link between those semiotic artifacts that a culture defines as autonomous meaningful objects - namely texts - and the cognitive performance of memory that allows to remember them. An active dialogue between Semiotics and Cognitive Science is the theoretical paradigm in which this research is set, the major intend is to establish a productive alignment between the "theory of text" developed in Semiotics and the "theory of memory" outlined in Cognitive Science. In particular the research is an attempt to study how human subjects remember and/or misremember a film, as a specific case study; in semiotics, films are “cinematographic texts”. The research is based on the production of a corpus of data gained through the qualitative method of interviewing. After an initial screening of a fulllength feature film each participant of the experiment has been interviewed twice, according to a pre-established set of questions. The first interview immediately after the screening: the subsequent, follow-up interview three months from screening. The purpose of this design is to elicit two types of recall from the participants. In order to conduce a comparative inquiry, three films have been used in the experimental setting. Each film has been watched by thirteen subjects, that have been interviewed twice. The corpus of data is then made by seventy-eight interviews. The present dissertation displays the results of the investigation of these interviews. It is divided into six main parts. Chapter one presents a theoretical framework about the two main issues: memory and text. The issue of the memory is introduced through many recherches drown up in the field of Cognitive Science and Neuroscience. It is developed, at the same time, a possible relationship with a semiotic approach. The theoretical debate about textuality, characterizing the field of Semiotics, is examined in the same chapter. Chapter two deals with methodology, showing the process of definition of the whole method used for production of the corpus of data. The interview is explored in detail: how it is born, what are the expected results, what are the main underlying hypothesis. In Chapter three the investigation of the answers given by the spectators starts. It is examined the phenomenon of the outstanding details of the process of remembering, trying to define them in a semiotic way. Moreover there is an investigation of the most remembered scenes in the movie. Chapter four considers how the spectators deal with the whole narrative. At the same time it is examined what they think about the global meaning of the film. Chapter five is about affects. It tries to define the role of emotions in the process of comprehension and remembering. Chapter six presents a study of how the spectators account for a single scene of the movie. The complete work offers a broad perspective about the semiotic issue of textuality, using both a semiotic competence and a cognitive one. At the same time it presents a new outlook on the issue of memory, opening several direction of research.

Methods for the estimation of the cortical activity and connectivity during cognitive tasks in humans

The research activity characterizing the present thesis was mainly centered on the design, development and validation of methodologies for the estimation of stationary and time-varying connectivity between different regions of the human brain during specific complex cognitive tasks. Such activity involved two main aspects: i) the development of a stable, consistent and reproducible procedure for functional connectivity estimation with a high impact on neuroscience field and ii) its application to real data from healthy volunteers eliciting specific cognitive processes (attention and memory). In particular the methodological issues addressed in the present thesis consisted in finding out an approach to be applied in neuroscience field able to: i) include all the cerebral sources in connectivity estimation process; ii) to accurately describe the temporal evolution of connectivity networks; iii) to assess the significance of connectivity patterns; iv) to consistently describe relevant properties of brain networks. The advancement provided in this thesis allowed finding out quantifiable descriptors of cognitive processes during a high resolution EEG experiment involving subjects performing complex cognitive tasks.

Seeing emotions in others: Improving cognitive and neuronal mechanisms of facial expression processing

In the conceptual framework of affective neuroscience, this thesis intends to advance the understanding of the plasticity mechanisms of other’s emotional facial expression representations. Chapter 1 outlines a description of the neurophysiological bases of Hebbian plasticity, reviews influential studies that adopted paired associative stimulation procedures, and introduces new lines of research where the impact of cortico-cortical paired associative stimulation protocols on higher order cognitive functions is investigated. The experiments in Chapter 2 aimed to test the modulatory influence of a perceptual-motor training, based on the execution of emotional expressions, on the subsequent emotion intensity judgements of others’ high (i.e., full visible) and low-intensity (i.e., masked) emotional expressions. As a result of the training-induced learning, participants showed a significant congruence effect, as indicated by relatively higher expression intensity ratings for the same emotion as the one that was previously trained. Interestingly, although judged as overall less emotionally intense, surgical facemasks did not prevent the emotion-specific effects of the training to occur, suggesting that covering the lower part of other’s face do not interact with the training-induced congruence effect. In Chapter 3 it was implemented a transcranial magnetic stimulation study targeting neural pathways involving re-entrant input from higher order brain regions into lower levels of the visual processing hierarchy. We focused on cortical visual networks within the temporo-occipital stream underpinning the processing of emotional faces and susceptible to plastic adaptations. Importantly, we tested the plasticity-induced effects in a state dependent manner, by administering ccPAS while presenting different facial expressions yet afferent to a specific emotion. Results indicated that the discrimination accuracy of emotion-specific expressions is enhanced following the ccPAS treatment, suggesting that a multi-coil TMS intervention might represent a suitable tool to drive brain remodeling at a neural network level, and consequently influence a specific behavior.