Visual and verbal processing effects on human memory

In this study in the field of Consumer Behavior, brand name memory of consumers with regard to verbal and visual incongruent and congruent information such as memory structure of brands was tested. Hence, four experimental groups with different constellations of verbal and visual congruity and incongruity were created to compare their brand name memory performance. The experiment was conducted in several classes with 128 students, each group with 32 participants. It was found that brands, which are presented in a congruent or moderately incongruent relation to their brand schema, result in a better brand recall than their incongruent counterparts. A difference between visual congruity and moderately incongruity could not be confirmed. In contrast to visual incongruent information, verbal incongruent information does not result in a worse brand recall performance.

Automatic visual detection of human behavior: a review from 2000 to 2014

Due to advances in information technology (e.g., digital video cameras, ubiquitous sensors), the automatic detection of human behaviors from video is a very recent research topic. In this paper, we perform a systematic and recent literature review on this topic, from 2000 to 2014, covering a selection of 193 papers that were searched from six major scientific publishers. The selected papers were classified into three main subjects: detection techniques, datasets and applications. The detection techniques were divided into four categories (initialization, tracking, pose estimation and recognition). The list of datasets includes eight examples (e.g., Hollywood action). Finally, several application areas were identified, including human detection, abnormal activity detection, action recognition, player modeling and pedestrian detection. Our analysis provides a road map to guide future research for designing automatic visual human behavior detection systems.

The Rank Lecture: Has imaging told us anything new about human visual organisation?

The functional architecture of the occipital cortex is being studied with increasing detail. Functional and structural MR based imaging are altering views about the organisation of the human visual system. Recent advances have ranged from comparative studies with non-human primates to predictive scanning. The latter multivariate technique describes with sub-voxel resolution patterns of activity that are characteristic of specific visual experiences. One can deduce what a subject experienced visually from the pattern of cortical activity recorded. The challenge for the future is to understand visual functions in terms of cerebral computations at a mesoscopic level of description and to relate this information to electrophysiology. The principal medical application of this new knowledge has focused to a large extent on plasticity and the capacity for functional reorganisation. Crossmodality visual-sensory interactions and cross-correlations between visual and other cerebral areas in the resting state are areas of considerable current interest. The lecture will review findings over the last two decades and reflect on possible roles for imaging studies in the future.

Visual search in spatial neglect studied with a preview paradigm.

Impaired visual search is a hallmark of spatial neglect. When searching for an unique feature (e.g., color) neglect patients often show only slight visual field asymmetries. In contrast, when the target is defined by a combination of features (e.g., color and form) they exhibit a severe deficit of contralesional search. This finding suggests a selective impairment of the serial deployment of spatial attention. Here, we examined this deficit with a preview paradigm. Neglect patients searched for a target defined by the conjunction of shape and color, presented together with varying numbers of distracters. The presentation time was varied such that on some trials participants previewed the target together with same-shape/different-color distracters, for 300 or 600 ms prior to the appearance of additional different-shape/same-color distracters. On the remaining trials the target and all distracters were shown simultaneously. Healthy participants exhibited a serial search strategy only when all items were presented simultaneously, whereas in both preview conditions a pop-out effect was observed. Neglect patients showed a similar pattern when the target was presented in the right hemifield. In contrast, when searching for a target in the left hemifield they showed serial search in the no-preview condition, as well as with a preview of 300 ms, and partly even at 600 ms. A control experiment suggested that the failure to fully benefit from item preview was probably independent of accurate perception of time. Our results, when viewed in the context of existing literature, lead us to conclude that the visual search deficit in neglect reflects two additive factors: a biased representation of attentional priority in favor of ipsilesional information and exaggerated capture of attention by ipsilesional abrupt onsets.

Visual management of sags and incidents gathered in distribution substations for power quality management

Monitor a distribution network implies working with a huge amount of data coining from the different elements that interact in the network. This paper presents a visualization tool that simplifies the task of searching the database for useful information applicable to fault management or preventive maintenance of the network

Time processing in visual cortices: How the visual brain encodes and keeps track of time

Time is embedded in any sensory experience: the movements of a dance, the rhythm of a piece of music, the words of a speaker are all examples of temporally structured sensory events. In humans, if and how visual cortices perform temporal processing remains unclear. Here we show that both primary visual cortex (V1) and extrastriate area V5/MT are causally involved in encoding and keeping time in memory and that this involvement is independent from low-level visual processing. Most importantly we demonstrate that V1 and V5/MT are functionally linked and temporally synchronized during time encoding whereas they are functionally independent and operate serially (V1 followed by V5/MT) while maintaining temporal information in working memory. These data challenge the traditional view of V1 and V5/MT as visuo-spatial features detectors and highlight the functional contribution and the temporal dynamics of these brain regions in the processing of time in millisecond range. The present project resulted in the paper entitled: 'How the visual brain encodes and keeps track of time' by Paolo Salvioni, Lysiann Kalmbach, Micah Murray and Domenica Bueti that is now submitted for publication to the Journal of Neuroscience.

Information technologies and communication between parents and children

This is an exploratory study that aims, on the one hand, to examine in more detail how children between 12 and 16 years of age use different audiovisual technologies, what they feel and think when using them, and whom they like to speak to about such experiences. On the other hand, we look more deeply into the interactions between adults and children, particularly between parents and their children, in relation to these technologies when children use them at home or in other places. We analysed responses to questionnaires with several common items, administered separately to parents and children. Children’s responses reflect an important level of dissatisfaction when talking with different adults about media activities. Our findings support the thesis that more and more children socialise through new information and communication technologies with little or no recourse to adult criteria, giving rise to the emergence of specific children’s cultures. Crossing of the responses of parents and those of their own children shows us which aspects of media reality adults overestimate or underestimate in comparison to children, and to what degree certain judgements coincide and differ between generations. The results can be applied to the improvement of relations between adults and adolescents, taking advantage of adolescents’ strong motivation to engage in activities using audiovisual media

QDIUS2 (Query Device Information Using Specific Sensors)

QDius2 és una aplicació pensada per a persones amb discapacitat visual. Consisteix en realitzar una descripció verbalitzada de l’estat d’un electrodomèstic utilitzant un dispositiu mòbil amb sistema operatiu Android. L’aplicació guiarà verbalment l’usuari perquè pugui enfocar el panell de control de l’electrodomèstic amb la càmera. Es processarà la imatge obtinguda per extreure la seva informació, ja sigui en el mateix mòbil o al núvol, segons les preferències de l’usuari, i es llegirà la informació resultant. Aquest projecte està dividit en vuit memòries, una per a cada integrant del grup.

Integration of olfactory information in a spatial representation enabling accurate arm choice in the radial arm maze

The aim of the present study was to determine whether and how rats can use local olfactory cues for spatial orientation. Rats were trained in an eight-arm radial maze under different conditions as defined by the presence or absence of supplementary olfactory cues marking each arm, the availability of distant visuospatial information, and the illumination of the maze (light or darkness). The different visual conditions were designed to dissociate among the effects of light per se and those of visuospatial cues, on the use of olfactory cues for accurate arm choice. Different procedures with modifications of the arrangement of olfactory cues were used to determine if rats formed a representation of the spatial configuration of the olfactory cues and if they could rely on such a representation for accurate arm choice in the radial maze. The present study demonstrated that the use of olfactory cues to direct arm choice in the radial arm maze was critically dependent on the illumination conditions and implied two different modes of processing of olfactory information according to the presence or the absence of light. Olfactory cues were used in an explicit manner and enabled accurate arm choice only in the absence of light. Rats, however, had an implicit memory of the location of the olfactory cues and formed a representation of the spatial position of these cues, whatever the lighting conditions. They did not memorize the spatial configuration of the olfactory cues per se but needed these cues to be linked to the external spatial frame of reference.

Visual functions in the healthy and schizophrenic brain : from stimulus control to illusory perceptions using electrical neuroimaging

ABSTRACT (FRENCH)Ce travail de thèse basé sur le système visuel chez les sujets sains et chez les patients schizophrènes, s'articule autour de trois articles scientifiques publiés ou en cours de publication. Ces articles traitent des sujets suivants : le premier article présente une nouvelle méthode de traitement des composantes physiques des stimuli (luminance et fréquence spatiale). Le second article montre, à l'aide d'analyses de données EEG, un déficit de la voie magnocellulaire dans le traitement visuel des illusions chez les patients schizophrènes. Ceci est démontré par l'absence de modulation de la composante PI chez les patients schizophrènes contrairement aux sujets sains. Cette absence est induite par des stimuli de type illusion Kanizsa de différentes excentricités. Finalement, le troisième article, également à l'aide de méthodes de neuroimagerie électrique (EEG), montre que le traitement des contours illusoires se trouve dans le complexe latéro-occipital (LOC), à l'aide d'illusion « misaligned gratings ». De plus il révèle que les activités démontrées précédemment dans les aires visuelles primaires sont dues à des inférences « top- down ».Afin de permettre la compréhension de ces trois articles, l'introduction de ce manuscrit présente les concepts essentiels. De plus des méthodes d'analyses de temps-fréquence sont présentées. L'introduction est divisée en quatre parties : la première présente le système visuel depuis les cellules retino-corticales aux deux voix du traitement de l'information en passant par les régions composant le système visuel. La deuxième partie présente la schizophrénie par son diagnostic, ces déficits de bas niveau de traitement des stimuli visuel et ces déficits cognitifs. La troisième partie présente le traitement des contours illusoires et les trois modèles utilisés dans le dernier article. Finalement, les méthodes de traitement des données EEG seront explicitées, y compris les méthodes de temps-fréquences.Les résultats des trois articles sont présentés dans le chapitre éponyme (du même nom). De plus ce chapitre comprendra les résultats obtenus à l'aide des méthodes de temps-fréquenceFinalement, la discussion sera orientée selon trois axes : les méthodes de temps-fréquence ainsi qu'une proposition de traitement de ces données par une méthode statistique indépendante de la référence. La discussion du premier article en montrera la qualité du traitement de ces stimuli. La discussion des deux articles neurophysiologiques, proposera de nouvelles d'expériences afin d'affiner les résultats actuels sur les déficits des schizophrènes. Ceci pourrait permettre d'établir un marqueur biologique fiable de la schizophrénie.ABSTRACT (ENGLISH)This thesis focuses on the visual system in healthy subjects and schizophrenic patients. To address this research, advanced methods of analysis of electroencephalographic (EEG) data were used and developed. This manuscript is comprised of three scientific articles. The first article showed a novel method to control the physical features of visual stimuli (luminance and spatial frequencies). The second article showed, using electrical neuroimaging of EEG, a deficit in spatial processing associated with the dorsal pathway in chronic schizophrenic patients. This deficit was elicited by an absent modulation of the PI component in terms of response strength and topography as well as source estimations. This deficit was orthogonal to the preserved ability to process Kanizsa-type illusory contours. Finally, the third article resolved ongoing debates concerning the neural mechanism mediating illusory contour sensitivity by using electrical neuroimaging to show that the first differentiation of illusory contour presence vs. absence is localized within the lateral occipital complex. This effect was subsequent to modulations due to the orientation of misaligned grating stimuli. Collectively, these results support a model where effects in V1/V2 are mediated by "top-down" modulation from the LOC.To understand these three articles, the Introduction of this thesis presents the major concepts used in these articles. Additionally, a section is devoted to time-frequency analysis methods not presented in the articles themselves. The introduction is divided in four parts. The first part presents three aspects of the visual system: cellular, regional, and its functional interactions. The second part presents an overview of schizophrenia and its sensoiy-cognitive deficits. The third part presents an overview of illusory contour processing and the three models examined in the third article. Finally, advanced analysis methods for EEG are presented, including time- frequency methodology.The Introduction is followed by a synopsis of the main results in the articles as well as those obtained from the time-frequency analyses.Finally, the Discussion chapter is divided along three axes. The first axis discusses the time frequency analysis and proposes a novel statistical approach that is independent of the reference. The second axis contextualizes the first article and discusses the quality of the stimulus control and direction for further improvements. Finally, both neurophysiologic articles are contextualized by proposing future experiments and hypotheses that may serve to improve our understanding of schizophrenia on the one hand and visual functions more generally.

Cultural differences on seeking information: an eye tracking study

The main goal of this research is to investigate how people with different cultural background differ in their interaction style and visual behavior on search engine results pages (SERP), more specifically between groups from the Middle Eastern region vs. Western Europe. We conducted a controlled eye-tracking experiment to explore and evaluate the visual behavior of Arabs and Spaniardusers when scanning through the first page of the search results in Google. Big differences can be observed in the 4 aspects studied: U.A.E. participants stayed on the SERPs for longer, they read more results and they read each snippet in a more complete way than Spaniards. In Spain, people tended to scan the SERP, reading less text on each snippet, and choose a result among the first top rankedones without hardly seeing those in bottom positions.

Decoding stimulus-related information from single-trial EEG responses based on voltage topographies

Neuroimaging studies typically compare experimental conditions using average brain responses, thereby overlooking the stimulus-related information conveyed by distributed spatio-temporal patterns of single-trial responses. Here, we take advantage of this rich information at a single-trial level to decode stimulus-related signals in two event-related potential (ERP) studies. Our method models the statistical distribution of the voltage topographies with a Gaussian Mixture Model (GMM), which reduces the dataset to a number of representative voltage topographies. The degree of presence of these topographies across trials at specific latencies is then used to classify experimental conditions. We tested the algorithm using a cross-validation procedure in two independent EEG datasets. In the first ERP study, we classified left- versus right-hemifield checkerboard stimuli for upper and lower visual hemifields. In a second ERP study, when functional differences cannot be assumed, we classified initial versus repeated presentations of visual objects. With minimal a priori information, the GMM model provides neurophysiologically interpretable features - vis à vis voltage topographies - as well as dynamic information about brain function. This method can in principle be applied to any ERP dataset testing the functional relevance of specific time periods for stimulus processing, the predictability of subject's behavior and cognitive states, and the discrimination between healthy and clinical populations.

Olfactory cues potentiate learning of distant visuospatial information

The influence of proximal olfactory cues on place learning and memory was tested in two different spatial tasks. Rats were trained to find a hole leading to their home cage or a single food source in an array of petri dishes. The two apparatuses differed both by the type of reinforcement (return to the home cage or food reward) and the local characteristics of the goal (masked holes or salient dishes). In both cases, the goal was in a fixed location relative to distant visual landmarks and could be marked by a local olfactory cue. Thus, the position of the goal was defined by two sets of redundant cues, each of which was sufficient to allow the discrimination of the goal location. These experiments were conducted with two strains of hooded rats (Long-Evans and PVG), which show different speeds of acquisition in place learning tasks. They revealed that the presence of an olfactory cue marking the goal facilitated learning of its location and that the facilitation persisted after the removal of the cue. Thus, the proximal olfactory cue appeared to potentiate learning and memory of the goal location relative to distant environmental cues. This facilitating effect was only detected when the expression of spatial memory was not already optimal, i.e., during the early phase of acquisition. It was not limited to a particular strain.

Interhemispheric visual integration in humans explored by multimodal brain imaging

Résumé: Les récents progrès techniques de l'imagerie cérébrale non invasives ont permis d'améliorer la compréhension des différents systèmes fonctionnels cérébraux. Les approches multimodales sont devenues indispensables en recherche, afin d'étudier dans sa globalité les différentes caractéristiques de l'activité neuronale qui sont à la base du fonctionnement cérébral. Dans cette étude combinée d'imagerie par résonance magnétique fonctionnelle (IRMf) et d'électroencéphalographie (EEG), nous avons exploité le potentiel de chacune d'elles, soit respectivement la résolution spatiale et temporelle élevée. Les processus cognitifs, de perception et de mouvement nécessitent le recrutement d'ensembles neuronaux. Dans la première partie de cette thèse nous étudions, grâce à la combinaison des techniques IRMf et EEG, la réponse des aires visuelles lors d'une stimulation qui demande le regroupement d'éléments cohérents appartenant aux deux hémi-champs visuels pour en faire une seule image. Nous utilisons une mesure de synchronisation (EEG de cohérence) comme quantification de l'intégration spatiale inter-hémisphérique et la réponse BOLD (Blood Oxygenation Level Dependent) pour évaluer l'activité cérébrale qui en résulte. L'augmentation de la cohérence de l'EEG dans la bande beta-gamma mesurée au niveau des électrodes occipitales et sa corrélation linéaire avec la réponse BOLD dans les aires de VP/V4, reflète et visualise un ensemble neuronal synchronisé qui est vraisemblablement impliqué dans le regroupement spatial visuel. Ces résultats nous ont permis d'étendre la recherche à l'étude de l'impact que le contenu en fréquence des stimuli a sur la synchronisation. Avec la même approche, nous avons donc identifié les réseaux qui montrent une sensibilité différente à l'intégration des caractéristiques globales ou détaillées des images. En particulier, les données montrent que l'implication des réseaux visuels ventral et dorsal est modulée par le contenu en fréquence des stimuli. Dans la deuxième partie nous avons a testé l'hypothèse que l'augmentation de l'activité cérébrale pendant le processus de regroupement inter-hémisphérique dépend de l'activité des axones calleux qui relient les aires visuelles. Comme le Corps Calleux présente une maturation progressive pendant les deux premières décennies, nous avons analysé le développement de la fonction d'intégration spatiale chez des enfants âgés de 7 à 13 ans et le rôle de la myelinisation des fibres calleuses dans la maturation de l'activité visuelle. Nous avons combiné l'IRMf et la technique de MTI (Magnetization Transfer Imaging) afin de suivre les signes de maturation cérébrale respectivement sous l'aspect fonctionnel et morphologique (myelinisation). Chez lés enfants, les activations associées au processus d'intégration entre les hémi-champs visuels sont, comme chez l'adulte, localisées dans le réseau ventral mais se limitent à une zone plus restreinte. La forte corrélation que le signal BOLD montre avec la myelinisation des fibres du splenium est le signe de la dépendance entre la maturation des fonctions visuelles de haut niveau et celle des connections cortico-corticales. Abstract: Recent advances in non-invasive brain imaging allow the visualization of the different aspects of complex brain dynamics. The approaches based on a combination of imaging techniques facilitate the investigation and the link of multiple aspects of information processing. They are getting a leading tool for understanding the neural basis of various brain functions. Perception, motion, and cognition involve the formation of cooperative neuronal assemblies distributed over the cerebral cortex. In this research, we explore the characteristics of interhemispheric assemblies in the visual brain by taking advantage of the complementary characteristics provided by EEG (electroencephalography) and fMRI (Functional Magnetic Resonance Imaging) techniques. These are the high temporal resolution for EEG and high spatial resolution for fMRI. In the first part of this thesis we investigate the response of the visual areas to the interhemispheric perceptual grouping task. We use EEG coherence as a measure of synchronization and BOLD (Blood Oxygenar tion Level Dependent) response as a measure of the related brain activation. The increase of the interhemispheric EEG coherence restricted to the occipital electrodes and to the EEG beta band and its linear relation to the BOLD responses in VP/V4 area points to a trans-hemispheric synchronous neuronal assembly involved in early perceptual grouping. This result encouraged us to explore the formation of synchronous trans-hemispheric networks induced by the stimuli of various spatial frequencies with this multimodal approach. We have found the involvement of ventral and medio-dorsal visual networks modulated by the spatial frequency content of the stimulus. Thus, based on the combination of EEG coherence and fMRI BOLD data, we have identified visual networks with different sensitivity to integrating low vs. high spatial frequencies. In the second part of this work we test the hypothesis that the increase of brain activity during perceptual grouping depends on the activity of callosal axons interconnecting the visual areas that are involved. To this end, in children of 7-13 years, we investigated functional (functional activation with fMRI) and morphological (myelination of the corpus callosum with Magnetization Transfer Imaging (MTI)) aspects of spatial integration. In children, the activation associated with the spatial integration across visual fields was localized in visual ventral stream and limited to a part of the area activated in adults. The strong correlation between individual BOLD responses in .this area and the myelination of the splenial system of fibers points to myelination as a significant factor in the development of the spatial integration ability.

Exposure to superfluous information reduces cooperation and increases antisocial punishment in reputation-based interactions

Human cooperation is often based on reputation gained from previous interactions with third parties. Such reputation can be built on generous or punitive actions, and both, one's own reputation and the reputation of others have been shown to influence decision making in experimental games that control for confounding variables. Here we test how reputation-based cooperation and punishment react to disruption of the cognitive processing in different kinds of helping games with observers. Saying a few superfluous words before each interaction was used to possibly interfere with working memory. In a first set of experiments, where reputation could only be based on generosity, the disruption reduced the frequency of cooperation and lowered mean final payoffs. In a second set of experiments where reputation could only be based on punishment, the disruption increased the frequency of antisocial punishment (i.e. of punishing those who helped) and reduced the frequency of punishing defectors. Our findings suggest that working memory can easily be constraining in reputation-based interactions within experimental games, even if these games are based on a few simple rules with a visual display that provides all the information the subjects need to play the strategies predicted from current theory. Our findings also highlight a weakness of experimental games, namely that they can be very sensitive to environmental variation and that quantitative conclusions about antisocial punishment or other behavioral strategies can easily be misleading.