Deep learning for computer vision: a comparison between convolutional neural networks and hierarchical temporal memories on object recognition tasks

In recent years, Deep Learning techniques have shown to perform well on a large variety of problems both in Computer Vision and Natural Language Processing, reaching and often surpassing the state of the art on many tasks. The rise of deep learning is also revolutionizing the entire field of Machine Learning and Pattern Recognition pushing forward the concepts of automatic feature extraction and unsupervised learning in general. However, despite the strong success both in science and business, deep learning has its own limitations. It is often questioned if such techniques are only some kind of brute-force statistical approaches and if they can only work in the context of High Performance Computing with tons of data. Another important question is whether they are really biologically inspired, as claimed in certain cases, and if they can scale well in terms of "intelligence". The dissertation is focused on trying to answer these key questions in the context of Computer Vision and, in particular, Object Recognition, a task that has been heavily revolutionized by recent advances in the field. Practically speaking, these answers are based on an exhaustive comparison between two, very different, deep learning techniques on the aforementioned task: Convolutional Neural Network (CNN) and Hierarchical Temporal memory (HTM). They stand for two different approaches and points of view within the big hat of deep learning and are the best choices to understand and point out strengths and weaknesses of each of them. CNN is considered one of the most classic and powerful supervised methods used today in machine learning and pattern recognition, especially in object recognition. CNNs are well received and accepted by the scientific community and are already deployed in large corporation like Google and Facebook for solving face recognition and image auto-tagging problems. HTM, on the other hand, is known as a new emerging paradigm and a new meanly-unsupervised method, that is more biologically inspired. It tries to gain more insights from the computational neuroscience community in order to incorporate concepts like time, context and attention during the learning process which are typical of the human brain. In the end, the thesis is supposed to prove that in certain cases, with a lower quantity of data, HTM can outperform CNN.

Imagery of a moving object: the role of occipital cortex and human MT/V5+

Visual imagery – similar to visual perception – activates feature-specific and category-specific visual areas. This is frequently observed in experiments where the instruction is to imagine stimuli that have been shown immediately before the imagery task. Hence, feature-specific activation could be related to the short-term memory retrieval of previously presented sensory information. Here, we investigated mental imagery of stimuli that subjects had not seen before, eliminating the effects of short-term memory. We recorded brain activation using fMRI while subjects performed a behaviourally controlled guided imagery task in predefined retinotopic coordinates to optimize sensitivity in early visual areas. Whole brain analyses revealed activation in a parieto-frontal network and lateral–occipital cortex. Region of interest (ROI) based analyses showed activation in left hMT/V5+. Granger causality mapping taking left hMT/V5+ as source revealed an imagery-specific directed influence from the left inferior parietal lobule (IPL). Interestingly, we observed a negative BOLD response in V1–3 during imagery, modulated by the retinotopic location of the imagined motion trace. Our results indicate that rule-based motion imagery can activate higher-order visual areas involved in motion perception, with a role for top-down directed influences originating in IPL. Lower-order visual areas (V1, V2 and V3) were down-regulated during this type of imagery, possibly reflecting inhibition to avoid visual input from interfering with the imagery construction. This suggests that the activation in early visual areas observed in previous studies might be related to short- or long-term memory retrieval of specific sensory experiences.

Neural networks engaged in tactile object manipulation: patterns of expression among healthy individuals

Somatosensory object discrimination has been shown to involve widespread cortical and subcortical structures in both cerebral hemispheres. In this study we aimed to identify the networks involved in tactile object manipulation by principal component analysis (PCA) of individual subjects. We expected to find more than one network.

Investigating human audio-visual object perception with a combination of hypothesis-generating and hypothesis-testing fMRI analysis tools

Primate multisensory object perception involves distributed brain regions. To investigate the network character of these regions of the human brain, we applied data-driven group spatial independent component analysis (ICA) to a functional magnetic resonance imaging (fMRI) data set acquired during a passive audio-visual (AV) experiment with common object stimuli. We labeled three group-level independent component (IC) maps as auditory (A), visual (V), and AV, based on their spatial layouts and activation time courses. The overlap between these IC maps served as definition of a distributed network of multisensory candidate regions including superior temporal, ventral occipito-temporal, posterior parietal and prefrontal regions. During an independent second fMRI experiment, we explicitly tested their involvement in AV integration. Activations in nine out of these twelve regions met the max-criterion (A < AV > V) for multisensory integration. Comparison of this approach with a general linear model-based region-of-interest definition revealed its complementary value for multisensory neuroimaging. In conclusion, we estimated functional networks of uni- and multisensory functional connectivity from one dataset and validated their functional roles in an independent dataset. These findings demonstrate the particular value of ICA for multisensory neuroimaging research and using independent datasets to test hypotheses generated from a data-driven analysis.

Use Of Conspecific Gestural Cues To Solve An Object-Choice Task In Tufted Capuchin Monkeys (Cebus Apella)

Capuchin monkeys, Cebus sp., utilize a wide array of gestural displays in the wild, including facial displays such as lip-smacking and bare-teeth displays. In captivity, they have been shown to respond to the head orientation of humans, show sensitivity to human attentional states, as well as follow human gazes behind barriers. In this study, I investigated whether tufted capuchin monkeys (Cebus apella) would attend to and utilize the gestural cues of a conspecific to obtain a hidden reward. Two capuchins faced each other in separate compartments of an apparatus with an open field in between. The open field contained two cups with holes on one side such that only one monkey, a so-called cuing monkey, could see the reward inside one of the cups. I then moved the cups toward the other signal-receiving monkey and assessed whether it would utilize untrained cues provided by the cuing monkey to select the cup containing the reward. Two of four female capuchin monkeys learned to select the cup containing the reward significantly more often than chance. Neither of these two monkeys performed over chance spontaneously, however, and the other two monkeys never performed above chance despite many blocks of trials. Successful choices by two monkeys to obtain hidden rewards provided experimental evidence that capuchin monkeys attend to and utilize the gestural cues of conspecifics.

The Use of Human and Conspecific Gestures By Capuchin Monkeys to Solve an Object-Choice Task

Most primates live in highly complex social systems, and therefore have evolved similarly complex methods of communicating with each other. One type of communication is the use of manual gestures, which are only found in primates. No substantial evidence exists indicating that monkeys use communicative gestures in the wild. However, monkeys may demonstrate the ability to learn and/or use gestures in certain experimental paradigms since they¿ve been shown to use other visual cues such as gaze. The purpose of this study was to investigate if ten brown capuchin monkeys (Cebus apella) were able to use gestural cues from monkeys and a pointing cue from a human to obtain a hidden reward. They were then tested to determine if they could transfer this skill from monkeys to humans and from humans to monkeys. One group of monkeys was trained and tested using a conspecific as the cue giver, and was then tested with a human cue-giver. The second group of monkeys began training and testing with a human cue giver, and was then tested with a monkey cue giver. I found that two monkeys were able to use gestural cues from conspecifics (e.g., reaching) to obtain a hidden reward and then transfer this ability to a pointing cue from a human. Four monkeys learned to use the human pointing cue first, and then transferred this ability to use the gestural cues from conspecifics to obtain a hidden reward. However, the number of trials it took for each monkey to transfer the ability varied considerably. Some subjects spontaneously transferred in the minimum number of trials needed to reach my criteria for successfully obtaining hidden rewards (N = 40 trials), while others needed a large number of trials to do so (e.g. N = 190 trials). Two subjects did not perform successfully in any of the conditions in which they were tested. One subject successfully used the human pointing cue and a human pointing plus vocalization cue, but did not learn the conspecific cue. One subject learned to use the conspecific cue but not the human pointing cue. This was the first study to test if brown capuchin monkeys could use gestural cues from conspecifics to solve an object choice task. The study was also the first to test if capuchins could transfer this skill from monkeys to humans and from humans to monkeys. Results showed that capuchin monkeys were able to flexibly use communicative gestures when they were both unintentionally given by a conspecific and intentionally given by a human to indicate a source of food.