Robust visual tracking based on online learning sparse representation

Handling appearance variations is a very challenging problem for visual tracking. Existing methods usually solve this problem by relying on an effective appearance model with two features: (1) being capable of discriminating the tracked target from its background, (2) being robust to the target's appearance variations during tracking. Instead of integrating the two requirements into the appearance model, in this paper, we propose a tracking method that deals with these problems separately based on sparse representation in a particle filter framework. Each target candidate defined by a particle is linearly represented by the target and background templates with an additive representation error. Discriminating the target from its background is achieved by activating the target templates or the background templates in the linear system in a competitive manner. The target's appearance variations are directly modeled as the representation error. An online algorithm is used to learn the basis functions that sparsely span the representation error. The linear system is solved via ℓ1 minimization. The candidate with the smallest reconstruction error using the target templates is selected as the tracking result. We test the proposed approach using four sequences with heavy occlusions, large pose variations, drastic illumination changes and low foreground-background contrast. The proposed approach shows excellent performance in comparison with two latest state-of-the-art trackers.

Visuotactile learning and body representation: an ERP study with rubber hands and rubber objects

We studied how the integration of seen and felt tactile stimulation modulates somatosensory processing, and investigated whether visuotactile integration depends on temporal contiguity of stimulation, and its coherence with a pre-existing body representation. During training, participants viewed a rubber hand or a rubber object that was tapped either synchronously with stimulation of their own hand, or in an uncorrelated fashion. In a subsequent test phase, somatosensory event-related potentials (ERPs) were recorded to tactile stimulation of the left or right hand, to assess how tactile processing was affected by previous visuotactile experience during training. An enhanced somatosensory N140 component was elicited after synchronous, compared with uncorrelated, visuotactile training, irrespective of whether participants viewed a rubber hand or rubber object. This early effect of visuotactile integration on somatosensory processing is interpreted as a candidate electrophysiological correlate of the rubber hand illusion that is determined by temporal contiguity, but not by pre-existing body representations. ERPmodulations were observed beyond 200msec post-stimulus, suggesting an attentional bias induced by visuotactile training. These late modulations were absent when the stimulation of a rubber hand and the participant’s own hand was uncorrelated during training, suggesting that pre-existing body representations may affect later stages of tactile processing.

Neural systems underlying learning and representation of global motion

We demonstrate performance-related changes in cortical and cerebellar activity. The largest learning-dependent changes were observed in the anterior lateral cerebellum, where the extent and intensity of activation correlated inversely with psychophysical performance. After learning had occurred (a few minutes), the cerebellar activation almost disappeared; however, it was restored when the subjects were presented with a novel, untrained direction of motion for which psychophysical performance also reverted to chance level. Similar reductions in the extent and intensity of brain activations in relation to learning occurred in the superior colliculus, anterior cingulate, and parts of the extrastriate cortex. The motion direction-sensitive middle temporal visual complex was a notable exception, where there was an expansion of the cortical territory activated by the trained stimulus. Together, these results indicate that the learning and representation of visual motion discrimination are mediated by different, but probably interacting, neuronal subsystems.

Representation and learning of visual information for pose recognition

Recovering position from sensor information is an important problem in mobile robotics, known as localisation. Localisation requires a map or some other description of the environment to provide the robot with a context to interpret sensor data. The mobile robot system under discussion is using an artificial neural representation of position. Building a geometrical map of the environment with a single camera and artificial neural networks is difficult. Instead it would be simpler to learn position as a function of the visual input. Usually when learning images, an intermediate representation is employed. An appropriate starting point for biologically plausible image representation is the complex cells of the visual cortex, which have invariance properties that appear useful for localisation. The effectiveness for localisation of two different complex cell models are evaluated. Finally the ability of a simple neural network with single shot learning to recognise these representations and localise a robot is examined.

The impact of context on pattern category learning and representation

Context traditionally has been regarded in vision research as a determinant for the interpretation of sensory information on the basis of previously acquired knowledge. Here we propose a novel, complementary perspective by showing that context also specifically affects visual category learning. In two experiments involving sets of Compound Gabor patterns we explored how context, as given by the stimulus set to be learned, affects the internal representation of pattern categories. In Experiment 1, we changed the (local) context of the individual signal classes by changing the configuration of the learning set. In Experiment 2, we varied the (global) context of a fixed class configuration by changing the degree of signal accentuation. Generalization performance was assessed in terms of the ability to recognize contrast-inverted versions of the learning patterns. Both contextual variations yielded distinct effects on learning and generalization thus indicating a change in internal category representation. Computer simulations suggest that the latter is related to changes in the set of attributes underlying the production rules of the categories. The implications of these findings for phenomena of contrast (in)variance in visual perception are discussed.

Noradrenergic modulatoins of odor learning and odor representation in the rat

How experience alters neuronal ensemble dynamics and how locus coeruleus-mediated norepinephrine release facilitates memory formation in the brain are the topics of this thesis. Here we employed a visualization technique, cellular compartment analysis of temporal activity by fluorescence in situ hybridization (catFISH), to assess activation patterns of neuronal ensembles in the olfactory bulb (OB) and anterior piriform cortex (aPC) to repeated odor inputs. Two associative learning models were used, early odor preference learning in rat pups and adult rat go-no-go odor discrimination learning. With catFISH of an immediate early gene, Arc, we showed that odor representation in the OB and aPC was sparse (~5-10%) and widely distributed. Odor associative learning enhanced the stability of the rewarded odor representation in the OB and aPC. The stable component, indexed by the overlap between the two ensembles activated by the rewarded odor at two time points, increased from ~25% to ~50% (p = 0.004-1.43E⁻4; Chapter 3 and 4). Adult odor discrimination learning promoted pattern separation between rewarded and unrewarded odor representations in the aPC. The overlap between rewarded and unrewarded odor representations reduced from ~25% to ~14% (p = 2.28E⁻⁵). However, learning an odor mixture as a rewarded odor increased the overlap of the component odor representations in the aPC from ~23% to ~44% (p = 0.010; Chapter 4). Blocking both α- and β-adrenoreceptors in the aPC prevented highly similar odor discrimination learning in adult rats, and reduced OB mitral and granule ensemble stability to the rewarded odor. Similar treatment in the OB only slowed odor discrimination learning. However, OB adrenoceptor blockade disrupted pattern separation and ensemble stability in the aPC when the rats demonstrated deficiency in discrimination (Chapter 5). In another project, the role of α₂-adrenoreceptors in the OB during early odor preference learning was studied. OB α2-adrenoceptor activation was necessary for odor learning in rat pups. α₂-adrenoceptor activation was additive with β-adrenoceptor mediated signalling to promote learning (Chapter 2). Together, these experiments suggest that odor representations are highly adaptive at the early stages of odor processing. The OB and aPC work in concert to support odor learning and top-down adrenergic input exerts a powerful modulation on both learning and odor representation.

Mid-level concept learning with visual contextual ontologies and probabilistic inference for image annotation

To date, automatic recognition of semantic information such as salient objects and mid-level concepts from images is a challenging task. Since real-world objects tend to exist in a context within their environment, the computer vision researchers have increasingly incorporated contextual information for improving object recognition. In this paper, we present a method to build a visual contextual ontology from salient objects descriptions for image annotation. The ontologies include not only partOf/kindOf relations, but also spatial and co-occurrence relations. A two-step image annotation algorithm is also proposed based on ontology relations and probabilistic inference. Different from most of the existing work, we specially exploit how to combine representation of ontology, contextual knowledge and probabilistic inference. The experiments show that image annotation results are improved in the LabelMe dataset.

Learning spatial concepts from RatSLAM representations

RatSLAM is a biologically-inspired visual SLAM and navigation system that has been shown to be effective indoors and outdoors on real robots. The spatial representation at the core of RatSLAM, the experience map, forms in a distributed fashion as the robot learns the environment. The activity in RatSLAM’s experience map possesses some geometric properties, but still does not represent the world in a human readable form. A new system, dubbed RatChat, has been introduced to enable meaningful communication with the robot. The intention is to use the “language games” paradigm to build spatial concepts that can be used as the basis for communication. This paper describes the first step in the language game experiments, showing the potential for meaningful categorization of the spatial representations in RatSLAM.

Three primary school students’ cognition about 3D rotation in a virtual reality learning environment

This paper reports on three primary school students’ explorations of 3D rotation in a virtual reality learning environment (VRLE) named VRMath. When asked to investigate if you would face the same direction when you turn right 45 degrees first then roll up 45 degrees, or when you roll up 45 degrees first then turn right 45 degrees, the students found that the different order of the two turns ended up with different directions in the VRLE. This was contrary to the students’ prior predictions based on using pen, paper and body movements. The findings of this study showed the difficulty young children have in perceiving and understanding the non-commutative nature of 3D rotation and the power of the computational VRLE in giving students experiences that they rarely have in real life with 3D manipulations and 3D mental movements.

Sequential nonlinear manifold learning

The computation of compact and meaningful representations of high dimensional sensor data has recently been addressed through the development of Nonlinear Dimensional Reduction (NLDR) algorithms. The numerical implementation of spectral NLDR techniques typically leads to a symmetric eigenvalue problem that is solved by traditional batch eigensolution algorithms. The application of such algorithms in real-time systems necessitates the development of sequential algorithms that perform feature extraction online. This paper presents an efficient online NLDR scheme, Sequential-Isomap, based on incremental singular value decomposition (SVD) and the Isomap method. Example simulations demonstrate the validity and significant potential of this technique in real-time applications such as autonomous systems.

A statistical framework for natural feature representation

This paper presents a robust stochastic framework for the incorporation of visual observations into conventional estimation, data fusion, navigation and control algorithms. The representation combines Isomap, a non-linear dimensionality reduction algorithm, with expectation maximization, a statistical learning scheme. The joint probability distribution of this representation is computed offline based on existing training data. The training phase of the algorithm results in a nonlinear and non-Gaussian likelihood model of natural features conditioned on the underlying visual states. This generative model can be used online to instantiate likelihoods corresponding to observed visual features in real-time. The instantiated likelihoods are expressed as a Gaussian mixture model and are conveniently integrated within existing non-linear filtering algorithms. Example applications based on real visual data from heterogenous, unstructured environments demonstrate the versatility of the generative models.

A stochastic model for natural feature representation

This paper presents a robust stochastic model for the incorporation of natural features within data fusion algorithms. The representation combines Isomap, a non-linear manifold learning algorithm, with Expectation Maximization, a statistical learning scheme. The representation is computed offline and results in a non-linear, non-Gaussian likelihood model relating visual observations such as color and texture to the underlying visual states. The likelihood model can be used online to instantiate likelihoods corresponding to observed visual features in real-time. The likelihoods are expressed as a Gaussian Mixture Model so as to permit convenient integration within existing nonlinear filtering algorithms. The resulting compactness of the representation is especially suitable to decentralized sensor networks. Real visual data consisting of natural imagery acquired from an Unmanned Aerial Vehicle is used to demonstrate the versatility of the feature representation.