Multimodal semantic-associative collateral labelling and indexing of still images

A novel framework for multimodal semantic-associative collateral image labelling, aiming at associating image regions with textual keywords, is described. Both the primary image and collateral textual modalities are exploited in a cooperative and complementary fashion. The collateral content and context based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix, of the visual keywords, A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. Finally, we use Self Organising Maps to examine the classification and retrieval effectiveness of the proposed high-level image feature vector model which is constructed based on the image labelling results.

Semantic-associative visual content labelling and retrieval: A multimodal approach

A novel framework referred to as collaterally confirmed labelling (CCL) is proposed, aiming at localising the visual semantics to regions of interest in images with textual keywords. Both the primary image and collateral textual modalities are exploited in a mutually co-referencing and complementary fashion. The collateral content and context-based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix of the visual keywords. A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. We introduce a novel high-level visual content descriptor that is devised for performing semantic-based image classification and retrieval. The proposed image feature vector model is fundamentally underpinned by the CCL framework. Two different high-level image feature vector models are developed based on the CCL labelling of results for the purposes of image data clustering and retrieval, respectively. A subset of the Corel image collection has been used for evaluating our proposed method. The experimental results to-date already indicate that the proposed semantic-based visual content descriptors outperform both traditional visual and textual image feature models. (C) 2007 Elsevier B.V. All rights reserved.

Queue-based agent architecture for multimodal interfaces

This paper presents a queue-based agent architecture for multimodal interfaces. Using a novel approach to intelligently organise both agents and input data, this system has the potential to outperform current state-of-the-art multimodal systems, while at the same time allowing greater levels of interaction and flexibility. This assertion is supported by simulation test results showing that significant improvements can be obtained over normal sequential agent scheduling architectures. For real usage, this translates into faster, more comprehensive systems, without the limited application domain that restricts current implementations.

Dynamic Niche Clustering: a fuzzy variable radius niching technique for multimodal optimisation in GAs

This paper describes the recent developments and improvements made to the variable radius niching technique called Dynamic Niche Clustering (DNC). DNC is fitness sharing based technique that employs a separate population of overlapping fuzzy niches with independent radii which operate in the decoded parameter space, and are maintained alongside the normal GA population. We describe a speedup process that can be applied to the initial generation which greatly reduces the complexity of the initial stages. A split operator is also introduced that is designed to counteract the excessive growth of niches, and it is shown that this improves the overall robustness of the technique. Finally, the effect of local elitism is documented and compared to the performance of the basic DNC technique on a selection of 2D test functions. The paper is concluded with a view to future work to be undertaken on the technique.

TESSA - Toolkit for experimentation with multimodal sensory substitution and augmentation

TESSA is a toolkit for experimenting with sensory augmentation. It includes hardware and software to facilitate rapid prototyping of interfaces that can enhance one sense using information gathered from another sense. The toolkit contains a range of sensors (e.g. ultrasonics, temperature sensors) and actuators (e.g. tactors or stereo sound), designed modularly so that inputs and outputs can be easily swapped in and out and customized using TESSA’s graphical user interface (GUI), with “real time” feedback. The system runs on a Raspberry Pi with a built-in touchscreen, providing a compact and portable form that is amenable for field trials. At CHI Interactivity, the audience will have the opportunity to experience sensory augmentation effects using this system, and design their own sensory augmentation interfaces.

Robust multimodal face and fingerprint fusion in the presence of spoofing attacks

Anti-spoofing is attracting growing interest in biometrics, considering the variety of fake materials and new means to attack biometric recognition systems. New unseen materials continuously challenge state-of-the-art spoofing detectors, suggesting for additional systematic approaches to target anti-spoofing. By incorporating liveness scores into the biometric fusion process, recognition accuracy can be enhanced, but traditional sum-rule based fusion algorithms are known to be highly sensitive to single spoofed instances. This paper investigates 1-median filtering as a spoofing-resistant generalised alternative to the sum-rule targeting the problem of partial multibiometric spoofing where m out of n biometric sources to be combined are attacked. Augmenting previous work, this paper investigates the dynamic detection and rejection of livenessrecognition pair outliers for spoofed samples in true multi-modal configuration with its inherent challenge of normalisation. As a further contribution, bootstrap aggregating (bagging) classifiers for fingerprint spoof-detection algorithm is presented. Experiments on the latest face video databases (Idiap Replay- Attack Database and CASIA Face Anti-Spoofing Database), and fingerprint spoofing database (Fingerprint Liveness Detection Competition 2013) illustrate the efficiency of proposed techniques.

Multi-level Input Fusion in multi-modal systems

Context-aware multimodal interactive systems aim to adapt to the needs and behavioural patterns of users and offer a way forward for enhancing the efficacy and quality of experience (QoE) in human-computer interaction. The various modalities that constribute to such systems each provide a specific uni-modal response that is integratively presented as a multi-modal interface capable of interpretation of multi-modal user input and appropriately responding to it through dynamically adapted multi-modal interactive flow management , This paper presents an initial background study in the context of the first phase of a PhD research programme in the area of optimisation of data fusion techniques to serve multimodal interactivite systems, their applications and requirements.

Investigation of the effects of the novel anticonvulsant compound carisbamate(RWJ-333369) on rat piriform cortical neurones in vitro

Background and purpose: Carisbamate is being developed for adjuvant treatment of partial onset epilepsy. Carisbamate produces anticonvulsant effects in primary generalized, complex partial and absence-type seizure models, and exhibits neuroprotective and antiepileptogenic properties in rodent epilepsy models. Phase IIb clinical trials of carisbamate demonstrated efficacy against partial onset seizures; however, its mechanisms of action remain unknown. Here, we report the effects of carisbamate on membrane properties, evoked and spontaneous synaptic transmission and induced epileptiform discharges in layer II-III neurones in piriform cortical brain slices. Experimental approach: Effects of carisbamate were investigated in rat piriform cortical neurones by using intracellular electrophysiological recordings. Key results: Carisbamate (50–400 mmol·L-1) reversibly decreased amplitude, duration and rise-time of evoked action potentials and inhibited repetitive firing, consistent with use-dependent Na+ channel block; 150–400 mmol·L-1 carisbamate reduced neuronal input resistance, without altering membrane potential. After microelectrode intracellular Cl- loading, carisbamate depolarized cells, an effect reversed by picrotoxin. Carisbamate (100–400 mmol·L-1) also selectively depressed lateral olfactory tract-afferent evoked excitatory synaptic transmission (opposed by picrotoxin), consistent with activation of a presynaptic Cl conductance. Lidocaine (40–320 mmol·L-1) mimicked carisbamate, implying similar modes of action. Carisbamate (300–600 mmol·L-1) had no effect on spontaneous GABAA miniature inhibitory postsynaptic currents and at lower concentrations (50–200 mmol·L-1) inhibited Mg2+-free or 4-aminopyridine-induced seizure-like discharges. Conclusions and implications: Carisbamate blocked evoked action potentials use-dependently, consistent with a primary action on Na+ channels and increased Cl- conductances presynaptically and, under certain conditions, postsynaptically to selectively depress excitatory neurotransmission in piriform cortical layer Ia-afferent terminals.

Nerve terminal GABAA receptors activate Ca2+/calmodulin-dependent signaling to inhibit voltage-gated Ca2+ influx and glutamate release

-Aminobutyric acid type A (GABAA) receptors, a family of Cl-permeable ion channels, mediate fast synaptic inhibition as postsynaptically enriched receptors for -aminobutyric acid at GABAergic synapses. Here we describe an alternative type of inhibition mediated byGABAA receptors present on neocortical glutamatergic nerve terminals and examine the underlying signaling mechanism(s). By monitoring the activity of the presynaptic CaM kinase II/synapsin I signaling pathway in isolated nerve terminals, we demonstrate that GABAA receptor activation correlated with an increase in basal intraterminal [Ca2]i. Interestingly, this activation of GABAA receptors resulted in a reduction of subsequent depolarization-evoked Ca2 influx, which thereby led to an inhibition of glutamate release. To investigate how the observed GABAA receptor-mediated modulation operates, we determined the sensitivity of this process to the Na-K-2Cl cotransporter 1 antagonist bumetanide, as well as substitution of Ca2 with Ba2, or Ca2/calmodulin inhibition by W7. All of these treatments abolished the modulation by GABAA receptors. Application of selective antagonists of voltage-gated Ca2 channels (VGCCs) revealed that the GABAA receptor-mediated modulation of glutamate release required the specific activity of L- and R-type VGCCs. Crucially, the inhibition of release by these receptors was abolished in terminals isolated from R-type VGCC knock-out mice. Together, our results indicate that a functional coupling between nerve terminal GABAA receptors and L- or R-type VGCCs is mediated by Ca2/calmodulin-dependent signaling. This mechanism provides a GABA-mediated control of glutamatergic synaptic activity by a direct inhibition of glutamate release.

G-protein-coupled-receptor-mediated presynaptic inhibition in the cerebellum

Throughout the central nervous system a dominant form of inhibition of neurotransmitter release from presynaptic terminals is mediated by G-protein-coupled receptors (GPCRs). Neurotransmitter release is typically induced by action potentials (APs), but can also occur spontaneously. Presynaptic inhibition by GPCRs has been associated with modulation of voltage-dependent ion channels. However, electrophysiological recordings of spontaneous, AP-independent (so-called ‘miniature’) postsynaptic events reveal an additional, important form of GPCR-mediated presynaptic inhibition, distinct from effects on ionic conductances and consistent with a direct action on the vesicle release machinery. Recent studies suggest that such miniature events might be of physiological relevance not only in signalling but also in development. In the cerebellum, neurotransmitter release onto Purkinje cells occurs by AP-dependent and AP-independent pathways. Here, I focus on inhibitory synapses between interneurons and Purkinje cells, which are subject to strong, identifiable regulation by endogenous GPCR agonists, to consider mechanisms of GPCR-mediated presynaptic inhibition.

Molecule modeling of human pentameric alpha(7) neuronal nicotinic acetylcholine receptor and its interaction with its agonist and competitive antagonist

The nicotinic Acetylcholine Receptor (nAChR) is the major class of neurotransmitter receptors that is involved in many neurodegenerative conditions such as schizophrenia, Alzheimer's and Parkinson's diseases. The N-terminal region or Ligand Binding Domain (LBD) of nAChR is located at pre- and post-synaptic nervous system, which mediates synaptic transmission. nAChR acts as the drug target for agonist and competitive antagonist molecules that modulate signal transmission at the nerve terminals. Based on Acetylcholine Binding Protein (AChBP) from Lymnea stagnalis as the structural template, the homology modeling approach was carried out to build three dimensional model of the N-terminal region of human alpha(7)nAChR. This theoretical model is an assembly of five alpha(7) subunits with 5 fold axis symmetry, constituting a channel, with the binding picket present at the interface region of the subunits. alpha-netlrotoxin is a potent nAChR competitive antagonist that readily blocks the channel resulting in paralysis. The molecular interaction of alpha-Bungarotoxin, a long chain alpha-neurotoxin from (Bungarus multicinctus) and human alpha(7)nAChR seas studied. Agonists such as acetylcholine, nicotine, which are used in it diverse array of biological activities, such as enhancements of cognitive performances, were also docked with the theoretical model of human alpha(7)nAChR. These docked complexes were analyzed further for identifying the crucial residues involved in interaction. These results provide the details of interaction of agonists and competitive antagonists with three dimensional model of the N-terminal region of human alpha(7)nAChR and thereby point to the design of novel lead compounds.

G protein ss gamma subunits mediate presynaptic inhibition of transmitter release from rat superior cervical ganglion neurones in culture

The activation of presynaptic G protein-coupled receptors (GPCRs) is widely reported to inhibit transmitter release; however, the lack of accessibility of many presynaptic terminals has limited direct analysis of signalling mediators. We studied GPCR-mediated inhibition of fast cholinergic transmission between superior cervical ganglion neurones (SCGNs) in culture. The adrenoceptor agonist noradrenaline (NA) caused a dose-related reduction in evoked excitatory postsynaptic potentials (EPSPs). NA-induced EPSP decrease was accompanied by effects on the presynaptic action potential (AP), reducing AP duration and amplitude of the after-hyperpolarization (AHP), without affecting the pre- and postsynaptic membrane potential. All effects of NA were blocked by yohimbine and synaptic transmission was reduced by clonidine, consistent with an action at presynaptic alpha 2-adrenoceptors. NA-induced inhibition of transmission was sensitive to pre-incubation of SCGNs with pertussis toxin (PTX), implicating the involvement of G alpha(i)/(o)beta y subunits. Expression of G alpha transducin, an agent which sequesters G protein beta gamma (G beta y) subunits, in the presynaptic neurone caused a time-dependent attenuation of NA-induced inhibition. Injection of purified G beta gamma subunits into the presynaptic neurone inhibited transmission, and also reduced the AHP amplitude. Furthermore, NA-induced inhibition was occluded by pre-injection of G beta gamma subunits. The Ca2+ channel blocker Cd2+ mimicked NA effects on transmitter release. Cd2+, NA and G beta gamma subunits also inhibited somatic Ca2+ current. In contrast to effects on AP-evoked transmitter release, NA had no clear action on AP-independent EPSPs induced by hypertonic solutions. These results demonstrate that G beta gamma subunits functionally mediate inhibition of transmitter release by alpha 2-adrenoceptors and represent important regulators of synaptic transmission at mammalian presynaptic terminals.