On the annealed VC entropy for margin classifiers: A statistical mechanics study

Using techniques from Statistical Physics, the annealed VC entropy for hyperplanes in high dimensional spaces is calculated as a function of the margin for a spherical Gaussian distribution of inputs.

Advances in large margin classifiers

We apply methods of Statistical Mechanics to study the generalization performance of Support vector Machines in large data spaces.

Associative knowledge controls deployment of visual selective attention

According to some models of visual selective attention, objects in a scene activate corresponding neural representations, which compete for perceptual awareness and motor behavior. During a visual search for a target object, top-down control exerted by working memory representations of the target's defining properties resolves competition in favor of the target. These models, however, ignore the existence of associative links among object representations. Here we show that such associations can strongly influence deployment of attention in humans. In the context of visual search, objects associated with the target were both recalled more often and recognized more accurately than unrelated distractors. Notably, both target and associated objects competitively weakened recognition of unrelated distractors and slowed responses to a luminance probe. Moreover, in a speeded search protocol, associated objects rendered search both slower and less accurate. Finally, the first saccades after onset of the stimulus array were more often directed toward associated than control items.

The n-tuple network: coping with sub-optimality

The subject of this thesis is the n-tuple net.work (RAMnet). The major advantage of RAMnets is their speed and the simplicity with which they can be implemented in parallel hardware. On the other hand, this method is not a universal approximator and the training procedure does not involve the minimisation of a cost function. Hence RAMnets are potentially sub-optimal. It is important to understand the source of this sub-optimality and to develop the analytical tools that allow us to quantify the generalisation cost of using this model for any given data. We view RAMnets as classifiers and function approximators and try to determine how critical their lack of' universality and optimality is. In order to understand better the inherent. restrictions of the model, we review RAMnets showing their relationship to a number of well established general models such as: Associative Memories, Kamerva's Sparse Distributed Memory, Radial Basis Functions, General Regression Networks and Bayesian Classifiers. We then benchmark binary RAMnet. model against 23 other algorithms using real-world data from the StatLog Project. This large scale experimental study indicates that RAMnets are often capable of delivering results which are competitive with those obtained by more sophisticated, computationally expensive rnodels. The Frequency Weighted version is also benchmarked and shown to perform worse than the binary RAMnet for large values of the tuple size n. We demonstrate that the main issues in the Frequency Weighted RAMnets is adequate probability estimation and propose Good-Turing estimates in place of the more commonly used :Maximum Likelihood estimates. Having established the viability of the method numerically, we focus on providillg an analytical framework that allows us to quantify the generalisation cost of RAMnets for a given datasetL. For the classification network we provide a semi-quantitative argument which is based on the notion of Tuple distance. It gives a good indication of whether the network will fail for the given data. A rigorous Bayesian framework with Gaussian process prior assumptions is given for the regression n-tuple net. We show how to calculate the generalisation cost of this net and verify the results numerically for one dimensional noisy interpolation problems. We conclude that the n-tuple method of classification based on memorisation of random features can be a powerful alternative to slower cost driven models. The speed of the method is at the expense of its optimality. RAMnets will fail for certain datasets but the cases when they do so are relatively easy to determine with the analytical tools we provide.

Hierarchical classification of G-protein-coupled receptors with data-driven selection of attributes and classifiers

We address the important bioinformatics problem of predicting protein function from a protein's primary sequence. We consider the functional classification of G-Protein-Coupled Receptors (GPCRs), whose functions are specified in a class hierarchy. We tackle this task using a novel top-down hierarchical classification system where, for each node in the class hierarchy, the predictor attributes to be used in that node and the classifier to be applied to the selected attributes are chosen in a data-driven manner. Compared with a previous hierarchical classification system selecting classifiers only, our new system significantly reduced processing time without significantly sacrificing predictive accuracy.

Increased salience of gains versus decreased associative learning differentiate bipolar disorder from schizophrenia during incentive decision making

Background Abnormalities in incentive decision making, typically assessed using the Iowa Gambling Task (IGT), have been reported in both schizophrenia (SZ) and bipolar disorder (BD). We applied the Expectancy-Valence (E-V) model to determine whether motivational, cognitive and response selection component processes of IGT performance are differentially affected in SZ and BD. Method Performance on the IGT was assessed in 280 individuals comprising 70 remitted patients with SZ, 70 remitted patients with BD and 140 age-, sex-and IQ-matched healthy individuals. Based on the E-V model, we extracted three parameters, 'attention to gains or loses', 'expectancy learning' and 'response consistency', that respectively reflect motivational, cognitive and response selection influences on IGT performance. Results Both patient groups underperformed in the IGT compared to healthy individuals. However, the source of these deficits was diagnosis specific. Associative learning underlying the representation of expectancies was disrupted in SZ whereas BD was associated with increased incentive salience of gains. These findings were not attributable to non-specific effects of sex, IQ, psychopathology or medication. Conclusions Our results point to dissociable processes underlying abnormal incentive decision making in BD and SZ that could potentially be mapped to different neural circuits. © 2012 Cambridge University Press.

Mean field methods for Gaussian process classifiers

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Stretching the life of Twitter classifiers with time-stamped semantic graphs

Social media has become an effective channel for communicating both trends and public opinion on current events. However the automatic topic classification of social media content pose various challenges. Topic classification is a common technique used for automatically capturing themes that emerge from social media streams. However, such techniques are sensitive to the evolution of topics when new event-dependent vocabularies start to emerge (e.g., Crimea becoming relevant to War Conflict during the Ukraine crisis in 2014). Therefore, traditional supervised classification methods which rely on labelled data could rapidly become outdated. In this paper we propose a novel transfer learning approach to address the classification task of new data when the only available labelled data belong to a previous epoch. This approach relies on the incorporation of knowledge from DBpedia graphs. Our findings show promising results in understanding how features age, and how semantic features can support the evolution of topic classifiers.

Age differences in associative and strategic processes in human conditional learning

Research indicates associative and strategic deficits mediate age related deficits in memory, whereas simple associative processes are independent of strategic processing and strategic processes mediate resistance to interference. The present study showed age-related deficits in a contingency learning task, although older participants' resistance to interference was not disproportionately affected. Recognition memory predicted discrimination, whereas general cognitive ability predicted resistance to interference, suggesting differentiation between associative and strategic processes in learning and memory, and age declines in associative processes. Older participants' generalisation of associative strength from existing to novel stimulus-response associations was consistent with elemental learning theories, whereas configural models predicted younger participants' responses. This is consistent with associative deficits and reliance on item-level representations in memory during later life. © 2011 Psychology Press Ltd.

Novel formulation strategies for the fabrication of lyophilised orally disintegrating tablets

Orally disintegrating Tablets (ODTs), also known as fast-disintegrating, fast-melt or fast-dissolving tablets, are a relatively novel dosage technology that involves the rapid disintegration or dissolution of the dosage form into a solution or suspension in the mouth without the need for water. The solution containing the active ingredients is swallowed, and the active ingredients are then absorbed through the gastrointestinal epithelium to reach the target and produce the desired effect. Formulation of ODTs was originally developed to address swallowing difficulties of conventional solid oral dosage forms (tablets and capsules) experienced by wide range of patient population, especially children and elderly. The current work investigates the formulation and development of ODTs prepared by freeze drying. Initial studies focused on formulation parameters that influence the manufacturing process and performance of lyophilised tablets based on excipients used in commercial products (gelatin and saccharides). The second phase of the work was followed up by comprehensive studies to address the essential need to create saccharide free ODTs using naturally accruing amino acids individually or in combinations. Furthermore, a factorial design study was carried out to investigate the feasibility of delivering multiparticulate systems of challenging drugs using a novel formulation that exploited the electrostatic associative interaction between gelatin and carrageenan. Finally, studies aimed to replace gelatin with ethically and morally accepted components to the end users were performed and the selected binder was used in factorial design studies to investigate and optimise ODT formulations that incorporated drugs with varies physicochemical properties. Our results show that formulation of elegant lyophilised ODTs with instant disintegration and adequate mechanical strength requires carful optimisation of gelatin concentration and bloom strength in addition to saccharide type and concentration. Successful formulation of saccharides free lyophilised ODTs requires amino acids that crystallise in the frozen state or display relatively high Tg', interact and integrate completely with the binder and, also, display short wetting time with the disintegrating medium. The use of an optimised mixture of gelatin, carrageenan and alanine was able to create viscous solutions to suspend multiparticulate systems and at the same time provide tablets with short disintegration times and adequate mechanical properties. On the other hand, gum arabic showed an outstanding potential for use as a binder in the formulation of lyophilised ODTs. Compared to gelatin formulations, the use of gum arabic simplified the formulation stages, shortened the freeze drying cycles and produced tablets with superior performance in terms of the disintegration time and mechanical strength. Furthermore, formulation of lyophilised ODTs based on gum arabic showed capability to deliver diverse range of drugs with advantages over commercial products.

Classification of oximetry signals using Bayesian neural networks to assist in the detection of obstructive sleep apnoea syndrome

In the present study, multilayer perceptron (MLP) neural networks were applied to help in the diagnosis of obstructive sleep apnoea syndrome (OSAS). Oxygen saturation (SaO2) recordings from nocturnal pulse oximetry were used for this purpose. We performed time and spectral analysis of these signals to extract 14 features related to OSAS. The performance of two different MLP classifiers was compared: maximum likelihood (ML) and Bayesian (BY) MLP networks. A total of 187 subjects suspected of suffering from OSAS took part in the study. Their SaO2 signals were divided into a training set with 74 recordings and a test set with 113 recordings. BY-MLP networks achieved the best performance on the test set with 85.58% accuracy (87.76% sensitivity and 82.39% specificity). These results were substantially better than those provided by ML-MLP networks, which were affected by overfitting and achieved an accuracy of 76.81% (86.42% sensitivity and 62.83% specificity). Our results suggest that the Bayesian framework is preferred to implement our MLP classifiers. The proposed BY-MLP networks could be used for early OSAS detection. They could contribute to overcome the difficulties of nocturnal polysomnography (PSG) and thus reduce the demand for these studies.

Gaussian processes and SVM: Mean field results and leave-one-out

In this chapter, we elaborate on the well-known relationship between Gaussian processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational problems arising in GP and SVM. The first one is the calculation of the posterior mean for GP classifiers using a `naive' mean field approach. The second one is a leave-one-out estimator for the generalization error of SVM based on a linear response method. Simulation results on a benchmark dataset show similar performances for the GP mean field algorithm and the SVM algorithm. The approximate leave-one-out estimator is found to be in very good agreement with the exact leave-one-out error.

The TAP approach to intensive and extensive connectivity systems

The Thouless-Anderson-Palmer (TAP) approach was originally developed for analysing the Sherrington-Kirkpatrick model in the study of spin glass models and has been employed since then mainly in the context of extensively connected systems whereby each dynamical variable interacts weakly with the others. Recently, we extended this method for handling general intensively connected systems where each variable has only O(1) connections characterised by strong couplings. However, the new formulation looks quite different with respect to existing analyses and it is only natural to question whether it actually reproduces known results for systems of extensive connectivity. In this chapter, we apply our formulation of the TAP approach to an extensively connected system, the Hopfield associative memory model, showing that it produces identical results to those obtained by the conventional formulation.

Mapping dissociations in verb morphology

Substantial behavioural and neuropsychological evidence has been amassed to support the dual-route model of morphological processing, which distinguishes between a rule-based system for regular items (walk–walked, call–called) and an associative system for the irregular items (go–went). Some neural-network models attempt to explain the neuropsychological and brain-mapping dissociations in terms of single-system associative processing. We show that there are problems in the accounts of homogeneous networks in the light of recent brain-mapping evidence of systematic double-dissociation. We also examine the superior capabilities of more internally differentiated connectionist models, which, under certain conditions, display systematic double-dissociations. It appears that the more differentiation models show, the more easily they account for dissociation patterns, yet without implementing symbolic computations.

Formulation of multiparticulate systems as lyophilised orally disintegrating tablets

The current study aimed to exploit the electrostatic associative interaction between carrageenan and gelatin to optimise a formulation of lyophilised orally disintegrating tablets (ODTs) suitable for multiparticulate delivery. A central composite face centred (CCF) design was applied to study the influence of formulation variables (gelatin, carrageenan and alanine concentrations) on the crucial responses of the formulation (disintegration time, hardness, viscosity and pH). The disintegration time and viscosity were controlled by the associative interaction between gelatin and carrageenan upon hydration which forms a strong complex that increases the viscosity of the stock solution and forms tablet with higher resistant to disintegration in aqueous medium. Therefore, the levels of carrageenan, gelatin and their interaction in the formulation were the significant factors. In terms of hardness, increasing gelatin and alanine concentration was the most effective way to improve tablet hardness. Accordingly, optimum concentrations of these excipients were needed to find the best balance that fulfilled all formulation requirements. The revised model showed high degree of predictability and optimisation reliability and therefore was successful in developing an ODT formulation with optimised properties that were able deliver enteric coated multiparticulates of omeprazole without compromising their functionality.