Variational inference for Student-t MLP models

This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.

Classification of oximetry signals using Bayesian neural networks to assist in the detection of obstructive sleep apnoea syndrome

In the present study, multilayer perceptron (MLP) neural networks were applied to help in the diagnosis of obstructive sleep apnoea syndrome (OSAS). Oxygen saturation (SaO2) recordings from nocturnal pulse oximetry were used for this purpose. We performed time and spectral analysis of these signals to extract 14 features related to OSAS. The performance of two different MLP classifiers was compared: maximum likelihood (ML) and Bayesian (BY) MLP networks. A total of 187 subjects suspected of suffering from OSAS took part in the study. Their SaO2 signals were divided into a training set with 74 recordings and a test set with 113 recordings. BY-MLP networks achieved the best performance on the test set with 85.58% accuracy (87.76% sensitivity and 82.39% specificity). These results were substantially better than those provided by ML-MLP networks, which were affected by overfitting and achieved an accuracy of 76.81% (86.42% sensitivity and 62.83% specificity). Our results suggest that the Bayesian framework is preferred to implement our MLP classifiers. The proposed BY-MLP networks could be used for early OSAS detection. They could contribute to overcome the difficulties of nocturnal polysomnography (PSG) and thus reduce the demand for these studies.

Estimating conditional volatility with neural networks

It is well known that one of the obstacles to effective forecasting of exchange rates is heteroscedasticity (non-stationary conditional variance). The autoregressive conditional heteroscedastic (ARCH) model and its variants have been used to estimate a time dependent variance for many financial time series. However, such models are essentially linear in form and we can ask whether a non-linear model for variance can improve results just as non-linear models (such as neural networks) for the mean have done. In this paper we consider two neural network models for variance estimation. Mixture Density Networks (Bishop 1994, Nix and Weigend 1994) combine a Multi-Layer Perceptron (MLP) and a mixture model to estimate the conditional data density. They are trained using a maximum likelihood approach. However, it is known that maximum likelihood estimates are biased and lead to a systematic under-estimate of variance. More recently, a Bayesian approach to parameter estimation has been developed (Bishop and Qazaz 1996) that shows promise in removing the maximum likelihood bias. However, up to now, this model has not been used for time series prediction. Here we compare these algorithms with two other models to provide benchmark results: a linear model (from the ARIMA family), and a conventional neural network trained with a sum-of-squares error function (which estimates the conditional mean of the time series with a constant variance noise model). This comparison is carried out on daily exchange rate data for five currencies.

Neural network-based wind vector retrieval from satellite scatterometer data

Obtaining wind vectors over the ocean is important for weather forecasting and ocean modelling. Several satellite systems used operationally by meteorological agencies utilise scatterometers to infer wind vectors over the oceans. In this paper we present the results of using novel neural network based techniques to estimate wind vectors from such data. The problem is partitioned into estimating wind speed and wind direction. Wind speed is modelled using a multi-layer perceptron (MLP) and a sum of squares error function. Wind direction is a periodic variable and a multi-valued function for a given set of inputs; a conventional MLP fails at this task, and so we model the full periodic probability density of direction conditioned on the satellite derived inputs using a Mixture Density Network (MDN) with periodic kernel functions. A committee of the resulting MDNs is shown to improve the results.

DVMS 1.5: A user manual (the data visualization and modeling system)

The data available during the drug discovery process is vast in amount and diverse in nature. To gain useful information from such data, an effective visualisation tool is required. To provide better visualisation facilities to the domain experts (screening scientist, biologist, chemist, etc.),we developed a software which is based on recently developed principled visualisation algorithms such as Generative Topographic Mapping (GTM) and Hierarchical Generative Topographic Mapping (HGTM). The software also supports conventional visualisation techniques such as Principal Component Analysis, NeuroScale, PhiVis, and Locally Linear Embedding (LLE). The software also provides global and local regression facilities . It supports regression algorithms such as Multilayer Perceptron (MLP), Radial Basis Functions network (RBF), Generalised Linear Models (GLM), Mixture of Experts (MoE), and newly developed Guided Mixture of Experts (GME). This user manual gives an overview of the purpose of the software tool, highlights some of the issues to be taken care while creating a new model, and provides information about how to install & use the tool. The user manual does not require the readers to have familiarity with the algorithms it implements. Basic computing skills are enough to operate the software.

Neural network-based wind vector retrieval from satellite scatterometer data

Obtaining wind vectors over the ocean is important for weather forecasting and ocean modelling. Several satellite systems used operationally by meteorological agencies utilise scatterometers to infer wind vectors over the oceans. In this paper we present the results of using novel neural network based techniques to estimate wind vectors from such data. The problem is partitioned into estimating wind speed and wind direction. Wind speed is modelled using a multi-layer perceptron (MLP) and a sum of squares error function. Wind direction is a periodic variable and a multi-valued function for a given set of inputs; a conventional MLP fails at this task, and so we model the full periodic probability density of direction conditioned on the satellite derived inputs using a Mixture Density Network (MDN) with periodic kernel functions. A committee of the resulting MDNs is shown to improve the results.

Short-term electricity demand and gas price forecasts using wavelet transforms and adaptive models

This paper presents some forecasting techniques for energy demand and price prediction, one day ahead. These techniques combine wavelet transform (WT) with fixed and adaptive machine learning/time series models (multi-layer perceptron (MLP), radial basis functions, linear regression, or GARCH). To create an adaptive model, we use an extended Kalman filter or particle filter to update the parameters continuously on the test set. The adaptive GARCH model is a new contribution, broadening the applicability of GARCH methods. We empirically compared two approaches of combining the WT with prediction models: multicomponent forecasts and direct forecasts. These techniques are applied to large sets of real data (both stationary and non-stationary) from the UK energy markets, so as to provide comparative results that are statistically stronger than those previously reported. The results showed that the forecasting accuracy is significantly improved by using the WT and adaptive models. The best models on the electricity demand/gas price forecast are the adaptive MLP/GARCH with the multicomponent forecast; their MSEs are 0.02314 and 0.15384 respectively.

Macrophages and lymphocyte responiveness to mitogen

Purified B-cells fail to proliferate in response to the strong thymus-independent (TI) antigen Lipopolysaccharide (LPS) in the absence of macrophages (Corbel and Melchers, 1983). The fact that macrophages, or factors derived from them are required is supported by the inability of marginal zone B-cells in infants to respond to highly virulent strains of bacteria such as Neisseria meningitidis and Streptococcus pneumoniae (Timens, 1989). This may be due to the lack of CD21 expression on B-cells in infants which could associate with its co-receptor (C3d) on adjacent macrophages. It is not clear whether cell surface contacts and/or soluble products are involved in lymphocyte-macrophage interactions in response to certain antigens. This thesis describes the importance of the macrophage in lymphocyte responses to T-dependent (TD) and TI antigens. The major findings of this thesis were as follows: (1). Macrophages were essential for a full proliferative response to a range of T - and B-cell mitogens and TI-1 and TI-2 antigens, including Concanavalin A, LPS, Pokeweed mitogen (PWM), Dextran sulphate, Phytohaemagglutinin-P (PHA-P) and Poly[I][C]. (2). A ratio of 1 macrophage to 1000 lymphocytes was sufficient for the mitogens to exert their effects. (3). The optimal conditions were established for the activation of an oxidative burst in cells of the monocyte/macrophage lineage as measured by luminometry. The order of ability was OpZ >PMA/lonomycin >f-MLP >Con A >DS >PHA >Poly[I][C] >LPS >PWM. Responses were only substantial and protracted with OpZ and PMA. Peritoneal macrophages were the most responsive cells, whereas splenic and alveolar macrophages were significantly less active and no response could be elicited with Kupffer cells, thus demonstrating heterogeneity between macrophages. (4). Activated macrophages that were then fixed with paraformaldehyde were unable to restore mitogenic responsiveness, even with a ratio of 1 macrophage to 5 lymphocytes. (5). Although highly purified T- and B-cells could respond to mitogen provided live macrophages were present, maximum activation was only observed when all 3 cell types were present. (6). Supernatants from purified macrophage cultures treated with a range of activators were able to partially restore lymphocyte responses to mitogen in macrophage-depleted splenocyte cultures, and purified T - and B-cell cultures. In fact supernatants from macrophages treated with LPS for only 30 minutes could restore responsiveness. Supernatants from OpZ treated macrophages were without effect. (7). Macrophage supernatants could not induce proliferation in the absence of mitogen. They therefore provide a co-mitogenic signal required by lymphocytes in order to respond to mitogen. (8). Macrophage product profiles revealed that LPS and Con A-treated macrophage supernatants showed elevated levels of IL-1β, TNF -α L TB4 and TXB2. These products were therefore good candidates as the co-mitogenic factor. The possible inhibitory factors secreted by OpZ-treated macrophages were PGE2, IL-10 and NO. (9). The removal of cytokines, eicosanoids and TNF-α from LPS-treated macrophage supernatants using Cycloheximide, Dexamethasone and an MMPI respectively, resulted in the inability of these supernatants to restore macrophage-depleted lymphocyte responses to mitogen. (10). rIL-1β and rTNF-α are co-mitogenic factors, as macrophage-depleted lymphocytes incubated with rIL-1β and rTNF-α can respond to mitogen.

AllerTOP v.2 - a server for in silico prediction of allergens

Allergy is an overreaction by the immune system to a previously encountered, ordinarily harmless substance - typically proteins - resulting in skin rash, swelling of mucous membranes, sneezing or wheezing, or other abnormal conditions. The use of modified proteins is increasingly widespread: their presence in food, commercial products, such as washing powder, and medical therapeutics and diagnostics, makes predicting and identifying potential allergens a crucial societal issue. The prediction of allergens has been explored widely using bioinformatics, with many tools being developed in the last decade; many of these are freely available online. Here, we report a set of novel models for allergen prediction utilizing amino acid E-descriptors, auto- and cross-covariance transformation, and several machine learning methods for classification, including logistic regression (LR), decision tree (DT), naïve Bayes (NB), random forest (RF), multilayer perceptron (MLP) and k nearest neighbours (kNN). The best performing method was kNN with 85.3% accuracy at 5-fold cross-validation. The resulting model has been implemented in a revised version of the AllerTOP server (http://www.ddg-pharmfac.net/AllerTOP). © Springer-Verlag 2014.

Influence of QT correction on temporal and amplitude features for human identification via ECG

Identification of humans via ECG is being increasingly studied because it can have several advantages over the traditional biometric identification techniques. However, difficulties arise because of the heartrate variability. In this study we analysed the influence of QT interval correction on the performance of an identification system based on temporal and amplitude features of ECG. In particular we tested MLP, Naive Bayes and 3-NN classifiers on the Fantasia database. Results indicate that QT correction can significantly improve the overall system performance. © 2013 IEEE.

CO2 emissions reduction of Chinese light manufacturing industries:a novel RAM-based global Malmquist-Luenberger productivity index

Climate change has become one of the most challenging issues facing the world. Chinese government has realized the importance of energy conservation and prevention of the climate changes for sustainable development of China's economy and set targets for CO2 emissions reduction in China. In China industry contributes 84.2% of the total CO2 emissions, especially manufacturing industries. Data envelopment analysis (DEA) and Malmquist productivity (MP) index are the widely used mathematical techniques to address the relative efficiency and productivity of a group of homogenous decision making units, e.g. industries or countries. However, in many real applications, especially those related to energy efficiency, there are often undesirable outputs, e.g. the pollutions, waste and CO2 emissions, which are produced inevitably with desirable outputs in the production. This paper introduces a novel Malmquist-Luenberger productivity (MLP) index based on directional distance function (DDF) to address the issue of productivity evolution of DMUs in the presence of undesirable outputs. The new RAM (Range-adjusted measure)-based global MLP index has been applied to evaluate CO2 emissions reduction in Chinese light manufacturing industries. Recommendations for policy makers have been discussed.