Using generative models for handwritten digit recognition

We describe a method of recognizing handwritten digits by fitting generative models that are built from deformable B-splines with Gaussian ``ink generators'' spaced along the length of the spline. The splines are adjusted using a novel elastic matching procedure based on the Expectation Maximization (EM) algorithm that maximizes the likelihood of the model generating the data. This approach has many advantages. (1) After identifying the model most likely to have generated the data, the system not only produces a classification of the digit but also a rich description of the instantiation parameters which can yield information such as the writing style. (2) During the process of explaining the image, generative models can perform recognition driven segmentation. (3) The method involves a relatively small number of parameters and hence training is relatively easy and fast. (4) Unlike many other recognition schemes it does not rely on some form of pre-normalization of input images, but can handle arbitrary scalings, translations and a limited degree of image rotation. We have demonstrated our method of fitting models to images does not get trapped in poor local minima. The main disadvantage of the method is it requires much more computation than more standard OCR techniques.

A hierarchical latent variable model for data visualization

Visualization has proven to be a powerful and widely-applicable tool the analysis and interpretation of data. Most visualization algorithms aim to find a projection from the data space down to a two-dimensional visualization space. However, for complex data sets living in a high-dimensional space it is unlikely that a single two-dimensional projection can reveal all of the interesting structure. We therefore introduce a hierarchical visualization algorithm which allows the complete data set to be visualized at the top level, with clusters and sub-clusters of data points visualized at deeper levels. The algorithm is based on a hierarchical mixture of latent variable models, whose parameters are estimated using the expectation-maximization algorithm. We demonstrate the principle of the approach first on a toy data set, and then apply the algorithm to the visualization of a synthetic data set in 12 dimensions obtained from a simulation of multi-phase flows in oil pipelines and to data in 36 dimensions derived from satellite images.

GTM: A principled alternative to the self-organizing map

The Self-Organizing Map (SOM) algorithm has been extensively studied and has been applied with considerable success to a wide variety of problems. However, the algorithm is derived from heuristic ideas and this leads to a number of significant limitations. In this paper, we consider the problem of modelling the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. We introduce a novel form of latent variable model, which we call the GTM algorithm (for Generative Topographic Mapping), which allows general non-linear transformations from latent space to data space, and which is trained using the EM (expectation-maximization) algorithm. Our approach overcomes the limitations of the SOM, while introducing no significant disadvantages. We demonstrate the performance of the GTM algorithm on simulated data from flow diagnostics for a multi-phase oil pipeline.

EM algorithm for GTM-FS

We propose a generative topographic mapping (GTM) based data visualization with simultaneous feature selection (GTM-FS) approach which not only provides a better visualization by modeling irrelevant features ("noise") using a separate shared distribution but also gives a saliency value for each feature which helps the user to assess their significance. This technical report presents a varient of the Expectation-Maximization (EM) algorithm for GTM-FS.

Quantifying simulator discrepancy in discrete-time dynamical simulators

When making predictions with complex simulators it can be important to quantify the various sources of uncertainty. Errors in the structural specification of the simulator, for example due to missing processes or incorrect mathematical specification, can be a major source of uncertainty, but are often ignored. We introduce a methodology for inferring the discrepancy between the simulator and the system in discrete-time dynamical simulators. We assume a structural form for the discrepancy function, and show how to infer the maximum-likelihood parameter estimates using a particle filter embedded within a Monte Carlo expectation maximization (MCEM) algorithm. We illustrate the method on a conceptual rainfall-runoff simulator (logSPM) used to model the Abercrombie catchment in Australia. We assess the simulator and discrepancy model on the basis of their predictive performance using proper scoring rules. This article has supplementary material online. © 2011 International Biometric Society.

Novel visualization methods for protein data

Visualization of high-dimensional data has always been a challenging task. Here we discuss and propose variants of non-linear data projection methods (Generative Topographic Mapping (GTM) and GTM with simultaneous feature saliency (GTM-FS)) that are adapted to be effective on very high-dimensional data. The adaptations use log space values at certain steps of the Expectation Maximization (EM) algorithm and during the visualization process. We have tested the proposed algorithms by visualizing electrostatic potential data for Major Histocompatibility Complex (MHC) class-I proteins. The experiments show that the variation in the original version of GTM and GTM-FS worked successfully with data of more than 2000 dimensions and we compare the results with other linear/nonlinear projection methods: Principal Component Analysis (PCA), Neuroscale (NSC) and Gaussian Process Latent Variable Model (GPLVM).

Discovering the preferences of physicians with regards to rank-ordered medical documents

The practice of evidence-based medicine involves consulting documents from repositories such as Scopus, PubMed, or the Cochrane Library. The most common approach for presenting retrieved documents is in the form of a list, with the assumption that the higher a document is on a list, the more relevant it is. Despite this list-based presentation, it is seldom studied how physicians perceive the importance of the order of documents presented in a list. This paper describes an empirical study that elicited and modeled physicians' preferences with regard to list-based results. Preferences were analyzed using a GRIP method that relies on pairwise comparisons of selected subsets of possible rank-ordered lists composed of 3 documents. The results allow us to draw conclusions regarding physicians' attitudes towards the importance of having documents ranked correctly on a result list, versus the importance of retrieving relevant but misplaced documents. Our findings should help developers of clinical information retrieval applications when deciding how retrieved documents should be presented and how performance of the application should be assessed. © 2012 Springer-Verlag Berlin Heidelberg.

Statistical mechanics of mutual information maximization

An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.

Capital markets and the market structure of foreign investments

Contrary to the long-received theory of FDI, interest rates or rates of return can motivate foreign direct investment (FDI) in concert with the benefits of direct ownership. Thus, access to investor capital and capital markets is a vital component of the multinational’s competitive market structure. Moreover, multinationals can use their superior financial capacity as a competitive advantage in exploiting FDI opportunities in dynamic markets. They can also mitigate higher levels of foreign business risks under dynamic conditions by shifting more financial risk to creditors in the host economy. Furthermore, the investor’s expectation of foreign business risk necessarily commands a risk premium for exposing their equity to foreign market risk. Multinationals can modify the profit maximization strategy of their foreign subsidiaries to maximize growth or profits to generate this risk premium. In this context, we investigate how foreign subsidiaries manage their capital funding, business risk, and profit strategies with a diverse sample of 8,000 matched parents and foreign subsidiary accounts from multiple industries in 38 countries.We find that interest rates, asset prices, and expectations in capital markets have a significant effect on the capital movements of foreign subsidiaries. We also find that foreign subsidiaries mitigate their exposure to foreign business risk by modifying their capital structure and debt maturity. Further, we show how the operating strategy of foreign subsidiaries affects their preference for growth or profit maximization. We further show that superior shareholder value, which is a vital link for access to capital for funding foreign expansion in open market economies, is achieved through maintaining stability in the rate of growth and good asset utilization.

Ordered mixing of drugs with particulate excipients

A novel direct compression tableting excipient has been made by recrystallisation of lactose. The particles produced had high porosity, high specific surface area and high surface roughness. The resistance to segregation of ordered mixes formed between a model drug; potassium chloride and the excipients recrystallised lactose, spray crystallised maltose-dextrose (Emdexl and a direct compacting sugar (Dipac) was studied using a vibrational segregation model. The highly porous excipients, Emdex and recrystallised lactose formed ordered mixes which did not segregate even at high accelerations and low frequencies whereas the relatively smooth excipient, Dipac, displayed marked segregation in most vibration conditions. The vibrations were related to practical conditions measured in pharmaceutical process machinery. The time required to form an ordered mix was inversely related to the stability of the mix when subjected to vibration. An ultracentrifuge technique was developed to determine the interparticle adhesion forces holding drug and excipient particles together as ordered units. Excipient powders such as Emdex and recrystallised lactose, which formed non-segregating ordered mixes, had high interparticle adhesion forces. Other ordered mixes that segregated when subjected to different vibration conditions were found to have large quantities of weekly-bound drug particles; such mixes included those with Dipac as the carrier excipient as well as those containing a high concentration of drug. The electrostatic properties of different drug and excipient powders were studied using a Faraday well and an electrometer. Excipient powders such as Emdex and recrystallised lactose which formed stable ordered mixes also had a widely different surface charge in comparison with drug particles, whereas Dipac had a similar surface charge to the drug particles and formed unstable ordered mixes. A specially constructed triboelectric charging apparatus based on an air cyclone was developed to increase the affinity of drug particles for different excipient particles. Using triboelectrification to increase the interparticle adhesion forces, the segregation tendencies of unstable ordered mixes were greatly reduced. The stability of ordered mixes is shown to be related to both the surface physical characteristics and the surface electrical properties of the constituent carrier (excipientl particles.

SAS/OWA:ordered weighted averaging in SAS optimization

This paper explores the use of the optimization procedures in SAS/OR software with application to the ordered weight averaging (OWA) operators of decision-making units (DMUs). OWA was originally introduced by Yager (IEEE Trans Syst Man Cybern 18(1):183-190, 1988) has gained much interest among researchers, hence many applications such as in the areas of decision making, expert systems, data mining, approximate reasoning, fuzzy system and control have been proposed. On the other hand, the SAS is powerful software and it is capable of running various optimization tools such as linear and non-linear programming with all type of constraints. To facilitate the use of OWA operator by SAS users, a code was implemented. The SAS macro developed in this paper selects the criteria and alternatives from a SAS dataset and calculates a set of OWA weights. An example is given to illustrate the features of SAS/OWA software. © Springer-Verlag 2009.

Network lifetime maximization with cross-layer design in wireless sensor networks

This paper investigates a cross-layer design approach for minimizing energy consumption and maximizing network lifetime (NL) of a multiple-source and single-sink (MSSS) WSN with energy constraints. The optimization problem for MSSS WSN can be formulated as a mixed integer convex optimization problem with the adoption of time division multiple access (TDMA) in medium access control (MAC) layer, and it becomes a convex problem by relaxing the integer constraint on time slots. Impacts of data rate, link access and routing are jointly taken into account in the optimization problem formulation. Both linear and planar network topologies are considered for NL maximization (NLM). With linear MSSS and planar single-source and single-sink (SSSS) topologies, we successfully use Karush-Kuhn-Tucker (KKT) optimality conditions to derive analytical expressions of the optimal NL when all nodes are exhausted simultaneously. The problem for planar MSSS topology is more complicated, and a decomposition and combination (D&C) approach is proposed to compute suboptimal solutions. An analytical expression of the suboptimal NL is derived for a small scale planar network. To deal with larger scale planar network, an iterative algorithm is proposed for the D&C approach. Numerical results show that the upper-bounds of the network lifetime obtained by our proposed optimization models are tight. Important insights into the NL and benefits of cross-layer design for WSN NLM are obtained.

Gaussian process quantile regression using expectation propagation

Direct quantile regression involves estimating a given quantile of a response variable as a function of input variables. We present a new framework for direct quantile regression where a Gaussian process model is learned, minimising the expected tilted loss function. The integration required in learning is not analytically tractable so to speed up the learning we employ the Expectation Propagation algorithm. We describe how this work relates to other quantile regression methods and apply the method on both synthetic and real data sets. The method is shown to be competitive with state of the art methods whilst allowing for the leverage of the full Gaussian process probabilistic framework.

Matrix-assisted diffusion-ordered spectroscopy:application of surfactant solutions to the resolution of isomer spectra

The component spectra of a mixture of isomers with nearly identical diffusion coefficients cannot normally be distinguished in a standard diffusion-ordered spectroscopy (DOSY) experiment but can often be easily resolved using matrix-assisted DOSY, in which diffusion behaviour is manipulated by the addition of a co-solute such as a surfactant. Relatively little is currently known about the conditions required for such a separation, for example, how the choice between normal and reverse micelles affects separation or how the isomer structures themselves affect the resolution. The aim of this study was to explore the application of sodium dodecyl sulfate (SDS) normal micelles in aqueous solution and sodium 1,4-bis(2-ethylhexyl)sulfosuccinate (AOT) aggregates in chloroform, at a range of concentrations, to the diffusion resolution of some simple model sets of isomers such as monomethoxyphenols and short chain alcohols. It is shown that SDS micelles offer better resolution where these isomers differ in the position of a hydroxyl group, whereas AOT aggregates are more effective for isomers differing in the position of a methyl group. For both the normal SDS micelles and the less well-defined AOT aggregates, differences in the resolution of the isomers can in part be rationalised in terms of differing degrees of hydrophobicity, amphiphilicity and steric effects. Copyright © 2012 John Wiley & Sons, Ltd.

Matrix-assisted diffusion-ordered spectroscopy:mixture resolution by NMR using SDSmicelles

Diffusion-ordered spectroscopy (DOSY) is a powerful technique for mixture analysis, but in its basic form it cannot separate the component spectra for species with very similar diffusion coefficients. It has been recently demonstrated that the component spectra of a mixture of isomers with nearly identical diffusion coefficients (the three dihydroxybenzenes) can be resolved using matrix-assisted DOSY (MAD), in which diffusion is perturbed by the addition of a co-solute such as a surfactant [R. Evans, S. Haiber, M. Nilsson, G. A. Morris, Anal. Chem. 2009, 81, 4548-4550]. However, little is known about the conditions required for such a separation, for example, the concentrations and concentration ratios of surfactant and solutes. The aim of this study was to explore the concentration range over whichmatrix-assisted DOSY using the surfactant SDS can achieve diffusion resolution of a simple model set of isomers, the monomethoxyphenols. The results show that the separation is remarkably robust with respect to both the concentrations and the concentration ratios of surfactant and solutes, supporting the idea that MAD may become a valuable tool formixture analysis. © 2010 John Wiley & Sons, Ltd.