Computational capabilities of multilayer committee machines

We obtained an analytical expression for the computational complexity of many layered committee machines with a finite number of hidden layers (L < 8) using the generalization complexity measure introduced by Franco et al (2006) IEEE Trans. Neural Netw. 17 578. Although our result is valid in the large-size limit and for an overlap synaptic matrix that is ultrametric, it provides a useful tool for inferring the appropriate architecture a network must have to reproduce an arbitrary realizable Boolean function.

Learning in ultrametric committee machines

The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

Storage capacity of ultrametric committee machines

The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K. © 2014 IOP Publishing Ltd.

Analysing time series structure with hidden Markov models

This paper consides the problem of extracting the relationships between two time series in a non-linear non-stationary environment with Hidden Markov Models (HMMs). We describe an algorithm which is capable of identifying associations between variables. The method is applied both to synthetic data and real data. We show that HMMs are capable of modelling the oil drilling process and that they outperform existing methods.

Time delay estimation with hidden Markov models

Most traditional methods for extracting the relationships between two time series are based on cross-correlation. In a non-linear non-stationary environment, these techniques are not sufficient. We show in this paper how to use hidden Markov models to identify the lag (or delay) between different variables for such data. Adopting an information-theoretic approach, we develop a procedure for training HMMs to maximise the mutual information (MMI) between delayed time series. The method is used to model the oil drilling process. We show that cross-correlation gives no information and that the MMI approach outperforms maximum likelihood.

Bayesian online algorithms for learning in discrete Hidden Markov Models

We propose and analyze two different Bayesian online algorithms for learning in discrete Hidden Markov Models and compare their performance with the already known Baldi-Chauvin Algorithm. Using the Kullback-Leibler divergence as a measure of generalization we draw learning curves in simplified situations for these algorithms and compare their performances.

Entrepreneurship and the informal economy: a study of Ukraine's hidden enterprise culture

How many entrepreneurs start-up their business ventures conducting some or all of their trade in the informal economy? The aim of this paper is to answer this key question that has been seldom addressed using data from 600 face-to-face structured interviews conducted in Ukraine in late 2005 and early 2006. Analyzing the 331 entrepreneurs identified (i.e., individuals starting-up an enterprise in the past three years), just 10 percent operate on a wholly legitimate basis, while 39 percent have a license to trade and/or have registered their business but conduct a portion of their trade in the informal economy, and 51 percent operate unregistered enterprises and conduct all of their trade on an off-the-books basis. Given that some 90 percent of all business start-ups operate partially or wholly in the informal economy, and that 40 percent of all respondents depend on the informal economy as either their principal or secondary contributor to their livelihoods, the paper concludes by considering the wider implications of these findings both for further research and public policy.

Transition in inclined internally heated fluid layers

Non-linear solutions and studies of their stability are presented for flows in a homogeneously heated fluid layer under the influence of a constant pressure gradient or when the mass flux across any lateral cross-section of the channel is required to vanish. The critical Grashof number is determined by a linear stability analysis of the basic state which depends only on the z-coordinate perpendicular to the boundary. Bifurcating longitudinal rolls as well as secondary solutions depending on the streamwise x-coordinate are investigated and their amplitudes are determined as functions of the supercritical Grashof number for various Prandtl numbers and angles of inclination of the layer. Solutions that emerge from a Hopf bifurcation assume the form of propagating waves and can thus be considered as steady flows relative to an appropriately moving frame of reference. The stability of these solutions with respect to three-dimensional disturbances is also analyzed in order to identify possible bifurcation points for evolving tertiary flows.

Pattern competition in homogeneously heated fluid layers

Simulations examining pattern competition have been performed on a horizontal homogeneously heated layer that is bounded by an isothermal plane above an adiabatic plane. Several different circulation patterns arose as the heating regime applied to the horizontal layer was modified. The sequence of the patterns formed as the Grashof number was increased had the following order: laminar, z-axis rolls, squares, hexagons and pentagons, pentagons and then two different square modes of differing orientations. Fourier analysis was used to determine how the key modes interact in the presence of different patterns.

Hidden state models for time series

Amongst all the objectives in the study of time series, uncovering the dynamic law of its generation is probably the most important. When the underlying dynamics are not available, time series modelling consists of developing a model which best explains a sequence of observations. In this thesis, we consider hidden space models for analysing and describing time series. We first provide an introduction to the principal concepts of hidden state models and draw an analogy between hidden Markov models and state space models. Central ideas such as hidden state inference or parameter estimation are reviewed in detail. A key part of multivariate time series analysis is identifying the delay between different variables. We present a novel approach for time delay estimating in a non-stationary environment. The technique makes use of hidden Markov models and we demonstrate its application for estimating a crucial parameter in the oil industry. We then focus on hybrid models that we call dynamical local models. These models combine and generalise hidden Markov models and state space models. Probabilistic inference is unfortunately computationally intractable and we show how to make use of variational techniques for approximating the posterior distribution over the hidden state variables. Experimental simulations on synthetic and real-world data demonstrate the application of dynamical local models for segmenting a time series into regimes and providing predictive distributions.

Nano-indentation measurement of oxide layers formed in LBE on F/M steels

Ferritic/martensitic (F/M) steels (T91, HT-9, EP 823) are candidate materials for future liquid lead or lead bismuth eutectic (LBE) cooled nuclear reactors. To understand the corrosion of these materials in LBE, samples of each material were exposed at 535 °C for 600 h and 200 h at an oxygen content of 10 wt%. After the corrosion tests, the samples were analyzed using SEM, WDX and nano-indentation in cross section. Multi-layered oxide scales were found on the sample surfaces. The compositions of these oxide layers are not entirely in agreement with the literature. The nano-indentation results showed that the E-modulus and hardness of the oxide layers are significantly lower than the values for dense bulk oxide materials. It is assumed that the low values stem from high porosity in the oxide layers. Comparison with in-air oxidized steels show that the E-modulus decreases with increasing oxide layer thickness. © 2008 Elsevier B.V. All rights reserved.

Simultaneous en-face imaging of two layers in human retina by low-coherence reflectometry

We show that with a fiberized multiple Michelson-interferometer-type configuration, transverse images from several layers in the human eye can be simultaneously obtained. We demonstrate the principle by producing simultaneous 100×100 pixel en-face images of a 4 mm×4 mm region on a postmortem retina for two depth positions 250 µm apart.

Characterization of oxide layers grown on D9 austenitic stainless steel in lead bismuth eutectic

Lead bismuth eutectic (LBE) is a possible coolant for fast reactors and targets in spallation neutron sources. Its low melting point, high evaporation point, good thermal conductivity, low reactivity, and good neutron yield make it a safe and high performance coolant in radiation environments. The disadvantage is that it is a corrosive medium for most steels and container materials. This study was performed to evaluate the corrosion behavior of the austenitic stainless steel D9 in oxygen controlled LBE. In order to predict the corrosion behavior of steel in this environment detailed analyses have to be performed on the oxide layers formed on these materials and various other relevant materials upon exposure to LBE. In this study the corrosion/oxidation of D9 stainless steel in LBE was investigated in great detail. The oxide layers formed were characterized using atomic force microscopy, magnetic force microscopy, nanoindentation, and scanning electron microscopy with wavelength-dispersive spectroscopy (WDS) to understand the corrosion and oxidation mechanisms of D9 stainless steel in contact with the LBE. What was previously believed to be a simple double oxide layer was identified here to consist of at least 4 different oxide layers. It was found that the inner most oxide layer takes over the grain structure of what used to be the bulk steel material while the outer oxide layer consists of freshly grown oxides with a columnar structure. These results lead to a descriptive model of how these oxide layers grow on this steel under the harsh environments encountered in these applications.

Biomedical events extraction using the hidden vector state model

Objective: Biomedical events extraction concerns about events describing changes on the state of bio-molecules from literature. Comparing to the protein-protein interactions (PPIs) extraction task which often only involves the extraction of binary relations between two proteins, biomedical events extraction is much harder since it needs to deal with complex events consisting of embedded or hierarchical relations among proteins, events, and their textual triggers. In this paper, we propose an information extraction system based on the hidden vector state (HVS) model, called HVS-BioEvent, for biomedical events extraction, and investigate its capability in extracting complex events. Methods and material: HVS has been previously employed for extracting PPIs. In HVS-BioEvent, we propose an automated way to generate abstract annotations for HVS training and further propose novel machine learning approaches for event trigger words identification, and for biomedical events extraction from the HVS parse results. Results: Our proposed system achieves an F-score of 49.57% on the corpus used in the BioNLP'09 shared task, which is only 2.38% lower than the best performing system by UTurku in the BioNLP'09 shared task. Nevertheless, HVS-BioEvent outperforms UTurku's system on complex events extraction with 36.57% vs. 30.52% being achieved for extracting regulation events, and 40.61% vs. 38.99% for negative regulation events. Conclusions: The results suggest that the HVS model with the hierarchical hidden state structure is indeed more suitable for complex event extraction since it could naturally model embedded structural context in sentences.

Extracting protein-protein interactions from MEDLINE using the hidden vector state model

A major challenge in text mining for biomedicine is automatically extracting protein-protein interactions from the vast amount of biomedical literature. We have constructed an information extraction system based on the Hidden Vector State (HVS) model for protein-protein interactions. The HVS model can be trained using only lightly annotated data whilst simultaneously retaining sufficient ability to capture the hierarchical structure. When applied in extracting protein-protein interactions, we found that it performed better than other established statistical methods and achieved 61.5% in F-score with balanced recall and precision values. Moreover, the statistical nature of the pure data-driven HVS model makes it intrinsically robust and it can be easily adapted to other domains.