Use of contexts in language model interpolation and adaptation

Language models (LMs) are often constructed by building multiple individual component models that are combined using context independent interpolation weights. By tuning these weights, using either perplexity or discriminative approaches, it is possible to adapt LMs to a particular task. This paper investigates the use of context dependent weighting in both interpolation and test-time adaptation of language models. Depending on the previous word contexts, a discrete history weighting function is used to adjust the contribution from each component model. As this dramatically increases the number of parameters to estimate, robust weight estimation schemes are required. Several approaches are described in this paper. The first approach is based on MAP estimation where interpolation weights of lower order contexts are used as smoothing priors. The second approach uses training data to ensure robust estimation of LM interpolation weights. This can also serve as a smoothing prior for MAP adaptation. A normalized perplexity metric is proposed to handle the bias of the standard perplexity criterion to corpus size. A range of schemes to combine weight information obtained from training data and test data hypotheses are also proposed to improve robustness during context dependent LM adaptation. In addition, a minimum Bayes' risk (MBR) based discriminative training scheme is also proposed. An efficient weighted finite state transducer (WFST) decoding algorithm for context dependent interpolation is also presented. The proposed technique was evaluated using a state-of-the-art Mandarin Chinese broadcast speech transcription task. Character error rate (CER) reductions up to 7.3 relative were obtained as well as consistent perplexity improvements. © 2012 Elsevier Ltd. All rights reserved.

Gas properties as a limit to gas turbine performance

Although increasing the turbine inlet temperature has traditionally proved the surest way to increase cycle efficiency, recent work suggests that the performance of future gas turbines may be limited by increased cooling flows and losses. Another limiting scenario concerns the effect on cycle performance of real gas properties at high temperatures. Cycle calculations of uncooled gas turbines show that when gas properties are modelled accurately, the variation of cycle efficiency with turbine inlet temperature at constant pressure ratio exhibits a maximum at temperatures well below the stoichiometric limit. Furthermore, the temperature at the maximum decreases with increasing compressor and turbine polytropic efficiency. This behaviour is examined in the context of a two-component model of the working fluid. The dominant influences come from the change of composition of the combustion products with varying air/fuel ratio (particularly the contribution from the water vapour) together with the temperature variation of the specific heat capacity of air. There are implications for future industrial development programmes, particularly in the context of advanced mixed gas-steam cycles.

Discriminative language model adaptation for Mandarin broadcast speech transcription and translation

This paper investigates unsupervised test-time adaptation of language models (LM) using discriminative methods for a Mandarin broadcast speech transcription and translation task. A standard approach to adapt interpolated language models to is to optimize the component weights by minimizing the perplexity on supervision data. This is a widely made approximation for language modeling in automatic speech recognition (ASR) systems. For speech translation tasks, it is unclear whether a strong correlation still exists between perplexity and various forms of error cost functions in recognition and translation stages. The proposed minimum Bayes risk (MBR) based approach provides a flexible framework for unsupervised LM adaptation. It generalizes to a variety of forms of recognition and translation error metrics. LM adaptation is performed at the audio document level using either the character error rate (CER), or translation edit rate (TER) as the cost function. An efficient parameter estimation scheme using the extended Baum-Welch (EBW) algorithm is proposed. Experimental results on a state-of-the-art speech recognition and translation system are presented. The MBR adapted language models gave the best recognition and translation performance and reduced the TER score by up to 0.54% absolute. © 2007 IEEE.

Model-based control of a Rijke tube combustion instability

A Rijke tube is used to demonstrate model-based control of a combustion instability, where controller design is based on measurement of the unstable system. The Rijke tube used was of length 0.75m and had a grid-stabilised laminar flame in its lower half. A microphone was used as a sensor and a loudspeaker as an actuator for active control. The open loop transfer function (OLTF) required for controller design was that from the actuator to the sensor. This was measured experimentally by sending a signal with two components to the actuator. The first was a control component from an empirically designed controller, which was used to stabilise the system, thus eliminating the non-linear limit cycle. The second was a high bandwidth signal for identification of the OLTF. This approach to measuring the OLTF is generic and can be applied to large-scale combustors. The measured OLTF showed that only the fundamental mode of the tube was unstable; this was consistent with the OLTF predicted by a mathematical model of the tube, involving 1-D linear acoustic waves and a time delay heat release model. Based on the measured OLTF, a controller to stabilise the instability was designed using Nyquist techniques. This was implemented and was seen to result in an 80dB reduction in the microphone pressure spectrum. A robustness study was performed by adding an additional length to the top of the Rijke tobe. The controller was found to achieve control up to an increase in tube length of 19%. This compared favourably with the empirical controller, which lost control for an increase in tube length of less than 3%.

Breakdown voltage for superjunction power devices with charge imbalance: An analytical model valid for both punch through and non punch through devices

An analytical model for the electric field and the breakdown voltage (BV) of an unbalanced superjunction (SJ) device is presented in this paper. The analytical technique uses a superposition approach treating the asymmetric charge in the pillars as an excess charge component superimposed on a balanced charge component. The proposed double-exponentialmodel is able to accurately predict the electric field and the BV for unbalanced SJ devices in both punch through and non punch through conditions. The model is also reasonably accurate at extremely high levels of charge imbalance when the devices behave similarly to a PiN diode or to a high-conductance layer. The analytical model is compared against numerical simulations of charge unbalanced SJ devices and against experimental results. © 2009 IEEE.

Fault-tolerant model predictive control of a wind turbine benchmark

This paper aims to solve the fault tolerant control problem of a wind turbine benchmark. A hierarchical controller with model predictive pre-compensators, a global model predictive controller and a supervisory controller is proposed. In the model predictive pre-compensator, an extended Kalman Filter is designed to estimate the system states and various fault parameters. Based on the estimation, a group of model predictive controllers are designed to compensate the fault effects for each component of the wind turbine. The global MPC is used to schedule the operation of the components and exploit potential system-level redundancies. Extensive simulations of various fault conditions show that the proposed controller has small transients when faults occur and uses smoother and smaller generator torque and pitch angle inputs than the default controller. This paper shows that MPC can be a good candidate for fault tolerant controllers, especially the one with an adaptive internal model combined with a parameter estimation and update mechanism, such as an extended Kalman Filter. © 2012 IFAC.

Biomimetic calcium carbonate-gelatin composites as a model system for eggshell mineralization

The composite nature of mineralized natural materials is achieved through both the microstructural inclusion of an organic component and an overall microstructure that is controlled by templating onto organic macromolecules. A modification of an existing laboratory technique is developed for the codeposition of a CaCO3-gelatin composite with a controllable organic content. First, calibration curves are developed to determine the organic content of a CaCO3-gelatin composite from infrared spectra. Second, a CaCO3-gelatin composite is deposited on either glass coverslips or demineralized eggshell membranes using an automated alternating soaking process. Electron microscopy images and use of the infrared spectra calibration curves show that by altering the amount of gelatin in the ionic growth solutions, the final organic component of the mineral can be regulated over the range of 1-10%, similar to that of natural eggshell. © 2012 Materials Research Societ.

State estimation schemes for independent component coupled hidden Markov models

Conventional Hidden Markov models generally consist of a Markov chain observed through a linear map corrupted by additive noise. This general class of model has enjoyed a huge and diverse range of applications, for example, speech processing, biomedical signal processing and more recently quantitative finance. However, a lesser known extension of this general class of model is the so-called Factorial Hidden Markov Model (FHMM). FHMMs also have diverse applications, notably in machine learning, artificial intelligence and speech recognition [13, 17]. FHMMs extend the usual class of HMMs, by supposing the partially observed state process is a finite collection of distinct Markov chains, either statistically independent or dependent. There is also considerable current activity in applying collections of partially observed Markov chains to complex action recognition problems, see, for example, [6]. In this article we consider the Maximum Likelihood (ML) parameter estimation problem for FHMMs. Much of the extant literature concerning this problem presents parameter estimation schemes based on full data log-likelihood EM algorithms. This approach can be slow to converge and often imposes heavy demands on computer memory. The latter point is particularly relevant for the class of FHMMs where state space dimensions are relatively large. The contribution in this article is to develop new recursive formulae for a filter-based EM algorithm that can be implemented online. Our new formulae are equivalent ML estimators, however, these formulae are purely recursive and so, significantly reduce numerical complexity and memory requirements. A computer simulation is included to demonstrate the performance of our results. © Taylor & Francis Group, LLC.

Active learning of model evidence using Bayesian quadrature

Numerical integration is a key component of many problems in scientific computing, statistical modelling, and machine learning. Bayesian Quadrature is a modelbased method for numerical integration which, relative to standard Monte Carlo methods, offers increased sample efficiency and a more robust estimate of the uncertainty in the estimated integral. We propose a novel Bayesian Quadrature approach for numerical integration when the integrand is non-negative, such as the case of computing the marginal likelihood, predictive distribution, or normalising constant of a probabilistic model. Our approach approximately marginalises the quadrature model's hyperparameters in closed form, and introduces an active learning scheme to optimally select function evaluations, as opposed to using Monte Carlo samples. We demonstrate our method on both a number of synthetic benchmarks and a real scientific problem from astronomy.

Coupling model uncertainty for coupled rainfall/runoff and surface water quality models in river problems

Coupled hydrology and water quality models are an important tool today, used in the understanding and management of surface water and watershed areas. Such problems are generally subject to substantial uncertainty in parameters, process understanding, and data. Component models, drawing on different data, concepts, and structures, are affected differently by each of these uncertain elements. This paper proposes a framework wherein the response of component models to their respective uncertain elements can be quantified and assessed, using a hydrological model and water quality model as two exemplars. The resulting assessments can be used to identify model coupling strategies that permit more appropriate use and calibration of individual models, and a better overall coupled model response. One key finding was that an approximate balance of water quality and hydrological model responses can be obtained using both the QUAL2E and Mike11 water quality models. The balance point, however, does not support a particularly narrow surface response (or stringent calibration criteria) with respect to the water quality calibration data, at least in the case examined here. Additionally, it is clear from the results presented that the structural source of uncertainty is at least as significant as parameter-based uncertainties in areal models. © 2012 John Wiley & Sons, Ltd.

How neurons migrate: a dynamic in-silico model of neuronal migration in the developing cortex.

BACKGROUND: Neuronal migration, the process by which neurons migrate from their place of origin to their final position in the brain, is a central process for normal brain development and function. Advances in experimental techniques have revealed much about many of the molecular components involved in this process. Notwithstanding these advances, how the molecular machinery works together to govern the migration process has yet to be fully understood. Here we present a computational model of neuronal migration, in which four key molecular entities, Lis1, DCX, Reelin and GABA, form a molecular program that mediates the migration process. RESULTS: The model simulated the dynamic migration process, consistent with in-vivo observations of morphological, cellular and population-level phenomena. Specifically, the model reproduced migration phases, cellular dynamics and population distributions that concur with experimental observations in normal neuronal development. We tested the model under reduced activity of Lis1 and DCX and found an aberrant development similar to observations in Lis1 and DCX silencing expression experiments. Analysis of the model gave rise to unforeseen insights that could guide future experimental study. Specifically: (1) the model revealed the possibility that under conditions of Lis1 reduced expression, neurons experience an oscillatory neuron-glial association prior to the multipolar stage; and (2) we hypothesized that observed morphology variations in rats and mice may be explained by a single difference in the way that Lis1 and DCX stimulate bipolar motility. From this we make the following predictions: (1) under reduced Lis1 and enhanced DCX expression, we predict a reduced bipolar migration in rats, and (2) under enhanced DCX expression in mice we predict a normal or a higher bipolar migration. CONCLUSIONS: We present here a system-wide computational model of neuronal migration that integrates theory and data within a precise, testable framework. Our model accounts for a range of observable behaviors and affords a computational framework to study aspects of neuronal migration as a complex process that is driven by a relatively simple molecular program. Analysis of the model generated new hypotheses and yet unobserved phenomena that may guide future experimental studies. This paper thus reports a first step toward a comprehensive in-silico model of neuronal migration.

Discrete multicomponent model for biodiesel spray combustion simulation

An important first step in spray combustion simulation is an accurate determination of the fuel properties which affects the modelling of spray formation and reaction. In a practical combustion simulation, the implementation of a multicomponent model is important in capturing the relative volatility of different fuel components. A Discrete Multicomponent (DM) model is deemed to be an appropriate candidate to model a composite fuel like biodiesel which consists of four components of fatty acid methyl esters (FAME). In this paper, the DM model is compared with the traditional Continuous Thermodynamics (CTM) model for both diesel and biodiesel. The CTM model is formulated based on mixing rules that incorporate the physical and thermophysical properties of pure components into a single continuous surrogate for the composite fuel. The models are implemented within the open-source CFD code OpenFOAM, and a semi-quantitative comparison is made between the predicted spray-combustion characteristics and optical measurements of a swirl-stabilised flame of diesel and biodiesel. The DM model performs better than the CTM model in predicting a higher magnitude of heat release rate in the top flame brush region of the biodiesel flame compared to that of the diesel flame. Using both the DM and CTM models, the simulation successfully reproduces the droplet size, volume flux, and droplet density profiles of diesel and biodiesel. The DM model predicts a longer spray penetration length for biodiesel compared to that of diesel, as seen in the experimental data. Also, the DM model reproduces a segregated biodiesel fuel vapour field and spray in which the most abundant FAME component has the longest vapour penetration. In the biodiesel flame, the relative abundance of each fuel component is found to dominate over the relative volatility in terms of the vapour species distribution and vice versa in the liquid species distribution. © 2014 Elsevier Ltd. All rights reserved.

MoT - Mixture of trees probabilistic graphical model for video segmentation

We present a novel mixture of trees (MoT) graphical model for video segmentation. Each component in this mixture represents a tree structured temporal linkage between super-pixels from the first to the last frame of a video sequence. Our time-series model explicitly captures the uncertainty in temporal linkage between adjacent frames which improves segmentation accuracy. We provide a variational inference scheme for this model to estimate super-pixel labels and their confidences in nearly realtime. The efficacy of our approach is demonstrated via quantitative comparisons on the challenging SegTrack joint segmentation and tracking dataset [23].