Use of contexts in language model interpolation and adaptation

Language models (LMs) are often constructed by building multiple individual component models that are combined using context independent interpolation weights. By tuning these weights, using either perplexity or discriminative approaches, it is possible to adapt LMs to a particular task. This paper investigates the use of context dependent weighting in both interpolation and test-time adaptation of language models. Depending on the previous word contexts, a discrete history weighting function is used to adjust the contribution from each component model. As this dramatically increases the number of parameters to estimate, robust weight estimation schemes are required. Several approaches are described in this paper. The first approach is based on MAP estimation where interpolation weights of lower order contexts are used as smoothing priors. The second approach uses training data to ensure robust estimation of LM interpolation weights. This can also serve as a smoothing prior for MAP adaptation. A normalized perplexity metric is proposed to handle the bias of the standard perplexity criterion to corpus size. A range of schemes to combine weight information obtained from training data and test data hypotheses are also proposed to improve robustness during context dependent LM adaptation. In addition, a minimum Bayes' risk (MBR) based discriminative training scheme is also proposed. An efficient weighted finite state transducer (WFST) decoding algorithm for context dependent interpolation is also presented. The proposed technique was evaluated using a state-of-the-art Mandarin Chinese broadcast speech transcription task. Character error rate (CER) reductions up to 7.3 relative were obtained as well as consistent perplexity improvements. © 2012 Elsevier Ltd. All rights reserved.

Generalization in Adaptation to Stable and Unstable Dynamics

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization. © 2012 Kadiallah et al.

Language model cross adaptation for LVCSR system combination

State-of-the-art large vocabulary continuous speech recognition (LVCSR) systems often combine outputs from multiple sub-systems that may even be developed at different sites. Cross system adaptation, in which model adaptation is performed using the outputs from another sub-system, can be used as an alternative to hypothesis level combination schemes such as ROVER. Normally cross adaptation is only performed on the acoustic models. However, there are many other levels in LVCSR systems' modelling hierarchy where complimentary features may be exploited, for example, the sub-word and the word level, to further improve cross adaptation based system combination. It is thus interesting to also cross adapt language models (LMs) to capture these additional useful features. In this paper cross adaptation is applied to three forms of language models, a multi-level LM that models both syllable and word sequences, a word level neural network LM, and the linear combination of the two. Significant error rate reductions of 4.0-7.1% relative were obtained over ROVER and acoustic model only cross adaptation when combining a range of Chinese LVCSR sub-systems used in the 2010 and 2011 DARPA GALE evaluations. © 2012 Elsevier Ltd. All rights reserved.

Tandem system adaptation using multiple linear feature transforms

Adaptation to speaker and environment changes is an essential part of current automatic speech recognition (ASR) systems. In recent years the use of multi-layer percpetrons (MLPs) has become increasingly common in ASR systems. A standard approach to handling speaker differences when using MLPs is to apply a global speaker-specific constrained MLLR (CMLLR) transform to the features prior to training or using the MLP. This paper considers the situation when there are both speaker and channel, communication link, differences in the data. A more powerful transform, front-end CMLLR (FE-CMLLR), is applied to the inputs to the MLP to represent the channel differences. Though global, these FE-CMLLR transforms vary from time-instance to time-instance. Experiments on a channel distorted dialect Arabic conversational speech recognition task indicates the usefulness of adapting MLP features using both CMLLR and FE-CMLLR transforms. © 2013 IEEE.

Effect of a feeding stimulant on feeding adaptation of gibel carp Carassius auratus gibelio (Bloch), fed diets with replacement of fish meal by meat and bone meal

The objectives of the study were to investigate the effect of a feeding stimulant on feeding adaptation of gibel carp (Carassius auratus gibelio Bloch) fed diets with replacement of fish meal by meat and bone meal (MBM), and whether or not the juvenile gibel carp could adapt to higher MBM level in the diet. Juvenile and adult gibel carp were tested. Two and one replacement levels were used for juvenile and adult fish respectively. Each group of diets was set as two types with or without a unique rare earth oxide: Y2O3, Yb2O3, La2O3, Sm2O3, Nd2O3 or Gd2O3 (only the first four rare earth oxides were used in adult diets) for four adaptation periods of 3, 7, 14 and 28 days respectively. After mixing, an equal mixture of all six diets for juvenile or four diets for adult was offered in excess for 2 days. During the last 2 days of each experiment, no feed was offered and faeces from each tank were collected. Feeding preference was expressed as relative feed intake of each diet, which was estimated based on the relative concentration of each marker in the faeces. Given some adaptation period, such as 3-28 days, the effects of MBM and squid extract inclusion on the preference to each diet were reduced. After 28 days adaptation, the preferences between groups were not significantly different.

Independent adaptation to riverine habitats allowed survival of ancient cetacean lineages

The four species of "river dolphins" are associated with six separate great river systems on three subcontinents and have been grouped for more than a century into a single taxon based on their similar appearance. However, several morphologists recently questioned the monophyly of that group. By using phylogenetic analyses of nucleotide sequences from three mitochondrial and two nuclear genes, we demonstrate with statistical significance that extant river dolphins are not monophyletic and suggest that they are relict species whose adaptation to riverine habitats incidentally insured their survival against major environmental changes in the marine ecosystem or the emergence of Delphinidae.

A VSLMS Style Tap-length Learning Algorithm for Structure Adaptation

Compared with the ordinary adaptive filter, the variable-length adaptive filter is more efficient (including smaller., lower power consumption and higher computational complexity output SNR) because of its tap-length learning algorithm, which is able to dynamically adapt its tap-length to the optimal tap-length that best balances the complexity and the performance of the adaptive filter. Among existing tap-length algorithms, the LMS-style Variable Tap-Length Algorithm (also called Fractional Tap-Length Algorithm or FT Algorithm) proposed by Y.Gong has the best performance because it has the fastest convergence rates and best stability. However, in some cases its performance deteriorates dramatically. To solve this problem, we first analyze the FT algorithm and point out some of its defects. Second, we propose a new FT algorithm called 'VSLMS' (Variable Step-size LMS) Style Tap-Length Learning Algorithm, which not only uses the concept of FT but also introduces a new concept of adaptive convergence slope. With this improvement the new FT algorithm has even faster convergence rates and better stability. Finally, we offer computer simulations to verify this improvement.

Kinetic spectra of light-adaptation dark-adaptation and M-intermediate of BR-D96N

BR-D96N is a kind of genetically site-specific mutants of bacteriorhodopsin (BR) with obvious photochromic effect. Compared to the wild type BR, the lifetime of M state of BR-D96N is prolonged to several minutes so that the photochromic kinetics and the intermediates formation can be studied by the conventional spectra analysis. In the experiment, the absorption spectra of the sample at different time after light illumination are measured with spectrophotometer. By fitting and analyzing the variation of the spectra, we suppose that there are three main states in the, photochromic process, i.e., B state (light-adapted state), M state and D state (dark-adapted state). The absorption spectra of the B state, M state and D state are extracted from the experimental data based on this three-state model and the spectra at various time are fitted with the least-square method. So, the variations of population percentages of the M state, B state and D state are obtained and the M state and B state lifetimes are estimated. In another way, from the measurement of the absorption dynamics at 407 and 568 nm, the M state and B state lifetimes are also obtained by two exponential data fitting, which give coincident results with those of the spectra analysis. (C) 2003 Elsevier Science B.V. All rights reserved.

A stochastic approximation algorithm with step size adaptation

Plakhov, A.Y.; Cruz, P., (2004) 'A stochastic approximation algorithm with step size adaptation', Journal of Mathematical Science 120(1) pp.964-973 RAE2008

Exploiting the Transients of Adaptation for RoQ Attacks on Internet Resources

In this paper, we expose an unorthodox adversarial attack that exploits the transients of a system's adaptive behavior, as opposed to its limited steady-state capacity. We show that a well orchestrated attack could introduce significant inefficiencies that could potentially deprive a network element from much of its capacity, or significantly reduce its service quality, while evading detection by consuming an unsuspicious, small fraction of that element's hijacked capacity. This type of attack stands in sharp contrast to traditional brute-force, sustained high-rate DoS attacks, as well as recently proposed attacks that exploit specific protocol settings such as TCP timeouts. We exemplify what we term as Reduction of Quality (RoQ) attacks by exposing the vulnerabilities of common adaptation mechanisms. We develop control-theoretic models and associated metrics to quantify these vulnerabilities. We present numerical and simulation results, which we validate with observations from real Internet experiments. Our findings motivate the need for the development of adaptation mechanisms that are resilient to these new forms of attacks.

Friendly Virtual Machine: Leveraging a Feedback-Control Model for Application Adaptation

With the increased use of "Virtual Machines" (VMs) as vehicles that isolate applications running on the same host, it is necessary to devise techniques that enable multiple VMs to share underlying resources both fairly and efficiently. To that end, one common approach is to deploy complex resource management techniques in the hosting infrastructure. Alternately, in this paper, we advocate the use of self-adaptation in the VMs themselves based on feedback about resource usage and availability. Consequently, we define a "Friendly" VM (FVM) to be a virtual machine that adjusts its demand for system resources, so that they are both efficiently and fairly allocated to competing FVMs. Such properties are ensured using one of many provably convergent control rules, such as AIMD. By adopting this distributed application-based approach to resource management, it is not necessary to make assumptions about the underlying resources nor about the requirements of FVMs competing for these resources. To demonstrate the elegance and simplicity of our approach, we present a prototype implementation of our FVM framework in User-Mode Linux (UML)-an implementation that consists of less than 500 lines of code changes to UML. We present an analytic, control-theoretic model of FVM adaptation, which establishes convergence and fairness properties. These properties are also backed up with experimental results using our prototype FVM implementation.