Improving LVCSR system combination using neural network language model cross adaptation

State-of-the-art large vocabulary continuous speech recognition (LVCSR) systems often combine outputs from multiple subsystems developed at different sites. Cross system adaptation can be used as an alternative to direct hypothesis level combination schemes such as ROVER. The standard approach involves only cross adapting acoustic models. To fully exploit the complimentary features among sub-systems, language model (LM) cross adaptation techniques can be used. Previous research on multi-level n-gram LM cross adaptation is extended to further include the cross adaptation of neural network LMs in this paper. Using this improved LM cross adaptation framework, significant error rate gains of 4.0%-7.1% relative were obtained over acoustic model only cross adaptation when combining a range of Chinese LVCSR sub-systems used in the 2010 and 2011 DARPA GALE evaluations. Copyright © 2011 ISCA.

Use of contexts in language model interpolation and adaptation

Language models (LMs) are often constructed by building multiple individual component models that are combined using context independent interpolation weights. By tuning these weights, using either perplexity or discriminative approaches, it is possible to adapt LMs to a particular task. This paper investigates the use of context dependent weighting in both interpolation and test-time adaptation of language models. Depending on the previous word contexts, a discrete history weighting function is used to adjust the contribution from each component model. As this dramatically increases the number of parameters to estimate, robust weight estimation schemes are required. Several approaches are described in this paper. The first approach is based on MAP estimation where interpolation weights of lower order contexts are used as smoothing priors. The second approach uses training data to ensure robust estimation of LM interpolation weights. This can also serve as a smoothing prior for MAP adaptation. A normalized perplexity metric is proposed to handle the bias of the standard perplexity criterion to corpus size. A range of schemes to combine weight information obtained from training data and test data hypotheses are also proposed to improve robustness during context dependent LM adaptation. In addition, a minimum Bayes' risk (MBR) based discriminative training scheme is also proposed. An efficient weighted finite state transducer (WFST) decoding algorithm for context dependent interpolation is also presented. The proposed technique was evaluated using a state-of-the-art Mandarin Chinese broadcast speech transcription task. Character error rate (CER) reductions up to 7.3 relative were obtained as well as consistent perplexity improvements. © 2012 Elsevier Ltd. All rights reserved.

Syllable language models for Mandarin speech recognition: exploiting character language models.

Mandarin Chinese is based on characters which are syllabic in nature and morphological in meaning. All spoken languages have syllabiotactic rules which govern the construction of syllables and their allowed sequences. These constraints are not as restrictive as those learned from word sequences, but they can provide additional useful linguistic information. Hence, it is possible to improve speech recognition performance by appropriately combining these two types of constraints. For the Chinese language considered in this paper, character level language models (LMs) can be used as a first level approximation to allowed syllable sequences. To test this idea, word and character level n-gram LMs were trained on 2.8 billion words (equivalent to 4.3 billion characters) of texts from a wide collection of text sources. Both hypothesis and model based combination techniques were investigated to combine word and character level LMs. Significant character error rate reductions up to 7.3% relative were obtained on a state-of-the-art Mandarin Chinese broadcast audio recognition task using an adapted history dependent multi-level LM that performs a log-linearly combination of character and word level LMs. This supports the hypothesis that character or syllable sequence models are useful for improving Mandarin speech recognition performance.

Discriminative spoken language understanding using word confusion networks

Current commercial dialogue systems typically use hand-crafted grammars for Spoken Language Understanding (SLU) operating on the top one or two hypotheses output by the speech recogniser. These systems are expensive to develop and they suffer from significant degradation in performance when faced with recognition errors. This paper presents a robust method for SLU based on features extracted from the full posterior distribution of recognition hypotheses encoded in the form of word confusion networks. Following [1], the system uses SVM classifiers operating on n-gram features, trained on unaligned input/output pairs. Performance is evaluated on both an off-line corpus and on-line in a live user trial. It is shown that a statistical discriminative approach to SLU operating on the full posterior ASR output distribution can substantially improve performance both in terms of accuracy and overall dialogue reward. Furthermore, additional gains can be obtained by incorporating features from the previous system output. © 2012 IEEE.

Language model cross adaptation for LVCSR system combination

State-of-the-art large vocabulary continuous speech recognition (LVCSR) systems often combine outputs from multiple sub-systems that may even be developed at different sites. Cross system adaptation, in which model adaptation is performed using the outputs from another sub-system, can be used as an alternative to hypothesis level combination schemes such as ROVER. Normally cross adaptation is only performed on the acoustic models. However, there are many other levels in LVCSR systems' modelling hierarchy where complimentary features may be exploited, for example, the sub-word and the word level, to further improve cross adaptation based system combination. It is thus interesting to also cross adapt language models (LMs) to capture these additional useful features. In this paper cross adaptation is applied to three forms of language models, a multi-level LM that models both syllable and word sequences, a word level neural network LM, and the linear combination of the two. Significant error rate reductions of 4.0-7.1% relative were obtained over ROVER and acoustic model only cross adaptation when combining a range of Chinese LVCSR sub-systems used in the 2010 and 2011 DARPA GALE evaluations. © 2012 Elsevier Ltd. All rights reserved.

Paraphrastic language models

In natural languages multiple word sequences can represent the same underlying meaning. Only modelling the observed surface word sequence can result in poor context coverage, for example, when using n-gram language models (LM). To handle this issue, this paper presents a novel form of language model, the paraphrastic LM. A phrase level transduction model that is statistically learned from standard text data is used to generate paraphrase variants. LM probabilities are then estimated by maximizing their marginal probability. Significant error rate reductions of 0.5%-0.6% absolute were obtained on a state-ofthe-art conversational telephone speech recognition task using a paraphrastic multi-level LM modelling both word and phrase sequences.

Paraphrastic language models and combination with neural network language models

In natural languages multiple word sequences can represent the same underlying meaning. Only modelling the observed surface word sequence can result in poor context coverage, for example, when using n-gram language models (LM). To handle this issue, paraphrastic LMs were proposed in previous research and successfully applied to a US English conversational telephone speech transcription task. In order to exploit the complementary characteristics of paraphrastic LMs and neural network LMs (NNLM), the combination between the two is investigated in this paper. To investigate paraphrastic LMs' generalization ability to other languages, experiments are conducted on a Mandarin Chinese broadcast speech transcription task. Using a paraphrastic multi-level LM modelling both word and phrase sequences, significant error rate reductions of 0.9% absolute (9% relative) and 0.5% absolute (5% relative) were obtained over the baseline n-gram and NNLM systems respectively, after a combination with word and phrase level NNLMs. © 2013 IEEE.

Subcellular localization and inductive expression of the dsRNA-dependent protein kinase PKR from Japanese flounder, Paralichthys olivaceus

The double-stranded RNA (dsRNA)-dependent protein kinase (PKR) belongs to the eIF2 alpha kinase family and plays a critical role in interferon (IFN)-mediated antiviral response. Recently, in Japanese flounder (Paralichthys olivaceus), a PKR gene has been identified. In this study, we showed that PoPKR localized to the cytoplasm, and the dsRNA-binding motifs (dsRBMs) played a determinative role in protein localization. In cultured FEC cells, PoPKR was detected at a low level of constitutive expression but was highly induced after treatment with UV-inactivated grass carp hemorrhagic virus, active SMRV and Poly I:C although with different expression kinetics. In flounder, PoPKR was ubiquitously distributed in all tested tissues, and SMRV infection resulted in significant upregulation at mRNA and protein levels. In order to reveal the role of PoPKR in host antiviral response, its expression upon exposure to various inducers was characterized and further compared with that of PoHRI, which is another eIF2 alpha kinase of flounder. Interestingly, expression comparison revealed that all inducers stimulated upregulation of PoHRI in cultured flounder embryonic cells and fish, with a similar kinetics to PoPKR but to a less extent. These results suggest that, during antiviral immune response, both flounder eIF2 alpha kinases might play similar roles and that PoPKR is the predominant kinase. (C) 2009 National Natural Science Foundation of China and Chinese Academy of Sciences. Published by Elsevier Limited and Science in China Press. All rights reserved.

Automatic construction and natural-language description of nonparametric regression models

Copyright © 2014, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural- language text. Our approach treats unknown regression functions non- parametrically using Gaussian processes, which has two important consequences. First, Gaussian processes can model functions in terms of high-level properties (e.g. smoothness, trends, periodicity, changepoints). Taken together with the compositional structure of our language of models this allows us to automatically describe functions in simple terms. Second, the use of flexible nonparametric models and a rich language for composing them in an open-ended manner also results in state- of-the-art extrapolation performance evaluated over 13 real time series data sets from various domains.

Unified form language: A domain-specific language for weak formulations of partial differential equations

We present the Unified Form Language (UFL), which is a domain-specific language for representing weak formulations of partial differential equations with a view to numerical approximation. Features of UFL include support for variational forms and functionals, automatic differentiation of forms and expressions, arbitrary function space hierarchies formultifield problems, general differential operators and flexible tensor algebra. With these features, UFL has been used to effortlessly express finite element methods for complex systems of partial differential equations in near-mathematical notation, resulting in compact, intuitive and readable programs. We present in this work the language and its construction. An implementation of UFL is freely available as an open-source software library. The library generates abstract syntax tree representations of variational problems, which are used by other software libraries to generate concrete low-level implementations. Some application examples are presented and libraries that support UFL are highlighted. © 2014 ACM.

Molecular identification and expression analysis of natural resistance associated macrophage protein (Nramp) cDNA from Japanese flounder (Paralichthys olivaceus)

Natural resistance associated macrophage protein (Nramp) controls partially innate resistance to intracellular parasites. Its function is to enhance the ability of macrophages to kill pathogens. However, little is known about the structure and function of Nramp in lower vertebrates such as teleosts. We have recently isolated a cDNA encoding Nramp from Japanese flounder (Paratichthys olivaceus). The full-length cDNA of the Nramp is 3066 bp in length, including 224 bp 5' terminal UTR, 1662 bp encoding region and 1180 bp 3' terminal UTR. The 1662-nt open reading frame was found to code for a protein with 554 amino acid residues. Comparison of amino acid sequence indicated that Japanese flounder Nramp consists of 12 transmembrane (TM) domains. A consensus transport motif (CTM) containing 20 residues was observed between transmembrane domains 8 and 9. The deduced amino acid sequence of Japanese flounder had 77.30%, 82.71%, 82.67%, 79.64%, 80.72%, 90.97%, 91.16%, 60.14%, 71.48%, 61.69%, 72.37% identity with that of rainbow trout Nramp alpha and beta, channel catfish Nramp, fathead minnow Nramp, common carp Nramp, striped sea bass Nramp, red sea bream Nramp, mouse Nramp 1 and 2, human Nramp 1 and 2, respectively. RT-PCR indicated that Nramp transcripts were highly abundant in spleen, head kidney, abundant in intestine, liver and gill, and less abundant in heart. The level of Nramp mRNA in embryos gradually increases during embryogenesis from 4 h (8 cell stage) to 80 h (hatched stage) after fertilization. (c) 2005 Elsevier Ltd. All rights reserved.

Biomarker responses and reproductive toxicity of the effluent from a Chinese large sewage treatment plant in Japanese medaka (Oryzias latipes)

The present study was conducted to assess the potential toxicity of the effluent from a large sewage treatment plant (GBD-STP) in Beijing. Japanese medakas (Oryzias latipes) at reproduction active period were exposed to a serial of graded concentrations of the effluent or 100 ng l(-1) of 17-alpha-ethinylestradiol (EE2, positive control). Growth, gonadosomatic index (GSI), hepatosomatic index (HSI), reproductive success, induction potency of vitellogenin (VTG) in male fish and that of 7-ethoxyresorufin-o-deethylase activity (EROD) in male fish liver were used as test endpoints. The growth suppression of fish was observed in a dose-dependent manner, resulting in significant differences in both body length and body weight of medaka above 5% effluent. This effluent can inhibit the growth of gonad of medakas and are more sensitive to male than to female. At exposure concentration of 40% and higher, there was an unexpected decrease of HSI values, which may be resulted from sub-lethal toxicity of effluent to fish liver. VTG of plasma in males were induced in all exposure concentration levels, but not in a dose-dependent manner. The concentration of 5% effluent would be the lowest observed adverse effect level (LOAEL) affecting reproductive success when examining fertile individuals, fecundity and fertilization rate. The overt CYP1A response and higher reproductive toxicity may be indicative of low process efficiency of this STP. (c) 2004 Elsevier Ltd. All rights reserved.