Investigation of acoustic units for LVCSR systems

One important issue in designing state-of-the-art LVCSR systems is the choice of acoustic units. Context dependent (CD) phones remain the dominant form of acoustic units. They can capture the co-articulatory effect in speech via explicit modelling. However, for other more complicated phonological processes, they rely on the implicit modelling ability of the underlying statistical models. Alternatively, it is possible to construct acoustic models based on higher level linguistic units, for example, syllables, to explicitly capture these complex patterns. When sufficient training data is available, this approach may show an advantage over implicit acoustic modelling. In this paper a wide range of acoustic units are investigated to improve LVCSR system performance. Significant error rate gains up to 7.1% relative (0.8% abs.) were obtained on a state-of-the-art Mandarin Chinese broadcast audio recognition task using word and syllable position dependent triphone and quinphone models. © 2011 IEEE.

Tracing the Origins of Hakka and Chaoshanese by Mitochondrial DNA Analysis

Hakka and Chaoshanese are two unique Han populations residing in southern China but with northern Han (NH) cultural traditions and linguistic influences. Although most of historical records indicate that both populations migrated from northern China in the last two thousand years, no consensus on their origins has been reached so far. To shed more light on the origins of Hakka and Chaoshanese, mitochondrial DNAs (mtDNAs) of 170 Hakka from Meizhou and 102 Chaoshanese from Chaoshan area, Guangdong Province, were analyzed. Our results show that some southern Chinese predominant haplogroups, e.g. B, F, and M7, have relatively high frequencies in both populations. Although median network analyses show that Hakka/Chaoshanese share some haplotypes with NH, interpopulation comparison reveals that both populations show closer affinity with southern Han (SH) populations than with NH. In consideration of previous results from nuclear gene (including Y chromosome) research, it is likely that matrilineal landscapes of both Hakka and Chaoshanese have largely been shaped by the local people during their migration southward and/or later colonization in southern China, and factors such as cultural assimilation, patrilocality, and even sex-bias in the immigrants might have played important roles during the process. Am J Phys Anthropol 141:124-130, 2010. (C) 2009 Wiley-Liss, Inc.

Population structure and history in East Asia

Archaeological, anatomical, linguistic, and genetic data have suggested that there is an old and significant boundary between the populations of north and south China. We use three human genetic marker systems and one human-carried virus to examine the north/south distinction. We find no support for a major north/south division in these markers; rather, the marker patterns suggest simple isolation by distance.

Unsupervised intra-lingual and cross-lingual speaker adaptation for HMM-based speech synthesis using two-pass decision tree construction

Hidden Markov model (HMM)-based speech synthesis systems possess several advantages over concatenative synthesis systems. One such advantage is the relative ease with which HMM-based systems are adapted to speakers not present in the training dataset. Speaker adaptation methods used in the field of HMM-based automatic speech recognition (ASR) are adopted for this task. In the case of unsupervised speaker adaptation, previous work has used a supplementary set of acoustic models to estimate the transcription of the adaptation data. This paper first presents an approach to the unsupervised speaker adaptation task for HMM-based speech synthesis models which avoids the need for such supplementary acoustic models. This is achieved by defining a mapping between HMM-based synthesis models and ASR-style models, via a two-pass decision tree construction process. Second, it is shown that this mapping also enables unsupervised adaptation of HMM-based speech synthesis models without the need to perform linguistic analysis of the estimated transcription of the adaptation data. Third, this paper demonstrates how this technique lends itself to the task of unsupervised cross-lingual adaptation of HMM-based speech synthesis models, and explains the advantages of such an approach. Finally, listener evaluations reveal that the proposed unsupervised adaptation methods deliver performance approaching that of supervised adaptation.

Syllable language models for Mandarin speech recognition: exploiting character language models.

Mandarin Chinese is based on characters which are syllabic in nature and morphological in meaning. All spoken languages have syllabiotactic rules which govern the construction of syllables and their allowed sequences. These constraints are not as restrictive as those learned from word sequences, but they can provide additional useful linguistic information. Hence, it is possible to improve speech recognition performance by appropriately combining these two types of constraints. For the Chinese language considered in this paper, character level language models (LMs) can be used as a first level approximation to allowed syllable sequences. To test this idea, word and character level n-gram LMs were trained on 2.8 billion words (equivalent to 4.3 billion characters) of texts from a wide collection of text sources. Both hypothesis and model based combination techniques were investigated to combine word and character level LMs. Significant character error rate reductions up to 7.3% relative were obtained on a state-of-the-art Mandarin Chinese broadcast audio recognition task using an adapted history dependent multi-level LM that performs a log-linearly combination of character and word level LMs. This supports the hypothesis that character or syllable sequence models are useful for improving Mandarin speech recognition performance.

An example of the use of neural computing techniques in materials science - The modelling of fatigue thresholds in Ni-base superalloys

Two adaptive numerical modelling techniques have been applied to prediction of fatigue thresholds in Ni-base superalloys. A Bayesian neural network and a neurofuzzy network have been compared, both of which have the ability to automatically adjust the network's complexity to the current dataset. In both cases, despite inevitable data restrictions, threshold values have been modelled with some degree of success. However, it is argued in this paper that the neurofuzzy modelling approach offers real benefits over the use of a classical neural network as the mathematical complexity of the relationships can be restricted to allow for the paucity of data, and the linguistic fuzzy rules produced allow assessment of the model without extensive interrogation and examination using a hypothetical dataset. The additive neurofuzzy network structure means that redundant inputs can be excluded from the model and simple sub-networks produced which represent global output trends. Both of these aspects are important for final verification and validation of the information extracted from the numerical data. In some situations neurofuzzy networks may require less data to produce a stable solution, and may be easier to verify in the light of existing physical understanding because of the production of transparent linguistic rules. © 1999 Elsevier Science S.A.

A Structured Prediction Approach for Statistical Machine Translation

We propose a new formally syntax-based method for statistical machine translation. Transductions between parsing trees are transformed into a problem of sequence tagging, which is then tackled by a search- based structured prediction method. This allows us to automatically acquire transla- tion knowledge from a parallel corpus without the need of complex linguistic parsing. This method can achieve compa- rable results with phrase-based method (like Pharaoh), however, only about ten percent number of translation table is used. Experiments show that the structured pre- diction approach for SMT is promising for its strong ability at combining words.

基于意象知识的消歧体系

本文提出了一种可以表示常识及语言知识的意象知识体系。在这种知识的形式化表示基础上,给出了NLP中的消歧知识及其表示形式,以及基于消歧知识的消歧策略。最后,论述了这种方法实现上的可行性。

学习困难儿童智力结构及其与心理语言、视动能力关系的研究

IQ Structure, Psycholinguistic and Visual-motor Abilities Study on Children Learning Disability TONG Fang Directed by professor Zhu Liqi (Developmental and educational psychology) ABSTRACT Objective To comprehensive analyze the IQ structures, and relationships among IQ, psychometric characteristics and visual-motor integration on children disability. At same time, to probe into the family factors that influenced IQ, psycholinguistic abilities and behavior of LD children. Method (1) Downloading the papers on children learning disability from www.cqvip.com and www.wanfangdata.com, in which, the articles were collected by key words from 1985 to 2005. To conduct meta-analysis on IQ construction, compare the case group and the control group, including full IQ, verbal and practice IQ. (2) Designed with model compared and self-compared, 59 diagnosed learning disability children, tested themes with WISC, ITPA and Berry’s VMI. WISC included 10 items, 5 of which subtotal to verbal and practice IQ respectively. IPTA included 10 items, too, 5 process of which subtotal to auditory and visual perception. The first 3 items shared representation level, the other 2 of that shared automatic level.VMI had one score. Analyzed factors and levels with description and Pearson Correlation. To probe to linguistic internal alternately functions of LD children, and compare the scores of groups in different IQ. (3) Analyzed the perspective questionnaire filled by parents. Early development facts compared with model groups. Factors relationships analyzed with Kendall correlation, KOM and Bartlett’s test of sphericity, Promax Rotation. Results: (1) There have been 319 papers related with LD, in which 36 with IQ and 14 valid reports have been analyzed by Meta. FIQ’s 95%CI (confidence interval) is 2.418 ~ 0.172, VIQ between the difficulty and non- difficulty group. C-WISC-R reports were 10 papers, of which, 95%CI of FIQ is 2.424 ~ 0.676, of VIQ is 2.314 ~ 1.196, of PIQ is 2.176 ~ 0.176. The VIQ comparing the PIQ, 95%CI is 1.1 ~ -0.07 in difficulty group and 0.5 ~ -0.0046 in non-difficult group. Nevertheless, in the other 4 tests, FIQ’s 95%CI is 2.00 ~ -0.818 between LD and NLD. (2) Children psycholinguistic abilities had strong relation with Berry’s VMI test excluding auditory reception, and with perceptive factor of intelligence excluding verbal expression. Auditory reception and visual closure had strong relation with FIQ and PIQ. Grammatic closure, visual association and manual expression had strong relation with concept factor. The representational and automatic levels are depended on integration of auditory and visual procession. Lower verbal expression (VE) let to lower expression process and low scores on representational level. Lower visual sequential memory (VSM) let to lower memory process and influenced automatic level. Groups compared by IQ 90 show that LD children with under IQ 90 had lower scores on items of IPTA than with up IQ 90 excluded verbal expression. It was proved that IQ administrated the linguistic ability. Nevertheless, general abilities deficiency didn’t show influencing on the types of the perceptive delay. There was mutual function among linguistic ability on LD children. Auditory and visual level are overlapped each other. Not only show higher Decoding and lower Encoding on Auditory perception, lower Decoding and higher Encoding on Visual perception, in representation, but also higher Sequential remember, lower Closure on Audition, and lower Sequential member, higher Closure on Vision, in Automation. Nevertheless, there was no different between Representational and Automatic level, which may be the relationship of parallel or evolution. (3) Major family factors were father’s education, occupation. Lower auditory perception related to unconcerned, lower visual perception related to premature delivery and written slowly. Threatened–abortion, childbirth-suffocated were known as influencing children’s IQ and later linguistic abilities. It wasn’t shown that dosage relationship with the types of perceptive delay. Conclusion: (1) The FIQ, VIQ and PIQ of Children with LD is lower than that of NLD group. There is no significantly different between VIQ and PIQ in LD and NLD groups. (2) The objectives of ITPA and WISC tests are differently. The psycholinguistic abilities had strong relation with perceptive factor and VMI. Some facts of IPTA related with FIQ. IQ had strong administration on linguistic abilities. There was mutual function among linguistic internal abilities. (3) Family facts on IQ and psycholinguistic abilities were Father’s education, abnormal pregnant and abortion. It would be pre-show development delay in early period.

通过语言任务和功能磁共振成像研究人类小脑功能

Since the 19th century, people have long believed that the function of cerebellum was restricted to fine motor control and modulation. In the past two decades, however, more and more studies challenged this traditional view. While the neuroanatomy of the cerebellum from cellular to system level has been well documented, the functions of this neural organ remain poorly understood. This study, including three experiments, attempted to further the understanding of cerebellar functions from different viewpoints. Experiment One used the parametric design to control motor effects. The activation in cerebellum was found to be associated with the difficulty levels of a semantic discrimination task, suggesting the involvement of the cerebellum in higher level of language functions. Moreover, activation of the right posterior cerebellum was found to co-vary with that of the frontal cortex. Experiment Two adopted the cue-go paradigm and event-related design to exclude the effects of phonological and semantic factors in a mental writing task. The results showed that bilateral anterior cerebellum and cerebral motor regions were significantly activated during the task and the hemodynamic response of the cerebellum was similar to those of the cerebral motor cortex. These results suggest that the cerebellum participates in motor imagination during orthographic output. Experiment Three investigated the learning process of a verb generation task. While both lateral and vermis cerebellum were found to be activation in the task, each was correlated a separate set of frontal regions. More importantly, activations both in the cerebellum and frontal cortex decreased with the repetition of the task. These results indicate that the cerebellum and frontal cortex is jointly engaged in some functions; each serves as a part of a single functional system. Taken these findings together, the following conclusions can be drawn: 1.The cerebellum is not only involved in functions related to speech or articulation, but also participates in the higher cognitive functions of language. 2.The cerebellum participates in various functions by supporting the corresponding regions in cerebral cortex, but not directly executes the functions as an independent module. 3.The anterior part of cerebellum is related to motor functions, whereas the posterior part is involved in cognitive functions. 4.While the motor functions rely on the engagement of both sides of the cerebellar hemispheres, the higher cognitive functions mainly depend on the right cerebellum.

Computational Consequences of Agreement and Ambiguity in Natural Language

The computer science technique of computational complexity analysis can provide powerful insights into the algorithm-neutral analysis of information processing tasks. Here we show that a simple, theory-neutral linguistic model of syntactic agreement and ambiguity demonstrates that natural language parsing may be computationally intractable. Significantly, we show that it may be syntactic features rather than rules that can cause this difficulty. Informally, human languages and the computationally intractable Satisfiability (SAT) problem share two costly computional mechanisms: both enforce agreement among symbols across unbounded distances (Subject-Verb agreement) and both allow ambiguity (is a word a Noun or a Verb?).

A Computational Model for the Acquisition and Use of Phonological Knowledge

Does knowledge of language consist of symbolic rules? How do children learn and use their linguistic knowledge? To elucidate these questions, we present a computational model that acquires phonological knowledge from a corpus of common English nouns and verbs. In our model the phonological knowledge is encapsulated as boolean constraints operating on classical linguistic representations of speech sounds in term of distinctive features. The learning algorithm compiles a corpus of words into increasingly sophisticated constraints. The algorithm is incremental, greedy, and fast. It yields one-shot learning of phonological constraints from a few examples. Our system exhibits behavior similar to that of young children learning phonological knowledge. As a bonus the constraints can be interpreted as classical linguistic rules. The computational model can be implemented by a surprisingly simple hardware mechanism. Our mechanism also sheds light on a fundamental AI question: How are signals related to symbols?

Sparse Representations for Fast, One-Shot Learning

Humans rapidly and reliably learn many kinds of regularities and generalizations. We propose a novel model of fast learning that exploits the properties of sparse representations and the constraints imposed by a plausible hardware mechanism. To demonstrate our approach we describe a computational model of acquisition in the domain of morphophonology. We encapsulate phonological information as bidirectional boolean constraint relations operating on the classical linguistic representations of speech sounds in term of distinctive features. The performance model is described as a hardware mechanism that incrementally enforces the constraints. Phonological behavior arises from the action of this mechanism. Constraints are induced from a corpus of common English nouns and verbs. The induction algorithm compiles the corpus into increasingly sophisticated constraints. The algorithm yields one-shot learning from a few examples. Our model has been implemented as a computer program. The program exhibits phonological behavior similar to that of young children. As a bonus the constraints that are acquired can be interpreted as classical linguistic rules.

Tense, Aspect and the Cognitive Representation of Time

This paper explores the relationships between a computation theory of temporal representation (as developed by James Allen) and a formal linguistic theory of tense (as developed by Norbert Hornstein) and aspect. It aims to provide explicit answers to four fundamental questions: (1) what is the computational justification for the primitive of a linguistic theory; (2) what is the computational explanation of the formal grammatical constraints; (3) what are the processing constraints imposed on the learnability and markedness of these theoretical constructs; and (4) what are the constraints that a linguistic theory imposes on representations. We show that one can effectively exploit the interface between the language faculty and the cognitive faculties by using linguistic constraints to determine restrictions on the cognitive representation and vice versa. Three main results are obtained: (1) We derive an explanation of an observed grammatical constraint on tense?? Linear Order Constraint??m the information monotonicity property of the constraint propagation algorithm of Allen's temporal system: (2) We formulate a principle of markedness for the basic tense structures based on the computational efficiency of the temporal representations; and (3) We show Allen's interval-based temporal system is not arbitrary, but it can be used to explain independently motivated linguistic constraints on tense and aspect interpretations. We also claim that the methodology of research developed in this study??oss-level" investigation of independently motivated formal grammatical theory and computational models??a powerful paradigm with which to attack representational problems in basic cognitive domains, e.g., space, time, causality, etc.

Computational Structure of GPSG Models: Revised Generalized Phrase Structure Grammar

The primary goal of this report is to demonstrate how considerations from computational complexity theory can inform grammatical theorizing. To this end, generalized phrase structure grammar (GPSG) linguistic theory is revised so that its power more closely matches the limited ability of an ideal speaker--hearer: GPSG Recognition is EXP-POLY time hard, while Revised GPSG Recognition is NP-complete. A second goal is to provide a theoretical framework within which to better understand the wide range of existing GPSG models, embodied in formal definitions as well as in implemented computer programs. A grammar for English and an informal explanation of the GPSG/RGPSG syntactic features are included in appendices.