Learning task specific distributed paragraph representations using a 2-tier convolutional neural network

We introduce a type of 2-tier convolutional neural network model for learning distributed paragraph representations for a special task (e.g. paragraph or short document level sentiment analysis and text topic categorization). We decompose the paragraph semantics into 3 cascaded constitutes: word representation, sentence composition and document composition. Specifically, we learn distributed word representations by a continuous bag-of-words model from a large unstructured text corpus. Then, using these word representations as pre-trained vectors, distributed task specific sentence representations are learned from a sentence level corpus with task-specific labels by the first tier of our model. Using these sentence representations as distributed paragraph representation vectors, distributed paragraph representations are learned from a paragraph-level corpus by the second tier of our model. It is evaluated on DBpedia ontology classification dataset and Amazon review dataset. Empirical results show the effectiveness of our proposed learning model for generating distributed paragraph representations.

A novel framework of training hidden Markov support vector machines from lightly-annotated data

Natural language understanding (NLU) aims to map sentences to their semantic mean representations. Statistical approaches to NLU normally require fully-annotated training data where each sentence is paired with its word-level semantic annotations. In this paper, we propose a novel learning framework which trains the Hidden Markov Support Vector Machines (HM-SVMs) without the use of expensive fully-annotated data. In particular, our learning approach takes as input a training set of sentences labeled with abstract semantic annotations encoding underlying embedded structural relations and automatically induces derivation rules that map sentences to their semantic meaning representations. The proposed approach has been tested on the DARPA Communicator Data and achieved 93.18% in F-measure, which outperforms the previously proposed approaches of training the hidden vector state model or conditional random fields from unaligned data, with a relative error reduction rate of 43.3% and 10.6% being achieved.

Pharmacy legislation:public protector or professional hindrance?

Pharmacy originates from a background of medication compounding and supply. More recently, this role has developed away from an absolute focus on the supply of pharmaceuticals with, for example, the advent of pharmacist prescribing. Nevertheless, for a majority of the profession, medication supply remains a core activity. Regulation of the pharmacy profession is now the responsibility of the General Pharmaceutical Council, although up until 27 September 2010, this role fell to the Royal Pharmaceutical Society of Great Britain (RPSGB). Before this change, in one of the most high-profile legal cases involving a pharmacist in a professional capacity, R. v Lee, a pharmacist was prosecuted firstly for gross negligence manslaughter, later revised to offences under the Medicines Act 1968, for a single error relating to medication supply, and was given a suspended custodial sentence. Offences against sections 64 or 85 of the Medicines Act are absolute offences and there is no due diligence defence. Prosecution of a pharmacist for the supply of incorrect medication may seem a measured course of action to protect the public from the wrongful supply of potent pharmacotherapeutic agents; however, further analysis of Lee indicates that this approach may be counterproductive. An appeal of the original conviction in the Lee case has resulted in a clarification of the interpretation of section 85(5); however currently, prosecutions under section 64 are still a possibility. Owing to the seriousness of a criminal conviction under section 64, this continuation will potentially stifle the profession's ability to learn from dispensing errors. © The Author [2013]. Published by Oxford University Press; all rights reserved.

A fuzzy decision tree-based duration model for Standard Yorùbá text-to-speech synthesis

In this paper, we present syllable-based duration modelling in the context of a prosody model for Standard Yorùbá (SY) text-to-speech (TTS) synthesis applications. Our prosody model is conceptualised around a modular holistic framework. This framework is implemented using the Relational Tree (R-Tree) techniques. An important feature of our R-Tree framework is its flexibility in that it facilitates the independent implementation of the different dimensions of prosody, i.e. duration, intonation, and intensity, using different techniques and their subsequent integration. We applied the Fuzzy Decision Tree (FDT) technique to model the duration dimension. In order to evaluate the effectiveness of FDT in duration modelling, we have also developed a Classification And Regression Tree (CART) based duration model using the same speech data. Each of these models was integrated into our R-Tree based prosody model. We performed both quantitative (i.e. Root Mean Square Error (RMSE) and Correlation (Corr)) and qualitative (i.e. intelligibility and naturalness) evaluations on the two duration models. The results show that CART models the training data more accurately than FDT. The FDT model, however, shows a better ability to extrapolate from the training data since it achieved a better accuracy for the test data set. Our qualitative evaluation results show that our FDT model produces synthesised speech that is perceived to be more natural than our CART model. In addition, we also observed that the expressiveness of FDT is much better than that of CART. That is because the representation in FDT is not restricted to a set of piece-wise or discrete constant approximation. We, therefore, conclude that the FDT approach is a practical approach for duration modelling in SY TTS applications. © 2006 Elsevier Ltd. All rights reserved.

Intonation contour realisation for Standard Yorùbá text-to-speech synthesis:a fuzzy computational approach

This paper presents a novel intonation modelling approach and demonstrates its applicability using the Standard Yorùbá language. Our approach is motivated by the theory that abstract and realised forms of intonation and other dimensions of prosody should be modelled within a modular and unified framework. In our model, this framework is implemented using the Relational Tree (R-Tree) technique. The R-Tree is a sophisticated data structure for representing a multi-dimensional waveform in the form of a tree. Our R-Tree for an utterance is generated in two steps. First, the abstract structure of the waveform, called the Skeletal Tree (S-Tree), is generated using tone phonological rules for the target language. Second, the numerical values of the perceptually significant peaks and valleys on the S-Tree are computed using a fuzzy logic based model. The resulting points are then joined by applying interpolation techniques. The actual intonation contour is synthesised by Pitch Synchronous Overlap Technique (PSOLA) using the Praat software. We performed both quantitative and qualitative evaluations of our model. The preliminary results suggest that, although the model does not predict the numerical speech data as accurately as contemporary data-driven approaches, it produces synthetic speech with comparable intelligibility and naturalness. Furthermore, our model is easy to implement, interpret and adapt to other tone languages.

Language as chunks, not words

Many people think of language as words. Words are small, convenient units, especially in written English, where they are separated by spaces. Dictionaries seem to reinforce this idea, because entries are arranged as a list of alphabetically-ordered words. Traditionally, linguists and teachers focused on grammar and treated words as self-contained units of meaning, which fill the available grammatical slots in a sentence. More recently, attention has shifted from grammar to lexis, and from words to chunks. Dictionary headwords are convenient points of access for the user, but modern dictionary entries usually deal with chunks, because meanings often do not arise from individual words, but from the chunks in which the words occur. Corpus research confirms that native speakers of a language actually work with larger “chunks” of language. This paper will show that teachers and learners will benefit from treating language as chunks rather than words.

Satisfaction with primary care:the perspectives of people with schizophrenia

Background. Schizophrenia affects up to 1% of the population in the UK. People with schizophrenia use the National Health Service frequently and over a long period of time. However, their views on satisfaction with primary care are rarely sought. Objectives. This study aimed to explore the elements of satisfaction with primary care for people with schizophrenia. Method. A primary care-based study was carried out using semi-structured interviews with 45 patients with schizophrenia receiving shared care with the Northern Birmingham Mental Health Trust between 1999 and 2000. Results. Five major themes that affect satisfaction emerged from the data: the exceptional potential of the consultation itself; the importance of aspects of the organization of primary care; the construction of the user in the doctor-patient relationship; the influence of stereotypes on GP behaviour; and the importance of hope for recovery. Conclusion. Satisfaction with primary care is multiply mediated. It is also rarely expected or achieved by this group of patients. There is a significant gap between the rhetoric and the reality of user involvement in primary care consultations. Acknowledging the tensions between societal and GP views of schizophrenia as an incurable life sentence and the importance to patients of hope for recovery is likely to lead to greater satisfaction with primary health care for people with schizophrenia.

Event trigger identification for biomedical events extraction using domain knowledge

Motivation: In molecular biology, molecular events describe observable alterations of biomolecules, such as binding of proteins or RNA production. These events might be responsible for drug reactions or development of certain diseases. As such, biomedical event extraction, the process of automatically detecting description of molecular interactions in research articles, attracted substantial research interest recently. Event trigger identification, detecting the words describing the event types, is a crucial and prerequisite step in the pipeline process of biomedical event extraction. Taking the event types as classes, event trigger identification can be viewed as a classification task. For each word in a sentence, a trained classifier predicts whether the word corresponds to an event type and which event type based on the context features. Therefore, a well-designed feature set with a good level of discrimination and generalization is crucial for the performance of event trigger identification. Results: In this article, we propose a novel framework for event trigger identification. In particular, we learn biomedical domain knowledge from a large text corpus built from Medline and embed it into word features using neural language modeling. The embedded features are then combined with the syntactic and semantic context features using the multiple kernel learning method. The combined feature set is used for training the event trigger classifier. Experimental results on the golden standard corpus show that >2.5% improvement on F-score is achieved by the proposed framework when compared with the state-of-the-art approach, demonstrating the effectiveness of the proposed framework. © 2014 The Author 2014. The source code for the proposed framework is freely available and can be downloaded at http://cse.seu.edu.cn/people/zhoudeyu/ETI_Sourcecode.zip.

Improving distributed representation of word sense via WordNet Gloss composition and context clustering

In recent years, there has been an increas-ing interest in learning a distributed rep-resentation of word sense. Traditional context clustering based models usually require careful tuning of model parame-ters, and typically perform worse on infre-quent word senses. This paper presents a novel approach which addresses these lim-itations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned represen-tations outperform the publicly available embeddings on 2 out of 4 metrics in the word similarity task, and 6 out of 13 sub tasks in the analogical reasoning task.

Are school-of-thought words characterizable?

School of thought analysis is an important yet not-well-elaborated scientific knowledge discovery task. This paper makes the first attempt at this problem. We focus on one aspect of the problem: do characteristic school-of-thought words exist and whether they are characterizable? To answer these questions, we propose a probabilistic generative School-Of-Thought (SOT) model to simulate the scientific authoring process based on several assumptions. SOT defines a school of thought as a distribution of topics and assumes that authors determine the school of thought for each sentence before choosing words to deliver scientific ideas. SOT distinguishes between two types of school-of-thought words for either the general background of a school of thought or the original ideas each paper contributes to its school of thought. Narrative and quantitative experiments show positive and promising results to the questions raised above © 2013 Association for Computational Linguistics. © 2013 Association for Computational Linguistics.

A gloss composition and context clustering based distributed word sense representation model

In recent years, there has been an increasing interest in learning a distributed representation of word sense. Traditional context clustering based models usually require careful tuning of model parameters, and typically perform worse on infrequent word senses. This paper presents a novel approach which addresses these limitations by first initializing the word sense embeddings through learning sentence-level embeddings from WordNet glosses using a convolutional neural networks. The initialized word sense embeddings are used by a context clustering based model to generate the distributed representations of word senses. Our learned representations outperform the publicly available embeddings on half of the metrics in the word similarity task, 6 out of 13 sub tasks in the analogical reasoning task, and gives the best overall accuracy in the word sense effect classification task, which shows the effectiveness of our proposed distributed distribution learning model.

Editorial

The year so far has been a slow start for many businesses, but at least we have not seen the collapse of as many businesses that we were seeing around two years ago. We are, however, still well and truly in the midst of a global recession. Interest rates are still at an all time low, UK house prices seem to be showing little signs of increase (except in London where everyone still seems to want to live!) and for the ardent shopper there are bargains to be had everywhere. It seems strange that prices on the high street do not seem to have increased in over ten years. Mobile phones, DVD players even furniture seems to be cheaper than they used to be. Whist much of this is down to cheaper manufacturing and the rest could probably be explained by competition within the market place. Does this mean that quality suffered too? Now that we live in a world when if a television is not working it is thrown away and replaced. There was a time when you would take it to some odd looking man that your father would know who could fix it for you. (I remember our local television fix-it man, with his thick rimmed bifocal spectacles and a poor comb-over; he had cardboard boxes full of resistors and electrical wires on the floor of his front room that smelt of soldering irons!) Is this consumerism at an extreme or has this move to disposability made us a better society? Before you think these are just ramblings there is a point to this. According to latest global figures of contact lens sales the vast majority of contact lenses fitted around the world are daily, fortnightly or monthly disposable hydrogel lenses. Certainly in the UK over 90% of lenses are disposable (with daily disposables being the most popular, having a market share of over 50%). This begs the question – is this a good thing? Maybe more importantly, do our patients benefit? I think it is worth reminding ourselves why we went down the disposability route with contact lenses in the first place, and unlike electrical goods it was not just so we did not have to take them for repair! There are the obvious advantages of overcoming problems of breakage and tearing of lenses and the lens deterioration with age. The lenses are less likely to be contaminated and the disinfection is either easier or not required at all (in the case of daily disposable lenses). Probably the landmark paper in the field was the work more commonly known as the ‘Gothenburg Study’. The paper, entitled ‘Strategies for minimizing the Ocular Effects of Extended Contact Lens Wear’ published in the American Journal of Optometry in 1987 (volume 64, pages 781-789) by Holden, B.A., Swarbrick, H.A., Sweeney, D.F., Ho, A., Efron, N., Vannas, A., Nilsson, K.T. They suggested that contact lens induced ocular effects were minimised by: •More frequently removed contact lenses •More regularly replaced contact lenses •A lens that was more mobile on the eye (to allow better removal of debris) •Better flow of oxygen through the lens All of these issues seem to be solved with disposability, except the oxygen issue which has been solved with the advent of silicone hydrogel materials. Newer issues have arisen and most can be solved in practice by the eye care practitioner. The emphasis now seems to be on making lenses more comfortable. The problems of contact lens related dry eyes symptoms seem to be ever present and maybe this would explain why in the UK we have a pretty constant contact lens wearing population of just over three million but every year we have over a million dropouts! That means we must be attracting a million new wearers every year (well done to the marketing departments!) but we are also losing a million wearers every year. We certainly are not losing them all to the refractive surgery clinics. We know that almost anyone can now wear a contact lens and we know that some lenses will solve problems of sharper vision, some will aid comfort, and some will be useful for patients with dry eyes. So if we still have so many dropouts then we must be doing something wrong! I think the take home message has to be ‘must try harder’! I must end with an apology for two errors in my editorial of issue 1 earlier this year. Firstly there was a typo in the first sentence; I meant to state that it was 40 years not 30 years since the first commercial soft lens was available in the UK. The second error was one that I was unaware of until colleagues Geoff Wilson (Birmingham, UK) and Tim Bowden (London, UK) wrote to me to explain that soft lenses were actually available in the UK before 1971 (please see their ‘Letters to the Editor’ in this issue). I am grateful to both of them for correcting the mistake.

Informational masking and the effects of differences in fundamental frequency and fundamental-frequency contour on phonetic integration in a formant ensemble

This study explored the effects on speech intelligibility of across-formant differences in fundamental frequency (ΔF0) and F0 contour. Sentence-length speech analogues were presented dichotically (left=F1+F3; right=F2), either alone or—because competition usually reveals grouping cues most clearly—accompanied in the left ear by a competitor for F2 (F2C) that listeners must reject to optimize recognition. F2C was created by inverting the F2 frequency contour. In experiment 1, all left-ear formants shared the same constant F0 and ΔF0F2 was 0 or ±4 semitones. In experiment 2, all left-ear formants shared the natural F0 contour and that for F2 was natural, constant, exaggerated, or inverted. Adding F2C lowered keyword scores, presumably because of informational masking. The results for experiment 1 were complicated by effects associated with the direction of ΔF0F2; this problem was avoided in experiment 2 because all four F0 contours had the same geometric mean frequency. When the target formants were presented alone, scores were relatively high and did not depend on the F0F2 contour. F2C impact was greater when F2 had a different F0 contour from the other formants. This effect was a direct consequence of the associated ΔF0; the F0F2 contour per se did not influence competitor impact.