EM optimization of latent-variable density models

There is currently considerable interest in developing general non-linear density models based on latent, or hidden, variables. Such models have the ability to discover the presence of a relatively small number of underlying `causes' which, acting in combination, give rise to the apparent complexity of the observed data set. Unfortunately, to train such models generally requires large computational effort. In this paper we introduce a novel latent variable algorithm which retains the general non-linear capabilities of previous models but which uses a training procedure based on the EM algorithm. We demonstrate the performance of the model on a toy problem and on data from flow diagnostics for a multi-phase oil pipeline.

A hierarchical latent variable model for data visualization

Visualization has proven to be a powerful and widely-applicable tool the analysis and interpretation of data. Most visualization algorithms aim to find a projection from the data space down to a two-dimensional visualization space. However, for complex data sets living in a high-dimensional space it is unlikely that a single two-dimensional projection can reveal all of the interesting structure. We therefore introduce a hierarchical visualization algorithm which allows the complete data set to be visualized at the top level, with clusters and sub-clusters of data points visualized at deeper levels. The algorithm is based on a hierarchical mixture of latent variable models, whose parameters are estimated using the expectation-maximization algorithm. We demonstrate the principle of the approach first on a toy data set, and then apply the algorithm to the visualization of a synthetic data set in 12 dimensions obtained from a simulation of multi-phase flows in oil pipelines and to data in 36 dimensions derived from satellite images.

Semi-supervised learning of hierarchical latent trait models for data visualisation

An interactive hierarchical Generative Topographic Mapping (HGTM) ¸iteH_GTM has been developed to visualise complex data sets. In this paper, we build a more general visualisation system by extending the HGTM visualisation system in 3 directions: bf (1) We generalize HGTM to noise models from the exponential family of distributions. The basic building block is the Latent Trait Model (LTM) developed in ¸iteKaban_pami. bf (2) We give the user a choice of initializing the child plots of the current plot in either em interactive, or em automatic mode. In the interactive mode the user interactively selects ``regions of interest'' as in ¸iteH_GTM, whereas in the automatic mode an unsupervised minimum message length (MML)-driven construction of a mixture of LTMs is employed. bf (3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualisation plots, since they can highlight the boundaries between data clusters. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. We illustrate our approach on a toy example and apply our system to three more complex real data sets.

Do changes in transglutaminase activity alter latent transforming growth factor beta activation in experimental diabetic nephropathy?

Background. Diabetic nephropathy is the leading cause of end-stage kidney failure worldwide. It is characterized by excessive extracellular matrix accumulation. Transforming growth factor beta 1 (TGF-ß1) is a fibrogenic cytokine playing a major role in the healing process and scarring by regulating extracellular matrix turnover, cell proliferation and epithelial mesanchymal transdifferentiation. Newly synthesized TGF-ß is released as a latent, biologically inactive complex. The cross-linking of the large latent TGF-ß to the extracellular matrix by transglutaminase 2 (TG2) is one of the key mechanisms of recruitment and activation of this cytokine. TG2 is an enzyme catalyzing an acyl transfer reaction leading to the formation of a stable e(?-glutamyl)-lysine cross-link between peptides.Methods. To investigate if changes in TG activity can modulate TGF-ß1 activation, we used the mink lung cell bioassay to assess TGF-ß activity in the streptozotocin model of diabetic nephropathy treated with TG inhibitor NTU281 and in TG2 overexpressing opossum kidney (OK) proximal tubular epithelial cells.Results. Application of the site-directed TG inhibitor NTU281 caused a 25% reduction in kidney levels of active TGF-ß1. Specific upregulation of TG2 in OK proximal tubular epithelial cells increased latent TGF-ß recruitment and activation by 20.7% and 19.7%, respectively, in co-cultures with latent TGF-ß binding protein producing fibroblasts.Conclusions. Regulation of TG2 directly influences the level of active TGF-ß1, and thus, TG inhibition may exert a renoprotective effect by targeting not only a direct extracellular matrix deposition but also TGF-ß1 activation and recruitment.

Current practice and opinion in special schools for the maladjusted in England and Wales

This study seeks to describe current practice and opinion in schools for the maladjusted in England and Wales and to exarnlne how far this coincides with earlier descriptions. A review of the literature provides an account of this earlier work, and data accrued from questionnaires completed by 114 schools describe current practice and opinion. The study represents the most extensive empirical enquiry into the work of these schools since 1955 and provides a wide data basis for future research and assessment of progress and change. The data suggest that there is much communality of practice and opinion within the schools, with most schools emphasising their therapeutic rather than their educational purpose. The work is characterised by the wide use and perceived efficacy of warm, caring adult to child relationships, improvement of pupil self-image through success, and individual counselling and discussion, which permeate a structure of routine, discipline and educational concern. Specialised treatments are not used widely and involve only a minority of pupils. Practice tends to be in reference to conduct disordered pupils who are now perceived as the largest single disorder group within the schools, whereas previously neurotic disorders formed the largest single group. The majority of pupils are perceived as underachieving on entry and requiring remedial help: consequently the educational programme has a remedial bias. For staff, qualities of personality are considered to be more valuable than professional skills. The schools differ in the emphasis they allocate to one or more of four identified areas of treatment described as concern for pupils' needs; degree of pupil participation; theoretical orientation: and the use of external controls. There is a diminished reference to psychoanalytical theory and an increased reference to behaviourist theory relative to previous practice. Similarly, the use and perceived importance and effectiveness of pupil participation and unconditional affection has diminished.

Public opinion and the making of foreign policy in the 'New Europe':a comparative study of Poland and Ukraine

Since 1989, by drawing a new boundary between the EU and its eastern neighbours, the European Union has created a frontier that has been popularly described in the frontier states as the new 'Berlin Wall'. This book is the first comparative study of the impact of public opinion on the making of foreign policy in two eastern European states that live on either side of the new European divide: Poland and Ukraine. Focusing on the vocal, informed segment of public opinion and drawing on results of both opinion polls and a series of innovative focus groups gathered since the Orange Revolution, Nathaniel Copsey unravels the mystery of how this crucial segment of the public impacts on foreign policy-makers in both states. In developing this argument, Copsey takes a closer look at the business community and how important economic factors are in forming public opinion. Filling a gap in the literature currently available on the topic, this book presents a fresh approach to our understanding of Polish-Ukrainian relations and how the public's view of the past influences contemporary politics. It is an ideal resource for those researching in the field of Russian and Eastern European Studies.

A Bayesian modeling approach to multi-dimensional sentiment distributions prediction

Sentiment analysis has long focused on binary classification of text as either positive or negative. There has been few work on mapping sentiments or emotions into multiple dimensions. This paper studies a Bayesian modeling approach to multi-class sentiment classification and multidimensional sentiment distributions prediction. It proposes effective mechanisms to incorporate supervised information such as labeled feature constraints and document-level sentiment distributions derived from the training data into model learning. We have evaluated our approach on the datasets collected from the confession section of the Experience Project website where people share their life experiences and personal stories. Our results show that using the latent representation of the training documents derived from our approach as features to build a maximum entropy classifier outperforms other approaches on multi-class sentiment classification. In the more difficult task of multi-dimensional sentiment distributions prediction, our approach gives superior performance compared to a few competitive baselines. © 2012 ACM.

Weakly supervised joint sentiment-topic detection from text

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework called joint sentiment-topic (JST) model based on latent Dirichlet allocation (LDA), which detects sentiment and topic simultaneously from text. A reparameterized version of the JST model called Reverse-JST, obtained by reversing the sequence of sentiment and topic generation in the modeling process, is also studied. Although JST is equivalent to Reverse-JST without a hierarchical prior, extensive experiments show that when sentiment priors are added, JST performs consistently better than Reverse-JST. Besides, unlike supervised approaches to sentiment classification which often fail to produce satisfactory performance when shifting to other domains, the weakly supervised nature of JST makes it highly portable to other domains. This is verified by the experimental results on data sets from five different domains where the JST model even outperforms existing semi-supervised approaches in some of the data sets despite using no labeled documents. Moreover, the topics and topic sentiment detected by JST are indeed coherent and informative. We hypothesize that the JST model can readily meet the demand of large-scale sentiment analysis from the web in an open-ended fashion.

Overcoming network overload and redundancy in inter-organizational networks:the roles of potential and latent ties

This paper builds on Granovetter's distinction between strong and weak ties [Granovetter, M. S. 1973. The strength of weak ties. Amer. J. Sociol. 78(6) 1360–1380] in order to respond to recent calls for a more dynamic and processual understanding of networks. The concepts of potential and latent tie are deductively identified, and their implications for understanding how and why networks emerge, evolve, and change are explored. A longitudinal empirical study conducted with companies operating in the European motorsport industry reveals that firms take strategic actions to search for potential ties and reactivate latent ties in order to solve problems of network redundancy and overload. Examples are given, and their characteristics are examined to provide theoretical elaboration of the relationship between the types of tie and network evolution. These conceptual and empirical insights move understanding of the managerial challenge of building effective networks beyond static structural contingency models of optimal network forms to highlight the processes and capabilities of dynamic relationship building and network development. In so doing, this paper highlights the interrelationship between search and redundancy and the scope for strategic action alongside path dependence and structural influences on network processes.

Opinion leaders' role in innovation diffusion:a simulation study

This study investigates the critical role that opinion leaders (or influentials) play in the adoption process of new products. Recent existing reseach evidence indicates a limited effect of opinion leaders on diffusion processes, yet these studies take into account merely the network position of opinion leaders without addressing their influential power. Empirical findings of our study show that opinion leaders, in addition to having a more central network position, possess more accurate knowledge about a product and tend to be less susceptible to norms and more innovative. Experiments that address these attributes, using an agent-based model, demonstrate that opinion leaders increase the speed of the information stream and the adoption process itself. Furthermore, they increase the maximum adoption percentage. These results indicate that targeting opinion leaders remains a valuable marketing strategy.

Joint sentiment/topic model for sentiment analysis

Sentiment analysis or opinion mining aims to use automated tools to detect subjective information such as opinions, attitudes, and feelings expressed in text. This paper proposes a novel probabilistic modeling framework based on Latent Dirichlet Allocation (LDA), called joint sentiment/topic model (JST), which detects sentiment and topic simultaneously from text. Unlike other machine learning approaches to sentiment classification which often require labeled corpora for classifier training, the proposed JST model is fully unsupervised. The model has been evaluated on the movie review dataset to classify the review sentiment polarity and minimum prior information have also been explored to further improve the sentiment classification accuracy. Preliminary experiments have shown promising results achieved by JST.

LRD : Latent Relation Discovery for vector space expansion and information retrieval

In this paper, we propose a text mining method called LRD (latent relation discovery), which extends the traditional vector space model of document representation in order to improve information retrieval (IR) on documents and document clustering. Our LRD method extracts terms and entities, such as person, organization, or project names, and discovers relationships between them by taking into account their co-occurrence in textual corpora. Given a target entity, LRD discovers other entities closely related to the target effectively and efficiently. With respect to such relatedness, a measure of relation strength between entities is defined. LRD uses relation strength to enhance the vector space model, and uses the enhanced vector space model for query based IR on documents and clustering documents in order to discover complex relationships among terms and entities. Our experiments on a standard dataset for query based IR shows that our LRD method performed significantly better than traditional vector space model and other five standard statistical methods for vector expansion.

Multi-level visualisation using Gaussian process latent variable models

Projection of a high-dimensional dataset onto a two-dimensional space is a useful tool to visualise structures and relationships in the dataset. However, a single two-dimensional visualisation may not display all the intrinsic structure. Therefore, hierarchical/multi-level visualisation methods have been used to extract more detailed understanding of the data. Here we propose a multi-level Gaussian process latent variable model (MLGPLVM). MLGPLVM works by segmenting data (with e.g. K-means, Gaussian mixture model or interactive clustering) in the visualisation space and then fitting a visualisation model to each subset. To measure the quality of multi-level visualisation (with respect to parent and child models), metrics such as trustworthiness, continuity, mean relative rank errors, visualisation distance distortion and the negative log-likelihood per point are used. We evaluate the MLGPLVM approach on the ‘Oil Flow’ dataset and a dataset of protein electrostatic potentials for the ‘Major Histocompatibility Complex (MHC) class I’ of humans. In both cases, visual observation and the quantitative quality measures have shown better visualisation at lower levels.