Dynamics and topographic organization of recursive self-organizing maps

Recently there has been an outburst of interest in extending topographic maps of vectorial data to more general data structures, such as sequences or trees. However, there is no general consensus as to how best to process sequences using topographicmaps, and this topic remains an active focus of neurocomputational research. The representational capabilities and internal representations of the models are not well understood. Here, we rigorously analyze a generalization of the self-organizingmap (SOM) for processing sequential data, recursive SOM (RecSOM) (Voegtlin, 2002), as a nonautonomous dynamical system consisting of a set of fixed input maps. We argue that contractive fixed-input maps are likely to produce Markovian organizations of receptive fields on the RecSOM map. We derive bounds on parameter β (weighting the importance of importing past information when processing sequences) under which contractiveness of the fixed-input maps is guaranteed. Some generalizations of SOM contain a dynamic module responsible for processing temporal contexts as an integral part of the model. We show that Markovian topographic maps of sequential data can be produced using a simple fixed (nonadaptable) dynamic module externally feeding a standard topographic model designed to process static vectorial data of fixed dimensionality (e.g., SOM). However, by allowing trainable feedback connections, one can obtain Markovian maps with superior memory depth and topography preservation. We elaborate on the importance of non-Markovian organizations in topographic maps of sequential data. © 2006 Massachusetts Institute of Technology.

Recursive self-organizing map as a contractive iterative function system

Recently, there has been a considerable research activity in extending topographic maps of vectorial data to more general data structures, such as sequences or trees. However, the representational capabilities and internal representations of the models are not well understood. We rigorously analyze a generalization of the Self-Organizing Map (SOM) for processing sequential data, Recursive SOM (RecSOM [1]), as a non-autonomous dynamical system consisting off a set of fixed input maps. We show that contractive fixed input maps are likely to produce Markovian organizations of receptive fields o the RecSOM map. We derive bounds on parameter $\beta$ (weighting the importance of importing past information when processing sequences) under which contractiveness of the fixed input maps is guaranteed.

GTM: A principled alternative to the self-organizing map

The Self-Organizing Map (SOM) algorithm has been extensively studied and has been applied with considerable success to a wide variety of problems. However, the algorithm is derived from heuristic ideas and this leads to a number of significant limitations. In this paper, we consider the problem of modelling the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. We introduce a novel form of latent variable model, which we call the GTM algorithm (for Generative Topographic Mapping), which allows general non-linear transformations from latent space to data space, and which is trained using the EM (expectation-maximization) algorithm. Our approach overcomes the limitations of the SOM, while introducing no significant disadvantages. We demonstrate the performance of the GTM algorithm on simulated data from flow diagnostics for a multi-phase oil pipeline.

Data visualization during the early stages of drug discovery

Multidimensional compound optimization is a new paradigm in the drug discovery process, yielding efficiencies during early stages and reducing attrition in the later stages of drug development. The success of this strategy relies heavily on understanding this multidimensional data and extracting useful information from it. This paper demonstrates how principled visualization algorithms can be used to understand and explore a large data set created in the early stages of drug discovery. The experiments presented are performed on a real-world data set comprising biological activity data and some whole-molecular physicochemical properties. Data visualization is a popular way of presenting complex data in a simpler form. We have applied powerful principled visualization methods, such as generative topographic mapping (GTM) and hierarchical GTM (HGTM), to help the domain experts (screening scientists, chemists, biologists, etc.) understand and draw meaningful decisions. We also benchmark these principled methods against relatively better known visualization approaches, principal component analysis (PCA), Sammon's mapping, and self-organizing maps (SOMs), to demonstrate their enhanced power to help the user visualize the large multidimensional data sets one has to deal with during the early stages of the drug discovery process. The results reported clearly show that the GTM and HGTM algorithms allow the user to cluster active compounds for different targets and understand them better than the benchmarks. An interactive software tool supporting these visualization algorithms was provided to the domain experts. The tool facilitates the domain experts by exploration of the projection obtained from the visualization algorithms providing facilities such as parallel coordinate plots, magnification factors, directional curvatures, and integration with industry standard software. © 2006 American Chemical Society.

Understanding policy implementation processes as self-organizing systems

Implementation studies and related research in organizational theory can be enhanced by drawing on the field of complex systems to understand better and, as a consequence, more successfully manage change. This article reinterprets data previously published in the British Journal of Management to reveal a new contribution, that policy implementation processes should be understood as a self-organizing system in which adaptive abilities are extremely important for stakeholders. In other words, national policy is reinterpreted at the local level, with each local organization uniquely mixing elements of national policy with their own requirements making policy implementation unpredictable and more sketchy. The original article explained different paces and directions of change in terms of traditional management processes: leadership, politics, implementation and vision. By reinterpreting the data, it is possible to reveal that deeper level processes, which are more emergent, are also at work influencing change, which the authors label possibility space. Implications for theory, policy and practice are identified.

Magnification factors for the SOM and GTM algorithms

Magnification factors specify the extent to which the area of a small patch of the latent (or `feature') space of a topographic mapping is magnified on projection to the data space, and are of considerable interest in both neuro-biological and data analysis contexts. Previous attempts to consider magnification factors for the self-organizing map (SOM) algorithm have been hindered because the mapping is only defined at discrete points (given by the reference vectors). In this paper we consider the batch version of SOM, for which a continuous mapping can be defined, as well as the Generative Topographic Mapping (GTM) algorithm of Bishop et al. (1997) which has been introduced as a probabilistic formulation of the SOM. We show how the techniques of differential geometry can be used to determine magnification factors as continuous functions of the latent space coordinates. The results are illustrated here using a problem involving the identification of crab species from morphological data.

Data visualization with simultaneous feature selection

Data visualization algorithms and feature selection techniques are both widely used in bioinformatics but as distinct analytical approaches. Until now there has been no method of measuring feature saliency while training a data visualization model. We derive a generative topographic mapping (GTM) based data visualization approach which estimates feature saliency simultaneously with the training of the visualization model. The approach not only provides a better projection by modeling irrelevant features with a separate noise model but also gives feature saliency values which help the user to assess the significance of each feature. We compare the quality of projection obtained using the new approach with the projections from traditional GTM and self-organizing maps (SOM) algorithms. The results obtained on a synthetic and a real-life chemoinformatics dataset demonstrate that the proposed approach successfully identifies feature significance and provides coherent (compact) projections. © 2006 IEEE.

Self-powered desalination of geothermal saline groundwater:technical feasibility

This theoretical study shows the technical feasibility of self-powered geothermal desalination of groundwater sources at <100 °C. A general method and framework are developed and then applied to specific case studies. First, the analysis considers an ideal limit to performance based on exergy analysis using generalised idealised assumptions. This thermodynamic limit applies to any type of process technology. Then, the analysis focuses specifically on the Organic Rankine Cycle (ORC) driving Reverse Osmosis (RO), as these are among the most mature and efficient applicable technologies. Important dimensionless parameters are calculated for the ideal case of the self-powered arrangement and semi-ideal case where only essential losses dependent on the RO system configuration are considered. These parameters are used to compare the performance of desalination systems using ORC-RO under ideal, semi-ideal and real assumptions for four case studies relating to geothermal sources located in India, Saudi Arabia, Tunisia and Turkey. The overall system recovery ratio (the key performance measure for the self-powered process) depends strongly on the geothermal source temperature. It can be as high as 91.5% for a hot spring emerging at 96 °C with a salinity of 1830 mg/kg.

The power of the powerless:neighbourhood based self-help networks of the poor in Istanbul

Recent poverty research focuses on the household responses to poverty through structure vs. agency perspectives. The human agency perspective, however, provides us important insights for looking beyond these simplistic tendencies which assume poor people as inherently passive, or envision them as helpless victims. In Turkey, politicians view poverty as a temporary and manageable problem which can be dealt with the provision of more charity or community support. Migrant networks, informal sector work and social assistance are considered to be important mechanisms that would provide resources for the poor. This paper argues that for some of the poor households none of these mechanisms provide sufficient resources. Instead, neighbourhood-based small-group solidarities and self-help networks enable those poor to develop collective capabilities and make ends meet. The paper also reveals that in Turkey, the implementation of social policies for poverty reduction could bring about relationships of patronage and in some cases contribute to existing inequalities.

Revisiting goal-oriented models for self-aware systems-of-systems

Systems-of-systems (SoS) are systems resulted from the interaction among other independent constituent systems that collaborate to offer new functionalities towards accomplishing global missions. Each of these constituent systems accomplishes its individual missions and is able to contribute to the achievement of the global missions of the SoS, both being viewed as a set of associated goals. In the perspective of self-aware systems, SoS need to exhibit goal-awareness, i.e., They need to be aware of their own goals and of how their constituent systems contribute to their accomplishment. In this paper, we revisit goal-oriented concepts aiming at identifying and modeling goals at both SoS level and the constituent systems level. Moreover, we take advantage of such goal-oriented models to express the relationship among goals at these levels as well as to define how each constituent system can contribute to the accomplishment of global goals of an SoS. In addition, we shed light on important issues related to goal modeling in self-aware SoS to be addressed in future research.

Discovering knowledge structures in mind maps of mental health risks

This thesis addressed the problem of risk analysis in mental healthcare, with respect to the GRiST project at Aston University. That project provides a risk-screening tool based on the knowledge of 46 experts, captured as mind maps that describe relationships between risks and patterns of behavioural cues. Mind mapping, though, fails to impose control over content, and is not considered to formally represent knowledge. In contrast, this thesis treated GRiSTs mind maps as a rich knowledge base in need of refinement; that process drew on existing techniques for designing databases and knowledge bases. Identifying well-defined mind map concepts, though, was hindered by spelling mistakes, and by ambiguity and lack of coverage in the tools used for researching words. A novel use of the Edit Distance overcame those problems, by assessing similarities between mind map texts, and between spelling mistakes and suggested corrections. That algorithm further identified stems, the shortest text string found in related word-forms. As opposed to existing approaches’ reliance on built-in linguistic knowledge, this thesis devised a novel, more flexible text-based technique. An additional tool, Correspondence Analysis, found patterns in word usage that allowed machines to determine likely intended meanings for ambiguous words. Correspondence Analysis further produced clusters of related concepts, which in turn drove the automatic generation of novel mind maps. Such maps underpinned adjuncts to the mind mapping software used by GRiST; one such new facility generated novel mind maps, to reflect the collected expert knowledge on any specified concept. Mind maps from GRiST are stored as XML, which suggested storing them in an XML database. In fact, the entire approach here is ”XML-centric”, in that all stages rely on XML as far as possible. A XML-based query language allows user to retrieve information from the mind map knowledge base. The approach, it was concluded, will prove valuable to mind mapping in general, and to detecting patterns in any type of digital information.

A survey of self-awareness and its application in computing systems

Novel computing systems are increasingly being composed of large numbers of heterogeneous components, each with potentially different goals or local perspectives, and connected in networks which change over time. Management of such systems quickly becomes infeasible for humans. As such, future computing systems should be able to achieve advanced levels of autonomous behaviour. In this context, the system's ability to be self-aware and be able to self-express becomes important. This paper surveys definitions and current understanding of self-awareness and self-expression in biology and cognitive science. Subsequently, previous efforts to apply these concepts to computing systems are described. This has enabled the development of novel working definitions for self-awareness and self-expression within the context of computing systems.

Self-adaptive probabilistic roadmap generation for intelligent virtual agents

Agents inhabiting large scale environments are faced with the problem of generating maps by which they can navigate. One solution to this problem is to use probabilistic roadmaps which rely on selecting and connecting a set of points that describe the interconnectivity of free space. However, the time required to generate these maps can be prohibitive, and agents do not typically know the environment in advance. In this paper we show that the optimal combination of different point selection methods used to create the map is dependent on the environment, no point selection method dominates. This motivates a novel self-adaptive approach for an agent to combine several point selection methods. The success rate of our approach is comparable to the state of the art and the generation cost is substantially reduced. Self-adaptation therefore enables a more efficient use of the agent's resources. Results are presented for both a set of archetypal scenarios and large scale virtual environments based in Second Life, representing real locations in London.

GTM: the generative topographic mapping

Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis which is based on a linear transformations between the latent space and the data space. In this paper we introduce a form of non-linear latent variable model called the Generative Topographic Mapping, for which the parameters of the model can be determined using the EM algorithm. GTM provides a principled alternative to the widely used Self-Organizing Map (SOM) of Kohonen (1982), and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multi-phase oil pipeline.

GTM: the generative topographic mapping

Latent variable models represent the probability density of data in a space of several dimensions in terms of a smaller number of latent, or hidden, variables. A familiar example is factor analysis which is based on a linear transformations between the latent space and the data space. In this paper we introduce a form of non-linear latent variable model called the Generative Topographic Mapping, for which the parameters of the model can be determined using the EM algorithm. GTM provides a principled alternative to the widely used Self-Organizing Map (SOM) of Kohonen (1982), and overcomes most of the significant limitations of the SOM. We demonstrate the performance of the GTM algorithm on a toy problem and on simulated data from flow diagnostics for a multi-phase oil pipeline.