The Gaussian process density sampler

We present the Gaussian Process Density Sampler (GPDS), an exchangeable generative model for use in nonparametric Bayesian density estimation. Samples drawn from the GPDS are consistent with exact, independent samples from a fixed density function that is a transformation of a function drawn from a Gaussian process prior. Our formulation allows us to infer an unknown density from data using Markov chain Monte Carlo, which gives samples from the posterior distribution over density functions and from the predictive distribution on data space. We can also infer the hyperparameters of the Gaussian process. We compare this density modeling technique to several existing techniques on a toy problem and a skullreconstruction task.

Derivative kernels for noise robust ASR

Recently there has been interest in combining generative and discriminative classifiers. In these classifiers features for the discriminative models are derived from the generative kernels. One advantage of using generative kernels is that systematic approaches exist to introduce complex dependencies into the feature-space. Furthermore, as the features are based on generative models standard model-based compensation and adaptation techniques can be applied to make discriminative models robust to noise and speaker conditions. This paper extends previous work in this framework in several directions. First, it introduces derivative kernels based on context-dependent generative models. Second, it describes how derivative kernels can be incorporated in structured discriminative models. Third, it addresses the issues associated with large number of classes and parameters when context-dependent models and high-dimensional feature-spaces of derivative kernels are used. The approach is evaluated on two noise-corrupted tasks: small vocabulary AURORA 2 and medium-to-large vocabulary AURORA 4 task. © 2011 IEEE.

Silhouette-based object phenotype recognition using 3D shape priors

This paper tackles the novel challenging problem of 3D object phenotype recognition from a single 2D silhouette. To bridge the large pose (articulation or deformation) and camera viewpoint changes between the gallery images and query image, we propose a novel probabilistic inference algorithm based on 3D shape priors. Our approach combines both generative and discriminative learning. We use latent probabilistic generative models to capture 3D shape and pose variations from a set of 3D mesh models. Based on these 3D shape priors, we generate a large number of projections for different phenotype classes, poses, and camera viewpoints, and implement Random Forests to efficiently solve the shape and pose inference problems. By model selection in terms of the silhouette coherency between the query and the projections of 3D shapes synthesized using the galleries, we achieve the phenotype recognition result as well as a fast approximate 3D reconstruction of the query. To verify the efficacy of the proposed approach, we present new datasets which contain over 500 images of various human and shark phenotypes and motions. The experimental results clearly show the benefits of using the 3D priors in the proposed method over previous 2D-based methods. © 2011 IEEE.

A Structured Model of Video Reproduces Primary Visual Cortical Organisation

The visual system must learn to infer the presence of objects and features in the world from the images it encounters, and as such it must, either implicitly or explicitly, model the way these elements interact to create the image. Do the response properties of cells in the mammalian visual system reflect this constraint? To address this question, we constructed a probabilistic model in which the identity and attributes of simple visual elements were represented explicitly and learnt the parameters of this model from unparsed, natural video sequences. After learning, the behaviour and grouping of variables in the probabilistic model corresponded closely to functional and anatomical properties of simple and complex cells in the primary visual cortex (V1). In particular, feature identity variables were activated in a way that resembled the activity of complex cells, while feature attribute variables responded much like simple cells. Furthermore, the grouping of the attributes within the model closely parallelled the reported anatomical grouping of simple cells in cat V1. Thus, this generative model makes explicit an interpretation of complex and simple cells as elements in the segmentation of a visual scene into basic independent features, along with a parametrisation of their moment-by-moment appearances. We speculate that such a segmentation may form the initial stage of a hierarchical system that progressively separates the identity and appearance of more articulated visual elements, culminating in view-invariant object recognition.

Occlusive Components Analysis

We study unsupervised learning in a probabilistic generative model for occlusion. The model uses two types of latent variables: one indicates which objects are present in the image, and the other how they are ordered in depth. This depth order then determines how the positions and appearances of the objects present, specified in the model parameters, combine to form the image. We show that the object parameters can be learnt from an unlabelled set of images in which objects occlude one another. Exact maximum-likelihood learning is intractable. However, we show that tractable approximations to Expectation Maximization (EM) can be found if the training images each contain only a small number of objects on average. In numerical experiments it is shown that these approximations recover the correct set of object parameters. Experiments on a novel version of the bars test using colored bars, and experiments on more realistic data, show that the algorithm performs well in extracting the generating causes. Experiments based on the standard bars benchmark test for object learning show that the algorithm performs well in comparison to other recent component extraction approaches. The model and the learning algorithm thus connect research on occlusion with the research field of multiple-causes component extraction methods.

on string languages generated by spiking neural p systems

We continue the study of spiking neural P systems by considering these computing devices as binary string generators: the set of spike trains of halting computations of a given system constitutes the language generated by that system. Although the "direct" generative capacity of spiking neural P systems is rather restricted (some very simple languages cannot be generated in this framework), regular languages are inverse-morphic images of languages of finite spiking neural P systems, and recursively enumerable languages are projections of inverse-morphic images of languages generated by spiking neural P systems.

A Review of Active Appearance Models

Active appearance model (AAM) is a powerful generative method for modeling deformable objects. The model decouples the shape and the texture variations of objects, which is followed by an efficient gradient-based model fitting method. Due to the flexible and simple framework, AAM has been widely applied in the fields of computer vision. However, difficulties are met when it is applied to various practical issues, which lead to a lot of prominent improvements to the model. Nevertheless, these difficulties and improvements have not been studied systematically. This motivates us to review the recent advances of AAM. This paper focuses on the improvements in the literature in turns of the problems suffered by AAM in practical applications. Therefore, these algorithms are summarized from three aspects, i.e., efficiency, discrimination, and robustness. Additionally, some applications and implementations of AAM are also enumerated. The main purpose of this paper is to serve as a guide for further research.

微动机器人的致动机理研究综述

MEMS是当前研究的一个热点,微机器人对于发展MEMS具有重要意义,也是MEMS的一项不可缺少的内容。微动技术是机器人学理论的一个重要分支,也是发展微机器人及相关微技术的基础。目前,各种新型微驱动器层出不穷,极大地推动了微机器人技术的发展。对于微动原理进行分析,从本质上弄清微动产生的机理,不仅可以丰富机器人学理论,还有可能使微动技术产生质的飞跃。从这一角度出发,对各种微动原理加以详细分析和比较,以期得出有意义的结论。

学习不良少年的自我意识情绪研究

Self-conscious emotions (guilt, shame, embarrassment, pride, etc) are social emotions, and involve complex appraisals of how one’s behavior has been evaluated by the self and other people according to some value standards. Self-conscious emotions play an important role in human life by arousing and regulating human action tendencies, feeling and thoughts, which can promote people to work hard in achievement and task fields, maintain good interpersonal relationship according with social morality and expectation. The present study aimed to examine complex self-conscious emotional understanding capabilities in junior middle school students with and without learning disabilities, how the self-conscious emotions generate, and relationship between self-conscious emotions and self-representation in academic and interpersonal fields. Situational experimental methods were used in this research, and the results would give further supports for learning disabilities intervention. The main results of present research are as follows. 1. The study included 4 parts and 6 experiments. The aim of study 1 was to explore whether juveniles with learning disabilities understood complex self-conscious emotions differently from juveniles without learning disabilities. We surveyed the self-conscious emotions understanding of 37 learning disabilities and 45 non-learning disabilities with the emotional situation stories. The results indicated that the self-conscious emotional recognition in others for learning disabilities was lower than that of non-learning disabilities in different emotional recognition tasks. Moreover, children with learning disabilities were more inclined to recognize emotions in themselves as elemental emotions, however, children without learning disabilities were more inclined to recognize emotions in themselves as self-conscious emotions. 2. The aim of study 2 was to explore the generative mechanism of self-conscious emotions in academic and interpersonal fields with the method of situational experiments, namely to examine whether the self-discrepancy could cause self-conscious emotions for learning disabilities. 84 learning disabilities (in experiment 1) and 80 learning disabilities (in experiment 2) participated in the research, and the results were as follows. (1) Self discrepancy caused participants’ self-conscious emotions effectively in academic and interpersonal fields. One’s own and parents’ perspercive on the actual-ideal self-discrepancy both produced dejection-related emotions (shame、embarrassment) and agitation-related emotions (guilt). （2）In academic fields, children with learning disabilities caused higher level negative self-conscious emotions (embarrassment, shame, and guilt) and lower level positive self-conscious emotion (pride). However, there were no differences of self-conscious emotions for children with and without learning disabilities in non-academic fields. 3. The aim of study 3 was to explore what influence had self-conscious emotions on self-representation for learning disabilities with the method of situational experiments. 57 learning disabilities (in experiment 1) and 67 learning disabilities (in experiment 2) participated in the research, and the results were as follows. （1）The negative self-conscious for learning disabilities could influence their positive or negative academic and positive interpersonal self-representation stability, the ways in which self-evaluation of ability mediate these effects. However, there was no significant effect for the negative self-conscious and self-evaluation of ability predicting negative interpersonal self-representation stability. （2）The stability level of positive academic and interpersonal self-representation for learning disabilities was lower than that of non-learning disabilities. There was no significant difference of the negative interpersonal self-representation stability for children with and without learning disabilities in the positive self-conscious valence condition. However, the stability level of negative interpersonal self-representation for learning disabilities was lower than that of non-learning disabilities in the negative self-conscious valence condition. 4. The aim of study 4 was to explore the intervention effects for self-conscious emotions training course on emotional comprehension cability. 65 learning disabilities (34 in experimental group, and 31 in control group) participated in the research. The results showed that self-conscious emotions course boosted the self-conscious emotions apprehensive level for children with learning disabilities.

Permutation Tests for Classification

We introduce and explore an approach to estimating statistical significance of classification accuracy, which is particularly useful in scientific applications of machine learning where high dimensionality of the data and the small number of training examples render most standard convergence bounds too loose to yield a meaningful guarantee of the generalization ability of the classifier. Instead, we estimate statistical significance of the observed classification accuracy, or the likelihood of observing such accuracy by chance due to spurious correlations of the high-dimensional data patterns with the class labels in the given training set. We adopt permutation testing, a non-parametric technique previously developed in classical statistics for hypothesis testing in the generative setting (i.e., comparing two probability distributions). We demonstrate the method on real examples from neuroimaging studies and DNA microarray analysis and suggest a theoretical analysis of the procedure that relates the asymptotic behavior of the test to the existing convergence bounds.

A Unified Statistical and Information Theoretic Framework for Multi-modal Image Registration

We formulate and interpret several multi-modal registration methods in the context of a unified statistical and information theoretic framework. A unified interpretation clarifies the implicit assumptions of each method yielding a better understanding of their relative strengths and weaknesses. Additionally, we discuss a generative statistical model from which we derive a novel analysis tool, the "auto-information function", as a means of assessing and exploiting the common spatial dependencies inherent in multi-modal imagery. We analytically derive useful properties of the "auto-information" as well as verify them empirically on multi-modal imagery. Among the useful aspects of the "auto-information function" is that it can be computed from imaging modalities independently and it allows one to decompose the search space of registration problems.

Generating Representative ISP Technologies From First-Principles

Understanding and modeling the factors that underlie the growth and evolution of network topologies are basic questions that impact capacity planning, forecasting, and protocol research. Early topology generation work focused on generating network-wide connectivity maps, either at the AS-level or the router-level, typically with an eye towards reproducing abstract properties of observed topologies. But recently, advocates of an alternative "first-principles" approach question the feasibility of realizing representative topologies with simple generative models that do not explicitly incorporate real-world constraints, such as the relative costs of router configurations, into the model. Our work synthesizes these two lines by designing a topology generation mechanism that incorporates first-principles constraints. Our goal is more modest than that of constructing an Internet-wide topology: we aim to generate representative topologies for single ISPs. However, our methods also go well beyond previous work, as we annotate these topologies with representative capacity and latency information. Taking only demand for network services over a given region as input, we propose a natural cost model for building and interconnecting PoPs and formulate the resulting optimization problem faced by an ISP. We devise hill-climbing heuristics for this problem and demonstrate that the solutions we obtain are quantitatively similar to those in measured router-level ISP topologies, with respect to both topological properties and fault-tolerance.

An Information Theoretic Framework for Field Monitoring Using Autonomously Mobile Sensors

We consider a mobile sensor network monitoring a spatio-temporal field. Given limited cache sizes at the sensor nodes, the goal is to develop a distributed cache management algorithm to efficiently answer queries with a known probability distribution over the spatial dimension. First, we propose a novel distributed information theoretic approach in which the nodes locally update their caches based on full knowledge of the space-time distribution of the monitored phenomenon. At each time instant, local decisions are made at the mobile nodes concerning which samples to keep and whether or not a new sample should be acquired at the current location. These decisions account for minimizing an entropic utility function that captures the average amount of uncertainty in queries given the probability distribution of query locations. Second, we propose a different correlation-based technique, which only requires knowledge of the second-order statistics, thus relaxing the stringent constraint of having a priori knowledge of the query distribution, while significantly reducing the computational overhead. It is shown that the proposed approaches considerably improve the average field estimation error by maintaining efficient cache content. It is further shown that the correlation-based technique is robust to model mismatch in case of imperfect knowledge of the underlying generative correlation structure.

Modeling the evolution of regulatory elements by simultaneous detection and alignment with phylogenetic pair HMMs.

The computational detection of regulatory elements in DNA is a difficult but important problem impacting our progress in understanding the complex nature of eukaryotic gene regulation. Attempts to utilize cross-species conservation for this task have been hampered both by evolutionary changes of functional sites and poor performance of general-purpose alignment programs when applied to non-coding sequence. We describe a new and flexible framework for modeling binding site evolution in multiple related genomes, based on phylogenetic pair hidden Markov models which explicitly model the gain and loss of binding sites along a phylogeny. We demonstrate the value of this framework for both the alignment of regulatory regions and the inference of precise binding-site locations within those regions. As the underlying formalism is a stochastic, generative model, it can also be used to simulate the evolution of regulatory elements. Our implementation is scalable in terms of numbers of species and sequence lengths and can produce alignments and binding-site predictions with accuracy rivaling or exceeding current systems that specialize in only alignment or only binding-site prediction. We demonstrate the validity and power of various model components on extensive simulations of realistic sequence data and apply a specific model to study Drosophila enhancers in as many as ten related genomes and in the presence of gain and loss of binding sites. Different models and modeling assumptions can be easily specified, thus providing an invaluable tool for the exploration of biological hypotheses that can drive improvements in our understanding of the mechanisms and evolution of gene regulation.

Minor Moves: Growth, Fugitivity, and Children's Physical Movement

From tendencies to reduce the Underground Railroad to the imperative "follow the north star" to the iconic images of Ruby Bridges' 1960 "step forward" on the stairs of William Frantz Elementary School, America prefers to picture freedom as an upwardly mobile development. This preoccupation with the subtractive and linear force of development makes it hard to hear the palpable steps of so many truant children marching in the Movement and renders illegible the nonlinear movements of minors in the Underground. Yet a black fugitive hugging a tree, a white boy walking alone in a field, or even pieces of a discarded raft floating downstream like remnants of child's play are constitutive gestures of the Underground's networks of care and escape. Responding to 19th-century Americanists and cultural studies scholars' important illumination of the child as central to national narratives of development and freedom, "Minor Moves" reads major literary narratives not for the child and development but for the fugitive trace of minor and growth.

In four chapters, I trace the physical gestures of Nathaniel Hawthorne's Pearl, Harriet Beecher Stowe's Topsy, Harriet Wilson's Frado, and Mark Twain's Huck against the historical backdrop of the Fugitive Slave Act and the passing of the first compulsory education bills that made truancy illegal. I ask how, within a discourse of independence that fails to imagine any serious movements in the minor, we might understand the depictions of moving children as interrupting a U.S. preoccupation with normative development and recognize in them the emergence of an alternative imaginary. To attend to the movement of the minor is to attend to what the discursive order of a development-centered imaginary deems inconsequential and what its grammar can render only as mistakes. Engaging the insights of performance studies, I regard what these narratives depict as childish missteps (Topsy's spins, Frado's climbing the roof) as dances that trouble the narrative's discursive order. At the same time, drawing upon the observations of black studies and literary theory, I take note of the pressure these "minor moves" put on the literal grammar of the text (Stowe's run-on sentences and Hawthorne's shaky subject-verb agreements). I regard these ungrammatical moves as poetic ruptures from which emerges an alternative and prior force of the imaginary at work in these narratives--a force I call "growth."

Reading these "minor moves" holds open the possibility of thinking about a generative association between blackness and childishness, one that neither supports racist ideas of biological inferiority nor mandates in the name of political uplift the subsequent repudiation of childishness. I argue that recognizing the fugitive force of growth indicated in the interplay between the conceptual and grammatical disjunctures of these minor moves opens a deeper understanding of agency and dependency that exceeds notions of arrested development and social death. For once we interrupt the desire to picture development (which is to say the desire to picture), dependency is no longer a state (of social death or arrested development) of what does not belong, but rather it is what Édouard Glissant might have called a "departure" (from "be[ing] a single being"). Topsy's hard-to-see pick-pocketing and Pearl's running amok with brown men in the market are not moves out of dependency but indeed social turns (a dance) by way of dependency. Dependent, moving and ungrammatical, the growth evidenced in these childish ruptures enables different stories about slavery, freedom, and childishness--ones that do not necessitate a repudiation of childishness in the name of freedom, but recognize in such minor moves a fugitive way out.