Semi-supervised Gaussian process latent variable model with pairwise constraints

In machine learning, Gaussian process latent variable model (GP-LVM) has been extensively applied in the field of unsupervised dimensionality reduction. When some supervised information, e.g., pairwise constraints or labels of the data, is available, the traditional GP-LVM cannot directly utilize such supervised information to improve the performance of dimensionality reduction. In this case, it is necessary to modify the traditional GP-LVM to make it capable of handing the supervised or semi-supervised learning tasks. For this purpose, we propose a new semi-supervised GP-LVM framework under the pairwise constraints. Through transferring the pairwise constraints in the observed space to the latent space, the constrained priori information on the latent variables can be obtained. Under this constrained priori, the latent variables are optimized by the maximum a posteriori (MAP) algorithm. The effectiveness of the proposed algorithm is demonstrated with experiments on a variety of data sets. © 2010 Elsevier B.V.

A novel latent factor model for recommender system

Matrix factorization (MF) has evolved as one of the better practice to handle sparse data in field of recommender systems. Funk singular value decomposition (SVD) is a variant of MF that exists as state-of-the-art method that enabled winning the Netflix prize competition. The method is widely used with modifications in present day research in field of recommender systems. With the potential of data points to grow at very high velocity, it is prudent to devise newer methods that can handle such data accurately as well as efficiently than Funk-SVD in the context of recommender system. In view of the growing data points, I propose a latent factor model that caters to both accuracy and efficiency by reducing the number of latent features of either users or items making it less complex than Funk-SVD, where latent features of both users and items are equal and often larger. A comprehensive empirical evaluation of accuracy on two publicly available, amazon and ml-100 k datasets reveals the comparable accuracy and lesser complexity of proposed methods than Funk-SVD.

Transient prenatal vitamin D deficiency is associated with subtle alterations in learning and memory functions in adult rats

Based on clues from epidemiology, low prenatal vitamin D has been proposed as a candidate risk factor for schizophrenia. Recent animal experiments have demonstrated that transient prenatal vitamin D deficiency is associated with persistent alterations in brain morphology and neurotrophin expression. In order to explore the utility of the vitamin D animal model of schizophrenia, we examined different types of learning and memory in adult rats exposed to transient prenatal vitamin D deficiency. Compared to control animals, the prenatally deplete animals had a significant impairment of latent inhibition, a feature often associated with schizophrenia. In addition, the deplete group was (a) significantly impaired on hole board habituation and (b) significantly better at maintaining previously learnt rules of brightness discrimination in a Y-chamber. In contrast, the prenatally deplete animals showed no impairment on the spatial learning task in the radial maze, nor on two-way active avoidance learning in the shuttle-box. The results indicate that transient prenatal vitamin D depletion in the rat is associated with subtle and discrete alterations in learning and memory. The behavioural phenotype associated with this animal model may provide insights into the neurobiological correlates of the cognitive impairments of schizophrenia. (c) 2005 Elsevier B.V. All rights reserved.

Overshadowing in conditioned taste aversion or in conditioned emotional response after neonatal ventral hippocampal lesions in rats

The neonatal hippocampus lesion thought to model schizophrenia should show the same modifications in behavioural tests as other models, especially pharmacological models. namely decreased latent inhibition, blocking and overshadowing. The present study is set out to evaluate overshadowing in order to complement our previous studies, which had tested latent inhibition. ""Overshadowing"" refers to the decreased conditioning that occurs when the to-be-conditioned stimulus is combined with another stimulus at the conditioning stage. We used the same two Pavlovian conditioning paradigms as in our previous works, namely conditioned taste aversion (CTA) and conditioned emotional response (CER). A sweet taste overshadowed a salty conditioned stimulus, and a tone overshadowed a flashing light. Totally different stimuli were used to counter possible sensory biases. The protocols were validated with two groups of Sprague Dawley rats. The same two protocols were then applied to a cohort of rats whose ventral hippocampus had been destroyed when they were 7 days old. Only rats with extended ventral hippocampus lesions were included. The overall effect of Pavlovian conditioning was attenuated, significantly so in the conditioned emotional response paradigm, but overshadowing appeared not to be modified in either the conditioned emotional response or the conditioned taste aversion paradigm. (C) 2008 Elsevier B.V. All rights reserved.

Rewarded associative and instrumental conditioning after neonatal ventral hippocampus lesions in rats

Sprague Dawley rats were submitted to bilateral ventral hippocampus lesions 7 days after birth. This corresponds to the Lipska and Weinberger`s procedure for modeling schizophrenia. The aim of the present work was to test the learning capacity of such rats with an associative Pavlovian and an instrumental learning paradigm, both methods using reward outcome (food, sucrose or polycose). The associative paradigm comprised also a second learning test with reversed learning contingencies. The instrumental conditioning comprised an extinction test under outcome devaluation conditions. Neonatally lesioned rats, once adults (over 60 days of age), showed a conditioning deficit in the associative paradigm but not in the instrumental one. Lesioned rats remained able to adapt as readily as controls to the reversed learning contingency and were as sensitive as controls to the devaluation of outcome. Such observations indicate that the active access (instrumental learning) to a reward could have compensated for the deficit observed under the ""passive"" stimulus-reward associative learning condition. This feature is compared to the memory management impairments observed in clinical patients. (c) 2008 Elsevier B.V. All rights reserved.

Life-Cycle Effects on Household Expenditures: A Latent-Variable Approach

Using data from the Spanish household budget survey, we investigate life- cycle effects on several product expenditures. A latent-variable model approach is adopted to evaluate the impact of income on expenditures, controlling for the number of members in the family. Two latent factors underlying repeated measures of monetary and non-monetary income are used as explanatory variables in the expenditure regression equations, thus avoiding possible bias associated to the measurement error in income. The proposed methodology also takes care of the case in which product expenditures exhibit a pattern of infrequent purchases. Multiple-group analysis is used to assess the variation of key parameters of the model across various household life-cycle typologies. The analysis discloses significant life-cycle effects on the mean levels of expenditures; it also detects significant life-cycle effects on the way expenditures are affected by income and family size. Asymptotic robust methods are used to account for possible non-normality of the data.

Behavioral phenotyping of glutathione-deficient mice: Relevance to schizophrenia and bipolar disorder.

Redox-dysregulation represents a common pathogenic mechanism in schizophrenia (SZ) and bipolar disorder (BP). It may in part arise from a genetically compromised synthesis of glutathione (GSH), the major cellular antioxidant and redox-regulator. Allelic variants of the genes coding for the rate-limiting GSH synthesizing enzyme glutamate-cysteine-ligase modifier (GCLM) and/or catalytic (GCLC) subunit have been associated with SZ and BP. Using mice knockout (KO) for GCLM we have previously shown that impaired GSH synthesis is associated with morphological, functional and neurochemical anomalies similar to those in patients. Here we asked whether GSH deficit is also associated with SZ- and BP-relevant behavioral and cognitive anomalies. Accordingly, we subjected young adult GCLM-wildtype (WT), heterozygous and KO males to a battery of standard tests. Compared to WT, GCLM-KO mice displayed hyperlocomotion in the open field and forced swim test but normal activity in the home cage, suggesting that hyperlocomotion was selective to environmental novelty and mildly stressful situations. While spatial working memory and latent inhibition remained unaffected, KO mice showed a potentiated hyperlocomotor response to an acute amphetamine injection, impaired sensorymotor gating in the form of prepulse inhibition and altered social behavior compared to WT. These anomalies resemble important aspects of both SZ and the manic component of BP. As such our data support the notion that redox-dysregulation due to GSH deficit is implicated in both disorders. Moreover, our data propose the GCLM-KO mouse as a valuable model to study the behavioral and cognitive consequences of redox dysregulation in the context of psychiatric disease.

Lyfe-cycle effects on household expenditures: A latent-variable approach

Using data from the Spanish household budget survey, we investigate life-cycle effects on several product expenditures. A latent-variable model approach is adopted to evaluate the impact of income on expenditures, controlling for the number of members in the family. Two latent factors underlying repeated measures of monetary and non-monetary income are used as explanatory variables in the expenditure regression equations, thus avoiding possible bias associated to the measurement error in income. The proposed methodology also takes care of the case in which product expenditures exhibit a pattern of infrequent purchases. Multiple-group analysis is used to assess the variation of key parameters of the model across various household life-cycle typologies. The analysis discloses significant life-cycle effects on the mean levels of expenditures; it also detects significant life-cycle effects on the way expenditures are affected by income and family size. Asymptotic robust methods are used to account for possible non-normality of the data.

Bayesian parameter estimation for latent Markov random fields and social networks

Undirected graphical models are widely used in statistics, physics and machine vision. However Bayesian parameter estimation for undirected models is extremely challenging, since evaluation of the posterior typically involves the calculation of an intractable normalising constant. This problem has received much attention, but very little of this has focussed on the important practical case where the data consists of noisy or incomplete observations of the underlying hidden structure. This paper specifically addresses this problem, comparing two alternative methodologies. In the first of these approaches particle Markov chain Monte Carlo (Andrieu et al., 2010) is used to efficiently explore the parameter space, combined with the exchange algorithm (Murray et al., 2006) for avoiding the calculation of the intractable normalising constant (a proof showing that this combination targets the correct distribution in found in a supplementary appendix online). This approach is compared with approximate Bayesian computation (Pritchard et al., 1999). Applications to estimating the parameters of Ising models and exponential random graphs from noisy data are presented. Each algorithm used in the paper targets an approximation to the true posterior due to the use of MCMC to simulate from the latent graphical model, in lieu of being able to do this exactly in general. The supplementary appendix also describes the nature of the resulting approximation.

A bayesian model for mixed responses and skew laten variable to measure HRQoL in children

The aim of the thesi is to formulate a suitable Item Response Theory (IRT) based model to measure HRQoL (as latent variable) using a mixed responses questionnaire and relaxing the hypothesis of normal distributed latent variable. The new model is a combination of two models already presented in literature, that is, a latent trait model for mixed responses and an IRT model for Skew Normal latent variable. It is developed in a Bayesian framework, a Markov chain Monte Carlo procedure is used to generate samples of the posterior distribution of the parameters of interest. The proposed model is test on a questionnaire composed by 5 discrete items and one continuous to measure HRQoL in children, the EQ-5D-Y questionnaire. A large sample of children collected in the schools was used. In comparison with a model for only discrete responses and a model for mixed responses and normal latent variable, the new model has better performances, in term of deviance information criterion (DIC), chain convergences times and precision of the estimates.

Multilevel Latent Class Models with Dirichlet Mixing Distribution

Latent class analysis (LCA) and latent class regression (LCR) are widely used for modeling multivariate categorical outcomes in social sciences and biomedical studies. Standard analyses assume data of different respondents to be mutually independent, excluding application of the methods to familial and other designs in which participants are clustered. In this paper, we develop multilevel latent class model, in which subpopulation mixing probabilities are treated as random effects that vary among clusters according to a common Dirichlet distribution. We apply the Expectation-Maximization (EM) algorithm for model fitting by maximum likelihood (ML). This approach works well, but is computationally intensive when either the number of classes or the cluster size is large. We propose a maximum pairwise likelihood (MPL) approach via a modified EM algorithm for this case. We also show that a simple latent class analysis, combined with robust standard errors, provides another consistent, robust, but less efficient inferential procedure. Simulation studies suggest that the three methods work well in finite samples, and that the MPL estimates often enjoy comparable precision as the ML estimates. We apply our methods to the analysis of comorbid symptoms in the Obsessive Compulsive Disorder study. Our models' random effects structure has more straightforward interpretation than those of competing methods, thus should usefully augment tools available for latent class analysis of multilevel data.

Modelling the Impact of Built Environment, Geographical Scales and Latent Constructs On Individual Travel Behaviour

La relación entre la estructura urbana y la movilidad ha sido estudiada desde hace más de 70 años. El entorno urbano incluye múltiples dimensiones como por ejemplo: la estructura urbana, los usos de suelo, la distribución de instalaciones diversas (comercios, escuelas y zonas de restauración, parking, etc.). Al realizar una revisión de la literatura existente en este contexto, se encuentran distintos análisis, metodologías, escalas geográficas y dimensiones, tanto de la movilidad como de la estructura urbana. En este sentido, se trata de una relación muy estudiada pero muy compleja, sobre la que no existe hasta el momento un consenso sobre qué dimensión del entorno urbano influye sobre qué dimensión de la movilidad, y cuál es la manera apropiada de representar esta relación. Con el propósito de contestar estas preguntas investigación, la presente tesis tiene los siguientes objetivos generales: (1) Contribuir al mejor entendimiento de la compleja relación estructura urbana y movilidad. y (2) Entender el rol de los atributos latentes en la relación entorno urbano y movilidad. El objetivo específico de la tesis es analizar la influencia del entorno urbano sobre dos dimensiones de la movilidad: número de viajes y tipo de tour. Vista la complejidad de la relación entorno urbano y movilidad, se pretende contribuir al mejor entendimiento de la relación a través de la utilización de 3 escalas geográficas de las variables y del análisis de la influencia de efectos inobservados en la movilidad. Para el análisis se utiliza una base de datos conformada por tres tipos de datos: (1) Una encuesta de movilidad realizada durante los años 2006 y 2007. Se obtuvo un total de 943 encuestas, en 3 barrios de Madrid: Chamberí, Pozuelo y Algete. (2) Información municipal del Instituto Nacional de Estadística: dicha información se encuentra enlazada con los orígenes y destinos de los viajes recogidos en la encuesta. Y (3) Información georeferenciada en Arc-GIS de los hogares participantes en la encuesta: la base de datos contiene información respecto a la estructura de las calles, localización de escuelas, parking, centros médicos y lugares de restauración. Se analizó la correlación entre e intra-grupos y se modelizaron 4 casos de atributos bajo la estructura ordinal logit. Posteriormente se evalúa la auto-selección a través de la estimación conjunta de las elecciones de tipo de barrio y número de viajes. La elección del tipo de barrio consta de 3 alternativas: CBD, Urban y Suburban, según la zona de residencia recogida en las encuestas. Mientras que la elección del número de viajes consta de 4 categorías ordinales: 0 viajes, 1-2 viajes, 3-4 viajes y 5 o más viajes. A partir de la mejor especificación del modelo ordinal logit. Se desarrolló un modelo joint mixed-ordinal conjunto. Los resultados indican que las variables exógenas requieren un análisis exhaustivo de correlaciones con el fin de evitar resultados sesgados. ha determinado que es importante medir los atributos del BE donde se realiza el viaje, pero también la información municipal es muy explicativa de la movilidad individual. Por tanto, la percepción de las zonas de destino a nivel municipal es considerada importante. En el contexto de la Auto-selección (self-selection) es importante modelizar conjuntamente las decisiones. La Auto-selección existe, puesto que los parámetros estimados conjuntamente son significativos. Sin embargo, sólo ciertos atributos del entorno urbano son igualmente importantes sobre la elección de la zona de residencia y frecuencia de viajes. Para analizar la Propensión al Viaje, se desarrolló un modelo híbrido, formado por: una variable latente, un indicador y un modelo de elección discreta. La variable latente se denomina “Propensión al Viaje”, cuyo indicador en ecuación de medida es el número de viajes; la elección discreta es el tipo de tour. El modelo de elección consiste en 5 alternativas, según la jerarquía de actividades establecida en la tesis: HOME, no realiza viajes durante el día de estudio, HWH tour cuya actividad principal es el trabajo o estudios, y no se realizan paradas intermedias; HWHs tour si el individuo reaiza paradas intermedias; HOH tour cuya actividad principal es distinta a trabajo y estudios, y no se realizan paradas intermedias; HOHs donde se realizan paradas intermedias. Para llegar a la mejor especificación del modelo, se realizó un trabajo importante considerando diferentes estructuras de modelos y tres tipos de estimaciones. De tal manera, se obtuvieron parámetros consistentes y eficientes. Los resultados muestran que la modelización de los tours, representa una ventaja sobre la modelización de los viajes, puesto que supera las limitaciones de espacio y tiempo, enlazando los viajes realizados por la misma persona en el día de estudio. La propensión al viaje (PT) existe y es específica para cada tipo de tour. Los parámetros estimados en el modelo híbrido resultaron significativos y distintos para cada alternativa de tipo de tour. Por último, en la tesis se verifica que los modelos híbridos representan una mejora sobre los modelos tradicionales de elección discreta, dando como resultado parámetros consistentes y más robustos. En cuanto a políticas de transporte, se ha demostrado que los atributos del entorno urbano son más importantes que los LOS (Level of Service) en la generación de tours multi-etapas. la presente tesis representa el primer análisis empírico de la relación entre los tipos de tours y la propensión al viaje. El concepto Propensity to Travel ha sido desarrollado exclusivamente para la tesis. Igualmente, el desarrollo de un modelo conjunto RC-Number of trips basado en tres escalas de medida representa innovación en cuanto a la comparación de las escalas geográficas, que no había sido hecha en la modelización de la self-selection. The relationship between built environment (BE) and travel behaviour (TB) has been studied in a number of cases, using several methods - aggregate and disaggregate approaches - and different focuses – trip frequency, automobile use, and vehicle miles travelled and so on. Definitely, travel is generated by the need to undertake activities and obtain services, and there is a general consensus that urban components affect TB. However researches are still needed to better understand which components of the travel behaviour are affected most and by which of the urban components. In order to fill the gap in the research, the present dissertation faced two main objectives: (1) To contribute to the better understanding of the relationship between travel demand and urban environment. And (2) To develop an econometric model for estimating travel demand with urban environment attributes. With this purpose, the present thesis faced an exhaustive research and computation of land-use variables in order to find the best representation of BE for modelling trip frequency. In particular two empirical analyses are carried out: 1. Estimation of three dimensions of travel demand using dimensions of urban environment. We compare different travel dimensions and geographical scales, and we measure self-selection contribution following the joint models. 2. Develop a hybrid model, integrated latent variable and discrete choice model. The implementation of hybrid models is new in the analysis of land-use and travel behaviour. BE and TB explicitly interact and allow richness information about a specific individual decision process For all empirical analysis is used a data-base from a survey conducted in 2006 and 2007 in Madrid. Spatial attributes describing neighbourhood environment are derived from different data sources: National Institute of Statistics-INE (Administrative: municipality and district) and GIS (circular units). INE provides raw data for such spatial units as: municipality and district. The construction of census units is trivial as the census bureau provides tables that readily define districts and municipalities. The construction of circular units requires us to determine the radius and associate the spatial information to our households. The first empirical part analyzes trip frequency by applying an ordered logit model. In this part is studied the effect of socio-economic, transport and land use characteristics on two travel dimensions: trip frequency and type of tour. In particular the land use is defined in terms of type of neighbourhoods and types of dwellers. Three neighbourhood representations are explored, and described three for constructing neighbourhood attributes. In particular administrative units are examined to represent neighbourhood and circular – unit representation. Ordered logit models are applied, while ordinal logit models are well-known, an intensive work for constructing a spatial attributes was carried out. On the other hand, the second empirical analysis consists of the development of an innovative econometric model that considers a latent variable called “propensity to travel”, and choice model is the choice of type of tour. The first two specifications of ordinal models help to estimate this latent variable. The latent variable is unobserved but the manifestation is called “indicators”, then the probability of choosing an alternative of tour is conditional to the probability of latent variable and type of tour. Since latent variable is unknown we fit the integral over its distribution. Four “sets of best variables” are specified, following the specification obtained from the correlation analysis. The results evidence that the relative importance of SE variables versus BE variables depends on how BE variables are measured. We found that each of these three spatial scales has its intangible qualities and drawbacks. Spatial scales play an important role on predicting travel demand due to the variability in measures at trip origin/destinations within the same administrative unit (municipality, district and so on). Larger units will produce less variation in data; but it does not affect certain variables, such as public transport supply, that are more significant at municipality level. By contrast, land-use measures are more efficient at district level. Self-selection in this context, is weak. Thus, the influence of BE attributes is true. The results of the hybrid model show that unobserved factors affect the choice of tour complexity. The latent variable used in this model is propensity to travel that is explained by socioeconomic aspects and neighbourhood attributes. The results show that neighbourhood attributes have indeed a significant impact on the choice of the type of tours either directly and through the propensity to travel. The propensity to travel has a different impact depending on the structure of each tour and increases the probability of choosing more complex tours, such as tours with many intermediate stops. The integration of choice and latent variable model shows that omitting important perception and attitudes leads to inconsistent estimates. The results also indicate that goodness of fit improves by adding the latent variable in both sequential and simultaneous estimation. There are significant differences in the sensitivity to the latent variable across alternatives. In general, as expected, the hybrid models show a major improvement into the goodness of fit of the model, compared to a classical discrete choice model that does not incorporate latent effects. The integrated model leads to a more detailed analysis of the behavioural process. Summarizing, the effect that built environment characteristics on trip frequency studied is deeply analyzed. In particular we tried to better understand how land use characteristics can be defined and measured and which of these measures do have really an impact on trip frequency. We also tried to test the superiority of HCM on this field. We can concluded that HCM shows a major improvement into the goodness of fit of the model, compared to classical discrete choice model that does not incorporate latent effects. And consequently, the application of HCM shows the importance of LV on the decision of tour complexity. People are more elastic to built environment attributes than level of services. Thus, policy implications must take place to develop more mixed areas, work-places in combination with commercial retails.

Semi-supervised learning of hierarchical latent trait models for data visualisation

An interactive hierarchical Generative Topographic Mapping (HGTM) ¸iteH_GTM has been developed to visualise complex data sets. In this paper, we build a more general visualisation system by extending the HGTM visualisation system in 3 directions: bf (1) We generalize HGTM to noise models from the exponential family of distributions. The basic building block is the Latent Trait Model (LTM) developed in ¸iteKaban_pami. bf (2) We give the user a choice of initializing the child plots of the current plot in either em interactive, or em automatic mode. In the interactive mode the user interactively selects ``regions of interest'' as in ¸iteH_GTM, whereas in the automatic mode an unsupervised minimum message length (MML)-driven construction of a mixture of LTMs is employed. bf (3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualisation plots, since they can highlight the boundaries between data clusters. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. We illustrate our approach on a toy example and apply our system to three more complex real data sets.

Multi-level visualisation using Gaussian process latent variable models

Projection of a high-dimensional dataset onto a two-dimensional space is a useful tool to visualise structures and relationships in the dataset. However, a single two-dimensional visualisation may not display all the intrinsic structure. Therefore, hierarchical/multi-level visualisation methods have been used to extract more detailed understanding of the data. Here we propose a multi-level Gaussian process latent variable model (MLGPLVM). MLGPLVM works by segmenting data (with e.g. K-means, Gaussian mixture model or interactive clustering) in the visualisation space and then fitting a visualisation model to each subset. To measure the quality of multi-level visualisation (with respect to parent and child models), metrics such as trustworthiness, continuity, mean relative rank errors, visualisation distance distortion and the negative log-likelihood per point are used. We evaluate the MLGPLVM approach on the ‘Oil Flow’ dataset and a dataset of protein electrostatic potentials for the ‘Major Histocompatibility Complex (MHC) class I’ of humans. In both cases, visual observation and the quantitative quality measures have shown better visualisation at lower levels.

Semisupervised learning of hierarchical latent trait models for data visualization

Recently, we have developed the hierarchical Generative Topographic Mapping (HGTM), an interactive method for visualization of large high-dimensional real-valued data sets. In this paper, we propose a more general visualization system by extending HGTM in three ways, which allows the user to visualize a wider range of data sets and better support the model development process. 1) We integrate HGTM with noise models from the exponential family of distributions. The basic building block is the Latent Trait Model (LTM). This enables us to visualize data of inherently discrete nature, e.g., collections of documents, in a hierarchical manner. 2) We give the user a choice of initializing the child plots of the current plot in either interactive, or automatic mode. In the interactive mode, the user selects "regions of interest," whereas in the automatic mode, an unsupervised minimum message length (MML)-inspired construction of a mixture of LTMs is employed. The unsupervised construction is particularly useful when high-level plots are covered with dense clusters of highly overlapping data projections, making it difficult to use the interactive mode. Such a situation often arises when visualizing large data sets. 3) We derive general formulas for magnification factors in latent trait models. Magnification factors are a useful tool to improve our understanding of the visualization plots, since they can highlight the boundaries between data clusters. We illustrate our approach on a toy example and evaluate it on three more complex real data sets. © 2005 IEEE.