Hierarchical cluster analysis of groups of individuals : application to business data

3rd SMTDA Conference Proceedings, 11-14 June 2014, Lisbon Portugal.

On cluster analysis of complex and heterogeneous data

3rd SMTDA Conference Proceedings, 11-14 June 2014, Lisbon Portugal.

Clustering an interval data set : are the main partitions similar to a priori partition?

This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Some reflections on acquisition, processing and analysis of statistical data in forest soils

Beyond the classical statistical approaches (determination of basic statistics, regression analysis, ANOVA, etc.) a new set of applications of different statistical techniques has increasingly gained relevance in the analysis, processing and interpretation of data concerning the characteristics of forest soils. This is possible to be seen in some of the recent publications in the context of Multivariate Statistics. These new methods require additional care that is not always included or refered in some approaches. In the particular case of geostatistical data applications it is necessary, besides to geo-reference all the data acquisition, to collect the samples in regular grids and in sufficient quantity so that the variograms can reflect the spatial distribution of soil properties in a representative manner. In the case of the great majority of Multivariate Statistics techniques (Principal Component Analysis, Correspondence Analysis, Cluster Analysis, etc.) despite the fact they do not require in most cases the assumption of normal distribution, they however need a proper and rigorous strategy for its utilization. In this work, some reflections about these methodologies and, in particular, about the main constraints that often occur during the information collecting process and about the various linking possibilities of these different techniques will be presented. At the end, illustrations of some particular cases of the applications of these statistical methods will also be presented.

Entropy Analysis of Industrial Accident Data Series

Complex industrial plants exhibit multiple interactions among smaller parts and with human operators. Failure in one part can propagate across subsystem boundaries causing a serious disaster. This paper analyzes the industrial accident data series in the perspective of dynamical systems. First, we process real world data and show that the statistics of the number of fatalities reveal features that are well described by power law (PL) distributions. For early years, the data reveal double PL behavior, while, for more recent time periods, a single PL fits better into the experimental data. Second, we analyze the entropy of the data series statistics over time. Third, we use the Kullback–Leibler divergence to compare the empirical data and multidimensional scaling (MDS) techniques for data analysis and visualization. Entropy-based analysis is adopted to assess complexity, having the advantage of yielding a single parameter to express relationships between the data. The classical and the generalized (fractional) entropy and Kullback–Leibler divergence are used. The generalized measures allow a clear identification of patterns embedded in the data.

Economic analysis in health care regulation : three essays

ABSTRACT - It is the purpose of the present thesis to emphasize, through a series of examples, the need and value of appropriate pre-analysis of the impact of health care regulation. Specifically, the thesis presents three papers on the theme of regulation in different aspects of health care provision and financing. The first two consist of economic analyses of the impact of health care regulation and the third comprises the creation of an instrument for supporting economic analysis of health care regulation, namely in the field of evaluation of health care programs. The first paper develops a model of health plan competition and pricing in order to understand the dynamics of health plan entry and exit in the presence of switching costs and alternative health premium payment systems. We build an explicit model of death spirals, in which profitmaximizing competing health plans find it optimal to adopt a pattern of increasing relative prices culminating in health plan exit. We find the steady-state numerical solution for the price sequence and the plan’s optimal length of life through simulation and do some comparative statics. This allows us to show that using risk adjusted premiums and imposing price floors are effective at reducing death spirals and switching costs, while having employees pay a fixed share of the premium enhances death spirals and increases switching costs. Price regulation of pharmaceuticals is one of the cost control measures adopted by the Portuguese government, as in many European countries. When such regulation decreases the products’ real price over time, it may create an incentive for product turnover. Using panel data for the period of 1997 through 2003 on drug packages sold in Portuguese pharmacies, the second paper addresses the question of whether price control policies create an incentive for product withdrawal. Our work builds the product survival literature by accounting for unobservable product characteristics and heterogeneity among consumers when constructing quality, price control and competition indexes. These indexes are then used as covariates in a Cox proportional hazard model. We find that, indeed, price control measures increase the probability of exit, and that such effect is not verified in OTC market where no such price regulation measures exist. We also find quality to have a significant positive impact on product survival. In the third paper, we develop a microsimulation discrete events model (MSDEM) for costeffectiveness analysis of Human Immunodeficiency Virus treatment, simulating individual paths from antiretroviral therapy (ART) initiation to death. Four driving forces determine the course of events: CD4+ cell count, viral load resistance and adherence. A novel feature of the model with respect to the previous MSDEMs is that distributions of time to event depend on individuals’ characteristics and past history. Time to event was modeled using parametric survival analysis. Events modeled include: viral suppression, regimen switch due virological failure, regimen switch due to other reasons, resistance development, hospitalization, AIDS events, and death. Disease progression is structured according to therapy lines and the model is parameterized with cohort Portuguese observational data. An application of the model is presented comparing the cost-effectiveness ART initiation with two nucleoside analogue reverse transcriptase inhibitors (NRTI) plus one non-nucleoside reverse transcriptase inhibitor(NNRTI) to two NRTI plus boosted protease inhibitor (PI/r) in HIV- 1 infected individuals. We find 2NRTI+NNRTI to be a dominant strategy. Results predicted by the model reproduce those of the data used for parameterization and are in line with those published in the literature.

An R package for the integrated analysis of metabolomics and spectral data

Recently, there has been a growing interest in the field of metabolomics, materialized by a remarkable growth in experimental techniques, available data and related biological applications. Indeed, techniques as Nuclear Magnetic Resonance, Gas or Liquid Chromatography, Mass Spectrometry, Infrared and UV-visible spectroscopies have provided extensive datasets that can help in tasks as biological and biomedical discovery, biotechnology and drug development. However, as it happens with other omics data, the analysis of metabolomics datasets provides multiple challenges, both in terms of methodologies and in the development of appropriate computational tools. Indeed, from the available software tools, none addresses the multiplicity of existing techniques and data analysis tasks. In this work, we make available a novel R package, named specmine, which provides a set of methods for metabolomics data analysis, including data loading in different formats, pre-processing, metabolite identification, univariate and multivariate data analysis, machine learning, and feature selection. Importantly, the implemented methods provide adequate support for the analysis of data from diverse experimental techniques, integrating a large set of functions from several R packages in a powerful, yet simple to use environment. The package, already available in CRAN, is accompanied by a web site where users can deposit datasets, scripts and analysis reports to be shared with the community, promoting the efficient sharing of metabolomics data analysis pipelines.

Recruitment and retention incentives in health labour markets: an analysis of participation in NHS Scotland following Dental Vocational

This paper uses a unique individual level administrative data set to analyse the participation of health professionals in the NHS after training. The data set contains information on over 1,000 dentists who received Dental Vocational Training in Scotland between 1995 and 2006. Using a dynamic nonlinear panel data model, we estimate the determinants of post-training participation. We nd there is signi cant persistence in these data and are able to show that the persistence arises from state dependence and individual heterogeneity. This finding has implications for the structure of policies designed to increase participation rates. We apply this empirical framework to assess the accuracy of predictions for workforce forecasting, and to provide a preliminary estimate of the impact of one of the recruitment and retention policies available to dentists in Scotland.

Distributional and Poverty Consequences of Globalization: A Dynamic Comparative Analysis for Developing Countries

This study examines the impact of globalization on cross-country inequality and poverty using a panel data set for 65 developing counties, over the period 1970-2008. With separate modelling for poverty and inequality, explicit control for financial intermediation, and comparative analysis for developing countries, the study attempts to provide a deeper understanding of cross country variations in income inequality and poverty. The major findings of the study are five fold. First, a non-monotonic relationship between income distribution and the level of economic development holds in all samples of countries. Second, both openness to trade and FDI do not have a favourable effect on income distribution in developing countries. Third, high financial liberalization exerts a negative and significant influence on income distribution in developing countries. Fourth, inflation seems to distort income distribution in all sets of countries. Finally, the government emerges as a major player in impacting income distribution in developing countries.

Regional productivity variation and the impact of public capital stock: an analysis with spatial interaction, with reference to Spain

In this paper we examine whether variations in the level of public capital across Spain‟s Provinces affected productivity levels over the period 1996-2005. The analysis is motivated by contemporary urban economics theory, involving a production function for the competitive sector of the economy („industry‟) which includes the level of composite services derived from „service‟ firms under monopolistic competition. The outcome is potentially increasing returns to scale resulting from pecuniary externalities deriving from internal increasing returns in the monopolistic competition sector. We extend the production function by also making (log) labour efficiency a function of (log) total public capital stock and (log) human capital stock, leading to a simple and empirically tractable reduced form linking productivity level to density of employment, human capital and public capital stock. The model is further extended to include technological externalities or spillovers across provinces. Using panel data methodology, we find significant elasticities for total capital stock and for human capital stock, and a significant impact for employment density. The finding that the effect of public capital is significantly different from zero, indicating that it has a direct effect even after controlling for employment density, is contrary to some of the earlier research findings which leave the question of the impact of public capital unresolved.

Mobility and Housing Satisfaction: An Empirical Analysis for Twelve EU Countries

Using panel data for twelve EU countries, we analyze the relationship between selfreported housing satisfaction and residential mobility. Our results indicate the existence of a positive link between the two variables and that housing satisfaction exerts a mediating effect between residential characteristics and dwellers' mobility propensities. Some interesting cross-country differences regarding the effect of other variables on mobility are also observed. Our results can be used in defining, implementing and evaluating housing and neighbourhood policies. Residential satisfaction is put forward as one of the most appropriate indicators of the success or failure of such policies. Keywords: Housing satisfaction, residential mobility JEL classification: R21, D19

High Growth Firms and Innovation: an empirical analysis for Spanish firms

This paper analyzes the effect of firms’ innovation activities on their growth performance. In particular, we observe how important innovation is for high-growth firms (HGFs) for an extensive sample of Spanish manufacturing and services firms. The panel data used comprises diverse waves of Spanish CIS over the the period 2004-2008. First, a probit analysis determines whether innovation affects the probability of being a high-growth firm. And second, a quantile regression technique is applied to explore the determinants and characteristics of specific groups of firms (manufacturing versus service firms and high-tech versus low-tech firms). It is revealed that R&D plays a significant role in the probability of becoming a HGF. Investment in internal and external R&D per employee has a positive impact on firm growth (although internal R&D presents a significant impact in the last quantiles, external R&D is significant up to the median). Furthermore, we show evidence that there is a positive impact of employment (sales) growth on the sales (employment) growth. Keywords: high-growth firms, firm growth, innovation activity JEL Classifications: L11, L25, O30

Robust Factor Analysis for Compositional Data

Factor analysis as frequent technique for multivariate data inspection is widely used also for compositional data analysis. The usual way is to use a centered logratio (clr)transformation to obtain the random vector y of dimension D. The factor model istheny = Λf + e (1)with the factors f of dimension k & D, the error term e, and the loadings matrix Λ.Using the usual model assumptions (see, e.g., Basilevsky, 1994), the factor analysismodel (1) can be written asCov(y) = ΛΛT + ψ (2)where ψ = Cov(e) has a diagonal form. The diagonal elements of ψ as well as theloadings matrix Λ are estimated from an estimation of Cov(y).Given observed clr transformed data Y as realizations of the random vectory. Outliers or deviations from the idealized model assumptions of factor analysiscan severely effect the parameter estimation. As a way out, robust estimation ofthe covariance matrix of Y will lead to robust estimates of Λ and ψ in (2), seePison et al. (2003). Well known robust covariance estimators with good statisticalproperties, like the MCD or the S-estimators (see, e.g. Maronna et al., 2006), relyon a full-rank data matrix Y which is not the case for clr transformed data (see,e.g., Aitchison, 1986).The isometric logratio (ilr) transformation (Egozcue et al., 2003) solves thissingularity problem. The data matrix Y is transformed to a matrix Z by usingan orthonormal basis of lower dimension. Using the ilr transformed data, a robustcovariance matrix C(Z) can be estimated. The result can be back-transformed tothe clr space byC(Y ) = V C(Z)V Twhere the matrix V with orthonormal columns comes from the relation betweenthe clr and the ilr transformation. Now the parameters in the model (2) can beestimated (Basilevsky, 1994) and the results have a direct interpretation since thelinks to the original variables are still preserved.The above procedure will be applied to data from geochemistry. Our specialinterest is on comparing the results with those of Reimann et al. (2002) for the Kolaproject data

Statistical Analysis of Eco-Toxicologic Data of Mammals from a Polluted Area

Several eco-toxicological studies have shown that insectivorous mammals, due to theirfeeding habits, easily accumulate high amounts of pollutants in relation to other mammal species. To assess the bio-accumulation levels of toxic metals and their in°uenceon essential metals, we quantified the concentration of 19 elements (Ca, K, Fe, B, P,S, Na, Al, Zn, Ba, Rb, Sr, Cu, Mn, Hg, Cd, Mo, Cr and Pb) in bones of 105 greaterwhite-toothed shrews (Crocidura russula) from a polluted (Ebro Delta) and a control(Medas Islands) area. Since chemical contents of a bio-indicator are mainly compositional data, conventional statistical analyses currently used in eco-toxicology can givemisleading results. Therefore, to improve the interpretation of the data obtained, weused statistical techniques for compositional data analysis to define groups of metalsand to evaluate the relationships between them, from an inter-population viewpoint.Hypothesis testing on the adequate balance-coordinates allow us to confirm intuitionbased hypothesis and some previous results. The main statistical goal was to test equalmeans of balance-coordinates for the two defined populations. After checking normality,one-way ANOVA or Mann-Whitney tests were carried out for the inter-group balances

Estimating the probability of leaving unemployment using uncompleted spells from repeated cross-section data

We propose a new econometric estimation method for analyzing the probabilityof leaving unemployment using uncompleted spells from repeated cross-sectiondata, which can be especially useful when panel data are not available. Theproposed method-of-moments-based estimator has two important features:(1) it estimates the exit probability at the individual level and(2) it does not rely on the stationarity assumption of the inflowcomposition. We illustrate and gauge the performance of the proposedestimator using the Spanish Labor Force Survey data, and analyze the changesin distribution of unemployment between the 1980s and 1990s during a periodof labor market reform. We find that the relative probability of leavingunemployment of the short-term unemployed versus the long-term unemployedbecomes significantly higher in the 1990s.