Statistical Tests for Joint Analysis of Performance Measures

Recently there has been an increasing interest in the development of new methods using Pareto optimality to deal with multi-objective criteria (for example, accuracy and architectural complexity). Once one has learned a model based on their devised method, the problem is then how to compare it with the state of art. In machine learning, algorithms are typically evaluated by comparing their performance on different data sets by means of statistical tests. Unfortunately, the standard tests used for this purpose are not able to jointly consider performance measures. The aim of this paper is to resolve this issue by developing statistical procedures that are able to account for multiple competing measures at the same time. In particular, we develop two tests: a frequentist procedure based on the generalized likelihood-ratio test and a Bayesian procedure based on a multinomial-Dirichlet conjugate model. We further extend them by discovering conditional independences among measures to reduce the number of parameter of such models, as usually the number of studied cases is very reduced in such comparisons. Real data from a comparison among general purpose classifiers is used to show a practical application of our tests.

Mixture modeling for joint analysis of survival, discrete, and continuous data

Mixture modeling is commonly used to model categorical latent variables that represent subpopulations in which population membership is unknown but can be inferred from the data. In relatively recent years, the potential of finite mixture models has been applied in time-to-event data. However, the commonly used survival mixture model assumes that the effects of the covariates involved in failure times differ across latent classes, but the covariate distribution is homogeneous. The aim of this dissertation is to develop a method to examine time-to-event data in the presence of unobserved heterogeneity under a framework of mixture modeling. A joint model is developed to incorporate the latent survival trajectory along with the observed information for the joint analysis of a time-to-event variable, its discrete and continuous covariates, and a latent class variable. It is assumed that the effects of covariates on survival times and the distribution of covariates vary across different latent classes. The unobservable survival trajectories are identified through estimating the probability that a subject belongs to a particular class based on observed information. We applied this method to a Hodgkin lymphoma study with long-term follow-up and observed four distinct latent classes in terms of long-term survival and distributions of prognostic factors. Our results from simulation studies and from the Hodgkin lymphoma study demonstrated the superiority of our joint model compared with the conventional survival model. This flexible inference method provides more accurate estimation and accommodates unobservable heterogeneity among individuals while taking involved interactions between covariates into consideration.^

Handedness in twins: Joint analysis of data from 35 samples

Simultaneous analysis of handedness data from 35 samples of twins (with a combined sample size of 21,127 twin pairs) found a small but significant additive genetic effect accounting for 25.47% of the variance (95% confidence interval [CI] 15.69-29.51%). No common environmental influences were detected (C = 0.00; 95% Cl 0.00-7.67%), with the majority of the variance, 74.53%, explained by factors unique to the individual (95% Cl 70.49-78.67%). No significant heterogeneity was observed within studies that used similar methods to assess handedness, or across studies that used different methods. At an individual level the majority of studies had insufficient power to reject a purely unique environmental model due to insufficient power to detect familial aggregation. This lack of power is seldom mentioned within studies, and has contributed to the misconception that twin studies of handedness are not informative.

Joint Analysis of Multiple Algorithms and Performance Measures

There has been an increasing interest in the development of new methods using Pareto optimality to deal with multi-objective criteria (for example, accuracy and time complexity). Once one has developed an approach to a problem of interest, the problem is then how to compare it with the state of art. In machine learning, algorithms are typically evaluated by comparing their performance on different data sets by means of statistical tests. Standard tests used for this purpose are able to consider jointly neither performance measures nor multiple competitors at once. The aim of this paper is to resolve these issues by developing statistical procedures that are able to account for multiple competing measures at the same time and to compare multiple algorithms altogether. In particular, we develop two tests: a frequentist procedure based on the generalized likelihood-ratio test and a Bayesian procedure based on a multinomial-Dirichlet conjugate model. We further extend them by discovering conditional independences among measures to reduce the number of parameters of such models, as usually the number of studied cases is very reduced in such comparisons. Data from a comparison among general purpose classifiers is used to show a practical application of our tests.

Joint pitch-analysis formant-synthesis framework for CS recovery of speech

A joint analysis-synthesis framework is developed for the compressive sensing (CS) recovery of speech signals. The signal is assumed to be sparse in the residual domain with the linear prediction filter used as the sparse transformation. Importantly this transform is not known apriori, since estimating the predictor filter requires the knowledge of the signal. Two prediction filters, one comb filter for pitch and another all pole formant filter are needed to induce maximum sparsity. An iterative method is proposed for the estimation of both the prediction filters and the signal itself. Formant prediction filter is used as the synthesis transform, while the pitch filter is used to model the periodicity in the residual excitation signal, in the analysis mode. Significant improvement in the LLR measure is seen over the previously reported formant filter estimation.

Thermodynamic and statistical analysis of soybean oil extraction process using renewable solvent

The main goal of this work was to evaluate thermodynamic parameters of the soybean oil extraction process using ethanol as solvent. The experimental treatments were as follows: aqueous solvents with water contents varying from 0 to 13% (mass basis) and extraction temperature varying from 50 to 100 degrees C. The distribution coefficients of oil at equilibrium have been used to calculate enthalpy, entropy and free energy changes. The results indicate that oil extraction process with ethanol is feasible and spontaneous, mainly under higher temperature. Also, the influence of water level in the solvent and temperature were analysed using the response surface methodology (RSM). It can be noted that the extraction yield was highly affected by both independent variables. A joint analysis of thermodynamic and RSM indicates the optimal level of solvent hydration and temperature to perform the extraction process.

A bayesian nonparametric joint factor model for learning shared and individual subspaces from multiple data sources

Joint analysis of multiple data sources is becoming increasingly popular in transfer learning, multi-task learning and cross-domain data mining. One promising approach to model the data jointly is through learning the shared and individual factor subspaces. However, performance of this approach depends on the subspace dimensionalities and the level of sharing needs to be specified a priori. To this end, we propose a nonparametric joint factor analysis framework for modeling multiple related data sources. Our model utilizes the hierarchical beta process as a nonparametric prior to automatically infer the number of shared and individual factors. For posterior inference, we provide a Gibbs sampling scheme using auxiliary variables. The effectiveness of the proposed framework is validated through its application on two real world problems - transfer learning in text and image retrieval.

Observation of the rare B-s(0)->mu(+)mu(-) decay from the combined analysis of CMS and LHCb data

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Analysis of hourly global, direct and diffuse solar radiations attenuation as a function of optical air mass

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Limits to dark matter annihilation cross-section from a combined analysis of MAGIC and Fermi-LAT observations of dwarf satellite galaxies

We present the first joint analysis of gamma-ray data from the MAGIC Cherenkov telescopes and the Fermi Large Area Telescope (LAT) to search for gamma-ray signals from dark matter annihilation in dwarf satellite galaxies. We combine 158 hours of Segue 1 observations with MAGIC with 6-year observations of 15 dwarf satellite galaxies by the Fermi-LAT. We obtain limits on the annihilation cross-section for dark matter particle masses between 10 GeV and 100 TeV – the widest mass range ever explored by a single gamma-ray analysis. These limits improve on previously published Fermi-LAT and MAGIC results by up to a factor of two at certain masses. Our new inclusive analysis approach is completely generic and can be used to perform a global, sensitivity-optimized dark matter search by combining data from present and future gamma-ray and neutrino detectors.

Do socioeconomic inequalities in mortality vary between different Spanish cities? a pooled cross-sectional analysis

Background: The relationship between deprivation and mortality in urban settings is well established. This relationship has been found for several causes of death in Spanish cities in independent analyses (the MEDEA project). However, no joint analysis which pools the strength of this relationship across several cities has ever been undertaken. Such an analysis would determine, if appropriate, a joint relationship by linking the associations found. Methods: A pooled cross-sectional analysis of the data from the MEDEA project has been carried out for each of the causes of death studied. Specifically, a meta-analysis has been carried out to pool the relative risks in eleven Spanish cities. Different deprivation-mortality relationships across the cities are considered in the analysis (fixed and random effects models). The size of the cities is also considered as a possible factor explaining differences between cities. Results: Twenty studies have been carried out for different combinations of sex and causes of death. For nine of them (men: prostate cancer, diabetes, mental illnesses, Alzheimer’s disease, cerebrovascular disease; women: diabetes, mental illnesses, respiratory diseases, cirrhosis) no differences were found between cities in the effect of deprivation on mortality; in four cases (men: respiratory diseases, all causes of mortality; women: breast cancer, Alzheimer’s disease) differences not associated with the size of the city have been determined; in two cases (men: cirrhosis; women: lung cancer) differences strictly linked to the size of the city have been determined, and in five cases (men: lung cancer, ischaemic heart disease; women: ischaemic heart disease, cerebrovascular diseases, all causes of mortality) both kinds of differences have been found. Except for lung cancer in women, every significant relationship between deprivation and mortality goes in the same direction: deprivation increases mortality. Variability in the relative risks across cities was found for general mortality for both sexes. Conclusions: This study provides a general overview of the relationship between deprivation and mortality for a sample of large Spanish cities combined. This joint study allows the exploration of and, if appropriate, the quantification of the variability in that relationship for the set of cities considered.

A joint electromagnetic and seismic study of an active pockmark within the hydrate stability field at the Vestnesa Ridge, West Svalbard margin

We acquired coincident marine controlled-source electromagnetic (CSEM), high-resolution seismic reflection and ocean-bottom seismometer (OBS) data over an active pockmark in the crest of the southern part of the Vestnesa Ridge, to estimate fluid composition within an underlying fluid-migration chimney. Synthetic model studies suggest resistivity obtained from CSEM data can resolve gas or hydrate saturation greater than 5% within the chimney. Acoustic chimneys imaged by seismic reflection data beneath the pockmark and on the ridge flanks, were found to be associated with high-resistivity anomalies (+2-4 m). High-velocity anomalies (+0.3 km/s), within the gas hydrate stability zone (GHSZ) and low-velocity anomalies (-0.2 km/s) underlying the GHSZ, were also observed. Joint analysis of the resistivity and velocity anomaly indicates pore saturation of up to 52% hydrate with 28% free gas, or up to 73% hydrate with 4% free gas, within the chimney beneath the pockmark assuming a non-uniform and uniform fluid distribution respectively. Similarly, we estimate up to 30% hydrate with 4% free gas or 30% hydrate with 2% free gas within the pore space of the GHSZ outside the central chimney assuming a non-uniform and uniform fluid distribution respectively. High levels of free-gas saturation in the top part of the chimney are consistent with episodic gas venting from the pockmark.

Biplot analysis of strawberry genotypes recommended for the State of Espírito Santo.

Most strawberry genotypes grown commercially in Brazil originate from breeding programs in the United States, and are therefore not adapted to the various soil and climatic conditions found in Brazil. Thus, quantifying the magnitude of genotype x environment (GE) interactions serves as a primary means for increasing average Brazilian strawberry yields, and helps provide specific recommendations for farmers on which genotypes meet high yield and phenotypic stability thresholds. The aim of this study was to use AMMI (additive main effects and multiplicative interaction) and GGE biplot (genotype main effects + genotype x environment interaction) analyses to identify high-yield, stable strawberry genotypes grown at three locations in Espírito Santo for two agricultural years. We evaluated seven strawberry genotypes (Dover, Camino Real, Ventana, Camarosa, Seascape, Diamante, and Aromas) at three locations (Domingos Martins, Iúna, and Muniz Freire) in agricultural years 2006 and 2007, totaling six study environments. Joint analysis of variance was calculated using yield data (t/ha), and AMMI and GGE biplot analysis was conducted following the detection of a significant genotypes x agricultural years x locations (G x A x L) interaction. During the two agricultural years, evaluated locations were allocated to different regions on biplot graphics using both methods, indicating distinctions among them. Based on the results obtained from the two methods used in this study to investigate the G x A x L interaction, we recommend growing the Camarosa genotype for production at the three locations assessed due to the high frequency of favorable alleles, which were expressed in all localities evaluated regardless of the agricultural year.

Evaluation of TxDOT’s J.A.C.K. model for revenue and expenditure projections

This research report documents work conducted by the Center for Transportation (CTR) at The University of Texas at Austin in analyzing the Joint Analysis using the Combined Knowledge (J.A.C.K.) program. This program was developed by the Texas Department of Transportation (TxDOT) to make projections of revenues and expenditures. This research effort was to span from September 2008 to August 2009, but the bulk of the work was completed and presented by December 2008. J.A.C.K. was subsequently renamed TRENDS, but for consistency with the scope of work, the original name is used throughout this report.