Improving parameter learning of Bayesian nets from incomplete data

This paper addresses the estimation of parameters of a Bayesian network from incomplete data. The task is usually tackled by running the Expectation-Maximization (EM) algorithm several times in order to obtain a high log-likelihood estimate. We argue that choosing the maximum log-likelihood estimate (as well as the maximum penalized log-likelihood and the maximum a posteriori estimate) has severe drawbacks, being affected both by overfitting and model uncertainty. Two ideas are discussed to overcome these issues: a maximum entropy approach and a Bayesian model averaging approach. Both ideas can be easily applied on top of EM, while the entropy idea can be also implemented in a more sophisticated way, through a dedicated non-linear solver. A vast set of experiments shows that these ideas produce significantly better estimates and inferences than the traditional and widely used maximum (penalized) log-likelihood and maximum a posteriori estimates. In particular, if EM is adopted as optimization engine, the model averaging approach is the best performing one; its performance is matched by the entropy approach when implemented using the non-linear solver. The results suggest that the applicability of these ideas is immediate (they are easy to implement and to integrate in currently available inference engines) and that they constitute a better way to learn Bayesian network parameters.

Measurements, modeling and simulation of the off-body radio channel for the implementation of bodyworn antenna diversity at 868 MHz

This paper presents a systematic measurement campaign of diversity reception techniques for use in multiple-antenna wearable systems operating at 868 MHz. The experiments were performed using six time-synchronized bodyworn receivers and considered mobile off-body communications in an anechoic chamber, open office area and a hallway. The cross-correlation coefficient between the signal fading measured by bodyworn receivers was dependent upon the local environment and typically below 0.7. All received signal envelopes were combined in post-processing to study the potential benefits of implementing receiver diversity based upon selection combination, equal-gain and maximal-ratio combining. It is shown that, in an open office area, the 5.7 dB diversity gain obtained using a dual-branch bodyworn maximal-ratio diversity system may be further improved to 11.1 dB if a six-branch system was used. First-and second-order theoretical equations for diversity reception techniques operating in Nakagami fading conditions were used to model the postdetection combined envelopes. Maximum likelihood estimates of the Nakagami-parameter suggest that the fading conditions encountered in this study were generally less severe than Rayleigh. The paper also describes an algorithm that may be used to simulate the measured output of an M-branch diversity combiner operating in independent and identically-distributed Nakagami fading environments.

A Simultaneous Equation Approach to Estimating HIV Prevalence with Non-Ignorable Missing Responses

Estimates of HIV prevalence are important for policy in order to establish the health status of a country's population and to evaluate the effectiveness of population-based interventions and campaigns. However, participation rates in testing for surveillance conducted as part of household surveys, on which many of these estimates are based, can be low. HIV positive individuals may be less likely to participate because they fear disclosure, in which case estimates obtained using conventional approaches to deal with missing data, such as imputation-based methods, will be biased. We develop a Heckman-type simultaneous equation approach which accounts for non-ignorable selection, but unlike previous implementations, allows for spatial dependence and does not impose a homogeneous selection process on all respondents. In addition, our framework addresses the issue of separation, where for instance some factors are severely unbalanced and highly predictive of the response, which would ordinarily prevent model convergence. Estimation is carried out within a penalized likelihood framework where smoothing is achieved using a parametrization of the smoothing criterion which makes estimation more stable and efficient. We provide the software for straightforward implementation of the proposed approach, and apply our methodology to estimating national and sub-national HIV prevalence in Swaziland, Zimbabwe and Zambia.

Maximum likelihood estimation of higher-order integer-valued autoregressive processes

In this article, we extend the earlier work of Freeland and McCabe [Journal of time Series Analysis (2004) Vol. 25, pp. 701–722] and develop a general framework for maximum likelihood (ML) analysis of higher-order integer-valued autoregressive processes. Our exposition includes the case where the innovation sequence has a Poisson distribution and the thinning is binomial. A recursive representation of the transition probability of the model is proposed. Based on this transition probability, we derive expressions for the score function and the Fisher information matrix, which form the basis for ML estimation and inference. Similar to the results in Freeland and McCabe (2004), we show that the score function and the Fisher information matrix can be neatly represented as conditional expectations. Using the INAR(2) speci?cation with binomial thinning and Poisson innovations, we examine both the asymptotic e?ciency and ?nite sample properties of the ML estimator in relation to the widely used conditional least
squares (CLS) and Yule–Walker (YW) estimators. We conclude that, if the Poisson assumption can be justi?ed, there are substantial gains to be had from using ML especially when the thinning parameters are large.

Maximum likelihood estimation of higher-order integer-valued autoregressive processes

Corrigendum Vol. 30, Issue 2, 259, Article first published online: 15 MAR 2009 to correct the order of authors names: Bu R., K. Hadri, and B. McCabe.

The death of massive stars - I. Observational constraints on the progenitors of Type II-P supernovae

We present the results of a 10.5-yr, volume-limited (28-Mpc) search for supernova (SN) progenitor stars. In doing so we compile all SNe discovered within this volume (132, of which 27 per cent are Type Ia) and determine the relative rates of each subtype from literature studies. The core-collapse SNe break down into 59 per cent II-P and 29 per cent Ib/c, with the remainder being IIb (5 per cent), IIn (4 per cent) and II-L (3 per cent). There have been 20 II-P SNe with high-quality optical or near-infrared pre-explosion images that allow a meaningful search for the progenitor stars. In five cases they are clearly red supergiants, one case is unconstrained, two fall on compact coeval star clusters and the other twelve have no progenitor detected. We review and update all the available data for the host galaxies and SN environments (distance, metallicity and extinction) and determine masses and upper mass estimates for these 20 progenitor stars using the STARS stellar evolutionary code and a single consistent homogeneous method. A maximum likelihood calculation suggests that the minimum stellar mass for a Type II-P to form is m(min) = 8.5(-1.5)(+1) M-circle dot and the maximum mass for II-P progenitors is m(max) = 16.5 +/- 1.5 M-circle dot, assuming a Salpeter initial mass function holds for the progenitor population (in the range Gamma = -1.35(-0.7)(+0.3)). The minimum mass is consistent with current estimates for the upper limit to white dwarf progenitor masses, but the maximum mass does not appear consistent with massive star populations in Local Group galaxies. Red supergiants in the Local Group have masses up to 25 M-circle dot and the minimum mass to produce a Wolf-Rayet star in single star evolution (between solar and LMC metallicity) is similarly 25-30 M-circle dot. The reason we have not detected any high-mass red supergiant progenitors above 17 M-circle dot is unclear, but we estimate that it is statistically significant at 2.4 sigma confidence. Two simple reasons for this could be that we have systematically underestimated the progenitor masses due to dust extinction or that stars between 17-25 M-circle dot produce other kinds of SNe which are not II-P. We discuss these possibilities and find that neither provides a satisfactory solution. We term this discrepancy the 'red supergiant problem' and speculate that these stars could have core masses high enough to form black holes and SNe which are too faint to have been detected. We compare the Ni-56 masses ejected in the SNe to the progenitor mass estimates and find that low-luminosity SNe with low Ni-56 production are most likely to arise from explosions of low-mass progenitors near the mass threshold that can produce a core-collapse.

Glaciers and climate in Pacific Far NE Russia during the Last Glacial Maximum

A combined geomorphological–physical model approach is used to generate three-dimensional reconstructions of glaciers in Pacific Far NE Russia during the global Last glacial Maximum (gLGM). The horizontal dimensions of these ice masses are delineated by moraines, their surface elevations are estimated using an iterative flowline model and temporal constraints upon their margins are derived from published age estimates. The equilibrium line altitudes (ELAs) of these ice masses are estimated, and gLGM climate is reconstructed using a simple degree–day melt model. The results indicate that, during the gLGM, ice masses occupying the Pekulney, Kankaren and Sredinny mountains of Pacific Far NE Russia were of valley glacier and ice field type. These glaciers were
between 7 and 80 km in length, and were considerably less extensive than during pre-LGM phases of advance. gLGM ice masses in these regions had ELAs of between 575± 22m and 1035±41m (above sea level) – corresponding to an ELA depression of 350–740 m, relative to present. Data indicate that, in the Pekulney Mountains, this ELA depression occurred because of a 6.48°C reduction
in mean July temperature, and 200mm a¯¹ reduction in precipitation, relative to present. Thus reconstructions support a restricted view of gLGM glaciation in Pacific Far NE Russia and indicate that the region’s aridity precluded the development of large continental ice sheets.

Comparison of progenitor mass estimates for the type IIP SN 2012A

We present the one-year long observing campaign of SN 2012A which exploded in the nearby (9.8 Mpc) irregular galaxy NGC 3239. The photometric evolution is that of a normal type IIP supernova. The absolute maximum magnitude, with MB = -16.23 +- 0.16 mag. SN2012A reached a peak luminosity of about 2X10**42 erg/s, which is brighter than those of other SNe with a similar 56Ni mass. The latter was estimated from the luminosity in the exponential tail of the light curve and found to be M(56Ni) = 0.011 +-0.004 Msun. The spectral evolution of SN 2012A is also typical of SN IIP, from the early spectra dominated by a blue continuum and very broad (~10**4 km/s) Balmer lines, to the late-photospheric spectra characterized by prominent P-Cygni features of metal lines (Fe II, Sc II, Ba II, Ti II, Ca II, Na ID). The photospheric velocity is moderately low, ~3X10**3 km/s at 50 days, for the low optical depth metal lines. The nebular spectrum obtained 394 days after the shock breakout shows the typical features of SNe IIP and the strength of the [O I] doublet suggests a progenitor of intermediate mass, similar to SN 2004et (~15 Msun). A candidate progenitor for SN 2012A has been identified in deep, pre-explosion K'-band Gemini North (NIRI) images, and found to be consistent with a star with a bolometric magnitude -7.08+-0.36 (log L/Lsun = 4.73 +- 0.14$ dex). The magnitude of the recovered progenitor in archival images points toward a moderate-mass 10.5 (-2/+4.5) Msun star as the precursor of SN 2012A. The explosion parameters and progenitor mass were also estimated by means of a hydrodynamical model, fitting the bolometric light curve, the velocity and the temperature evolution. We found a best fit for a kinetic energy of 0.48 foe, an initial radius of 1.8X10**13 cm and ejecta mass of 12.5 Msun.

Efficient Marginal Likelihood Computation for Gaussian Process Regression

In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive performance; this operation can be efficiently performed in an Empirical Bayes fashion by maximizing the posterior marginal likelihood of the observed data. Since the score function of this optimization problem is in general characterized by the presence of local optima, it is necessary to resort to global optimization strategies, which require a large number of function evaluations. Given that the evaluation is usually computationally intensive and badly scaled with respect to the dataset size, the maximum number of observations that can be treated simultaneously is quite limited. In this paper, we consider the case of hyperparameter tuning in Gaussian process regression. A straightforward implementation of the posterior log-likelihood for this model requires O(N^3) operations for every iteration of the optimization procedure, where N is the number of examples in the input dataset. We derive a novel set of identities that allow, after an initial overhead of O(N^3), the evaluation of the score function, as well as the Jacobian and Hessian matrices, in O(N) operations. We prove how the proposed identities, that follow from the eigendecomposition of the kernel matrix, yield a reduction of several orders of magnitude in the computation time for the hyperparameter optimization problem. Notably, the proposed solution provides computational advantages even with respect to state of the art approximations that rely on sparse kernel matrices.

Belief Updating and Learning in Semi-Qualitative Probabilistic Networks

This paper explores semi-qualitative probabilistic networks (SQPNs) that combine numeric and qualitative information. We first show that exact inferences with SQPNs are NPPP-Complete. We then show that existing qualitative relations in SQPNs (plus probabilistic logic and imprecise assessments) can be dealt effectively through multilinear programming. We then discuss learning: we consider a maximum likelihood method that generates point estimates given a SQPN and empirical data, and we describe a Bayesian-minded method that employs the Imprecise Dirichlet Model to generate set-valued estimates.