Bayesian parameter estimation for latent Markov random fields and social networks

Undirected graphical models are widely used in statistics, physics and machine vision. However Bayesian parameter estimation for undirected models is extremely challenging, since evaluation of the posterior typically involves the calculation of an intractable normalising constant. This problem has received much attention, but very little of this has focussed on the important practical case where the data consists of noisy or incomplete observations of the underlying hidden structure. This paper specifically addresses this problem, comparing two alternative methodologies. In the first of these approaches particle Markov chain Monte Carlo (Andrieu et al., 2010) is used to efficiently explore the parameter space, combined with the exchange algorithm (Murray et al., 2006) for avoiding the calculation of the intractable normalising constant (a proof showing that this combination targets the correct distribution in found in a supplementary appendix online). This approach is compared with approximate Bayesian computation (Pritchard et al., 1999). Applications to estimating the parameters of Ising models and exponential random graphs from noisy data are presented. Each algorithm used in the paper targets an approximation to the true posterior due to the use of MCMC to simulate from the latent graphical model, in lieu of being able to do this exactly in general. The supplementary appendix also describes the nature of the resulting approximation.

Parameter spaces of environmental fields responsible for cyclone development from tropics to extratropics

Objective cyclone tracking applied to a 30-yr reanalysis dataset shows that cyclone development in the summer and autumn seasons is active in the tropics and extratropics and inactive in the subtropics. To understand this geographically bimodal distribution of cyclone development associated with tropical and extratropical cyclones quantitatively, the direct relationship between cyclone types and their environments are assessed by using a parameter space of environmental variables [environmental parameter space (EPS)]. The number of cyclones is analyzed in terms of two different factors: the environmental conditions favorable for cyclone development and the area size that satisfies the favorable condition. The EPS analysis is mainly conducted for two representative environmental parameters that are commonly used for cyclone analysis: potential intensity for tropical cyclones and baroclinicity for extratropical cyclones. The geographically bimodal distribution is attributed to the high sensitivity of the cyclone development to the change in the environmental fields from tropics to extratropics. In addition, the bimodal distribution is partly attributed to the rapid change in the environmental fields from tropics to extratropics. The EPS analysis also shows that other environmental parameters, including relative humidity and vertical velocity, may enhance the contrast between the tropics (extratropics) and subtropics, whereas they are not essential for determining cyclone types. The relationship between cyclones and their environments is found to be similar between the hemispheres in the EPS, although the geographical distribution, particularly the longitudinal uniformity, is markedly different between the hemispheres.

An application of the GLUE methodology for estimating the parameters of the INCA-N model

The conceptual and parameter uncertainty of the semi-distributed INCA-N (Integrated Nutrients in Catchments-Nitrogen) model was studied using the GLUE (Generalized Likelihood Uncertainty Estimation) methodology combined with quantitative experimental knowledge, the concept known as 'soft data'. Cumulative inorganic N leaching, annual plant N uptake and annual mineralization proved to be useful soft data to constrain the parameter space. The INCA-N model was able to simulate the seasonal and inter-annual variations in the stream-water nitrate concentrations, although the lowest concentrations during the growing season were not reproduced. This suggested that there were some retention processes or losses either in peatland/wetland areas or in the river which were not included in the INCA-N model. The results of the study suggested that soft data was a way to reduce parameter equifinality, and that the calibration and testing of distributed hydrological and nutrient leaching models should be based both on runoff and/or nutrient concentration data and the qualitative knowledge of experimentalist. (c) 2006 Elsevier B.V. All rights reserved.

Enhancing dominant modes in nonstationary time series by means of the symbolic resonance analysis

We present the symbolic resonance analysis (SRA) as a viable method for addressing the problem of enhancing a weakly dominant mode in a mixture of impulse responses obtained from a nonlinear dynamical system. We demonstrate this using results from a numerical simulation with Duffing oscillators in different domains of their parameter space, and by analyzing event-related brain potentials (ERPs) from a language processing experiment in German as a representative application. In this paradigm, the averaged ERPs exhibit an N400 followed by a sentence final negativity. Contemporary sentence processing models predict a late positivity (P600) as well. We show that the SRA is able to unveil the P600 evoked by the critical stimuli as a weakly dominant mode from the covering sentence final negativity. (c) 2007 American Institute of Physics. (c) 2007 American Institute of Physics.

Non-testability of equal weights spatial dependence

We show that any invariant test for spatial autocorrelation in a spatial error or spatial lag model with equal weights matrix has power equal to size. This result holds under the assumption of an elliptical distribution. Under Gaussianity, we also show that any test whose power is larger than its size for at least one point in the parameter space must be biased.

Comparing a discrete and continuum model of the intestinal crypt

The integration of processes at different scales is a key problem in the modelling of cell populations. Owing to increased computational resources and the accumulation of data at the cellular and subcellular scales, the use of discrete, cell-level models, which are typically solved using numerical simulations, has become prominent. One of the merits of this approach is that important biological factors, such as cell heterogeneity and noise, can be easily incorporated. However, it can be difficult to efficiently draw generalizations from the simulation results, as, often, many simulation runs are required to investigate model behaviour in typically large parameter spaces. In some cases, discrete cell-level models can be coarse-grained, yielding continuum models whose analysis can lead to the development of insight into the underlying simulations. In this paper we apply such an approach to the case of a discrete model of cell dynamics in the intestinal crypt. An analysis of the resulting continuum model demonstrates that there is a limited region of parameter space within which steady-state (and hence biologically realistic) solutions exist. Continuum model predictions show good agreement with corresponding results from the underlying simulations and experimental data taken from murine intestinal crypts.

Dynamic Niche Clustering: a fuzzy variable radius niching technique for multimodal optimisation in GAs

This paper describes the recent developments and improvements made to the variable radius niching technique called Dynamic Niche Clustering (DNC). DNC is fitness sharing based technique that employs a separate population of overlapping fuzzy niches with independent radii which operate in the decoded parameter space, and are maintained alongside the normal GA population. We describe a speedup process that can be applied to the initial generation which greatly reduces the complexity of the initial stages. A split operator is also introduced that is designed to counteract the excessive growth of niches, and it is shown that this improves the overall robustness of the technique. Finally, the effect of local elitism is documented and compared to the performance of the basic DNC technique on a selection of 2D test functions. The paper is concluded with a view to future work to be undertaken on the technique.

The correlation structure of spatial autoregressions

This paper investigates how the correlations implied by a first-order simultaneous autoregressive (SAR(1)) process are affected by the weights matrix and the autocorrelation parameter. A graph theoretic representation of the covariances in terms of walks connecting the spatial units helps to clarify a number of correlation properties of the processes. In particular, we study some implications of row-standardizing the weights matrix, the dependence of the correlations on graph distance, and the behavior of the correlations at the extremes of the parameter space. Throughout the analysis differences between directed and undirected networks are emphasized. The graph theoretic representation also clarifies why it is difficult to relate properties ofW to correlation properties of SAR(1) models defined on irregular lattices.

Competing edge networks

We introduce a model for a pair of nonlinear evolving networks, defined over a common set of vertices, sub ject to edgewise competition. Each network may grow new edges spontaneously or through triad closure. Both networks inhibit the other’s growth and encourage the other’s demise. These nonlinear stochastic competition equations yield to a mean field analysis resulting in a nonlinear deterministic system. There may be multiple equilibria; and bifurcations of different types are shown to occur within a reduced parameter space. This situation models competitive peer-to-peer communication networks such as BlackBerry Messenger displacing SMS; or instant messaging displacing emails.

Likelihood-free estimation of model evidence

Statistical methods of inference typically require the likelihood function to be computable in a reasonable amount of time. The class of “likelihood-free” methods termed Approximate Bayesian Computation (ABC) is able to eliminate this requirement, replacing the evaluation of the likelihood with simulation from it. Likelihood-free methods have gained in efficiency and popularity in the past few years, following their integration with Markov Chain Monte Carlo (MCMC) and Sequential Monte Carlo (SMC) in order to better explore the parameter space. They have been applied primarily to estimating the parameters of a given model, but can also be used to compare models. Here we present novel likelihood-free approaches to model comparison, based upon the independent estimation of the evidence of each model under study. Key advantages of these approaches over previous techniques are that they allow the exploitation of MCMC or SMC algorithms for exploring the parameter space, and that they do not require a sampler able to mix between models. We validate the proposed methods using a simple exponential family problem before providing a realistic problem from human population genetics: the comparison of different demographic models based upon genetic data from the Y chromosome.

The long-term stability of a possible aqueous ammonium sulfate ocean inside Titan

We model the thermal evolution of a subsurface ocean of aqueous ammonium sulfate inside Titan using a parameterized convection scheme. The cooling and crystallization of such an ocean depends on its heat flux balance, and is governed by the pressure-dependent melting temperatures at the top and bottom of the ocean. Using recent observations and previous experimental data, we present a nominal model which predicts the thickness of the ocean throughout the evolution of Titan; after 4.5 Ga we expect an aqueous ammonium sulfate ocean 56 km thick, overlain by a thick (176 km) heterogeneous crust of methane clathrate, ice I and ammonium sulfate. Underplating of the crust by ice I will give rise to compositional diapirs that are capable of rising through the crust and providing a mechanism for cryovolcanism at the surface. We have conducted a parameter space survey to account for possible variations in the nominal model, and find that for a wide range of plausible conditions, an ocean of aqueous ammonium sulfate can survive to the present day, which is consistent with the recent observations of Titan's spin state from Cassini radar data [Lorenz, R.D., Stiles, B.W., Kirk, R.L., Allison, M.D., del Marmo, P.P., Iess, L., Lunine, J.I., Ostro, S.J., Hensley, S., 2008. Science 319, 1649–1651].

Femtosecond pulse shaping using differential evolutionary algorithm and wavelet operators

Evolutionary meta-algorithms for pulse shaping of broadband femtosecond duration laser pulses are proposed. The genetic algorithm searching the evolutionary landscape for desired pulse shapes consists of a population of waveforms (genes), each made from two concatenated vectors, specifying phases and magnitudes, respectively, over a range of frequencies. Frequency domain operators such as mutation, two-point crossover average crossover, polynomial phase mutation, creep and three-point smoothing as well as a time-domain crossover are combined to produce fitter offsprings at each iteration step. The algorithm applies roulette wheel selection; elitists and linear fitness scaling to the gene population. A differential evolution (DE) operator that provides a source of directed mutation and new wavelet operators are proposed. Using properly tuned parameters for DE, the meta-algorithm is used to solve a waveform matching problem. Tuning allows either a greedy directed search near the best known solution or a robust search across the entire parameter space.

Metabifurcation analysis of a mean field model of the cortex

Mean field models (MFMs) of cortical tissue incorporate salient, average features of neural masses in order to model activity at the population level, thereby linking microscopic physiology to macroscopic observations, e.g., with the electroencephalogram (EEG). One of the common aspects of MFM descriptions is the presence of a high-dimensional parameter space capturing neurobiological attributes deemed relevant to the brain dynamics of interest. We study the physiological parameter space of a MFM of electrocortical activity and discover robust correlations between physiological attributes of the model cortex and its dynamical features. These correlations are revealed by the study of bifurcation plots, which show that the model responses to changes in inhibition belong to two archetypal categories or “families”. After investigating and characterizing them in depth, we discuss their essential differences in terms of four important aspects: power responses with respect to the modeled action of anesthetics, reaction to exogenous stimuli such as thalamic input, and distributions of model parameters and oscillatory repertoires when inhibition is enhanced. Furthermore, while the complexity of sustained periodic orbits differs significantly between families, we are able to show how metamorphoses between the families can be brought about by exogenous stimuli. We here unveil links between measurable physiological attributes of the brain and dynamical patterns that are not accessible by linear methods. They instead emerge when the nonlinear structure of parameter space is partitioned according to bifurcation responses. We call this general method “metabifurcation analysis”. The partitioning cannot be achieved by the investigation of only a small number of parameter sets and is instead the result of an automated bifurcation analysis of a representative sample of 73,454 physiologically admissible parameter sets. Our approach generalizes straightforwardly and is well suited to probing the dynamics of other models with large and complex parameter spaces.

Dynamical complexity in a mean-field model of human EEG

A recently proposed mean-field theory of mammalian cortex rhythmogenesis describes the salient features of electrical activity in the cerebral macrocolumn, with the use of inhibitory and excitatory neuronal populations (Liley et al 2002). This model is capable of producing a range of important human EEG (electroencephalogram) features such as the alpha rhythm, the 40 Hz activity thought to be associated with conscious awareness (Bojak & Liley 2007) and the changes in EEG spectral power associated with general anesthetic effect (Bojak & Liley 2005). From the point of view of nonlinear dynamics, the model entails a vast parameter space within which multistability, pseudoperiodic regimes, various routes to chaos, fat fractals and rich bifurcation scenarios occur for physiologically relevant parameter values (van Veen & Liley 2006). The origin and the character of this complex behaviour, and its relevance for EEG activity will be illustrated. The existence of short-lived unstable brain states will also be discussed in terms of the available theoretical and experimental results. A perspective on future analysis will conclude the presentation.

Orographic drag associated with lee waves trapped at an inversion

The drag produced by 2D orographic gravity waves trapped at a temperature inversion and waves propagating in the stably stratified layer existing above are explicitly calculated using linear theory, for a two-layer atmosphere with neutral static stability near the surface, mimicking a well-mixed boundary layer. For realistic values of the flow parameters, trapped lee wave drag, which is given by a closed analytical expression, is comparable to propagating wave drag, especially in moderately to strongly non-hydrostatic conditions. In resonant flow, both drag components substantially exceed the single-layer hydrostatic drag estimate used in most parametrization schemes. Both drag components are optimally amplified for a relatively low-level inversion and Froude numbers Fr ≈ 1. While propagating wave drag is maximized for approximately hydrostatic flow, trapped lee wave drag is maximized for l_2 a = O(1) (where l_2 is the Scorer parameter in the stable layer and a is the mountain width). This roughly happens when the horizontal scale of trapped lee waves matches that of the mountain slope. The drag behavior as a function of Fr for l_2 H = 0.5 (where H is the inversion height) and different values of l2a shows good agreement with numerical simulations. Regions of parameter space with high trapped lee wave drag correlate reasonably well with those where lee wave rotors were found to occur in previous nonlinear numerical simulations including frictional effects. This suggests that trapped lee wave drag, besides giving a relevant contribution to low-level drag exerted on the atmosphere, may also be useful to diagnose lee rotor formation.