Bayesian experimental design for models with intractable likelihoods

In this paper we present a methodology for designing experiments for efficiently estimating the parameters of models with computationally intractable likelihoods. The approach combines a commonly used methodology for robust experimental design, based on Markov chain Monte Carlo sampling, with approximate Bayesian computation (ABC) to ensure that no likelihood evaluations are required. The utility function considered for precise parameter estimation is based upon the precision of the ABC posterior distribution, which we form efficiently via the ABC rejection algorithm based on pre-computed model simulations. Our focus is on stochastic models and, in particular, we investigate the methodology for Markov process models of epidemics and macroparasite population evolution. The macroparasite example involves a multivariate process and we assess the loss of information from not observing all variables.

Shrinkage empirical likelihood estimator in longitudinal analysis with time-dependent covariates-application to modeling the health of Filipino children

The method of generalized estimating equations (GEE) is a popular tool for analysing longitudinal (panel) data. Often, the covariates collected are time-dependent in nature, for example, age, relapse status, monthly income. When using GEE to analyse longitudinal data with time-dependent covariates, crucial assumptions about the covariates are necessary for valid inferences to be drawn. When those assumptions do not hold or cannot be verified, Pepe and Anderson (1994, Communications in Statistics, Simulations and Computation 23, 939–951) advocated using an independence working correlation assumption in the GEE model as a robust approach. However, using GEE with the independence correlation assumption may lead to significant efficiency loss (Fitzmaurice, 1995, Biometrics 51, 309–317). In this article, we propose a method that extracts additional information from the estimating equations that are excluded by the independence assumption. The method always includes the estimating equations under the independence assumption and the contribution from the remaining estimating equations is weighted according to the likelihood of each equation being a consistent estimating equation and the information it carries. We apply the method to a longitudinal study of the health of a group of Filipino children.

Understanding the abnormal brain activity in epilepsy as a potential predictor of the onset of an epileptic seizure

The Brain Research Institute (BRI) uses various types of indirect measurements, including EEG and fMRI, to understand and assess brain activity and function. As well as the recovery of generic information about brain function, research also focuses on the utilisation of such data and understanding to study the initiation, dynamics, spread and suppression of epileptic seizures. To assist with the future focussing of this aspect of their research, the BRI asked the MISG 2010 participants to examine how the available EEG and fMRI data and current knowledge about epilepsy should be analysed and interpreted to yield an enhanced understanding about brain activity occurring before, at commencement of, during, and after a seizure. Though the deliberations of the study group were wide ranging in terms of the related matters considered and discussed, considerable progress was made with the following three aspects. (1) The science behind brain activity investigations depends crucially on the quality of the analysis and interpretation of, as well as the recovery of information from, EEG and fMRI measurements. A number of specific methodologies were discussed and formalised, including independent component analysis, principal component analysis, profile monitoring and change point analysis (hidden Markov modelling, time series analysis, discontinuity identification). (2) Even though EEG measurements accurately and very sensitively record the onset of an epileptic event or seizure, they are, from the perspective of understanding the internal initiation and localisation, of limited utility. They only record neuronal activity in the cortical (surface layer) neurons of the brain, which is a direct reflection of the type of electrical activity they have been designed to record. Because fMRI records, through the monitoring of blood flow activity, the location of localised brain activity within the brain, the possibility of combining fMRI measurements with EEG, as a joint inversion activity, was discussed and examined in detail. (3) A major goal for the BRI is to improve understanding about ``when'' (at what time) an epileptic seizure actually commenced before it is identified on an eeg recording, ``where'' the source of this initiation is located in the brain, and ``what'' is the initiator. Because of the general agreement in the literature that, in one way or another, epileptic events and seizures represent abnormal synchronisations of localised and/or global brain activity the modelling of synchronisations was examined in some detail. References C. M. Michel, G. Thut, S. Morand, A. Khateb, A. J. Pegna, R. Grave de Peralta, S. Gonzalez, M. Seeck and T. Landis, Electric source imaging of human brain functions, Brain Res. Rev. , 36 (2--3), 2001, 108--118. doi:10.1016/S0165-0173(01)00086-8 S. Ogawa, R. S. Menon, S. G. Kim and K. Ugurbil, On the characteristics of functional magnetic resonance imaging of the brain, Annu. Rev. Bioph. Biom. , 27 , 1998, 447--474. doi:10.1146/annurev.biophys.27.1.447 C. D. Binnie and H. Stefan, Modern electroencephalography: its role in epilepsy management, Clin. Neurophysiol. , 110 (10), 1999, 1671--1697. doi:10.1016/S1388-2457(99)00125-X J. X. Tao, A. Ray, S. Hawes-Ebersole and J. S. Ebersole, Intracranial eeg substrates of scalp eeg interictal spikes, Epilepsia , 46 (5), 2005, 669--76. doi:10.1111/j.1528-1167.2005.11404.x S. Ogawa, D. W. Tank, R. Menon, J. M. Ellermann, S. G. Kim, H. Merkle and K. Ugurbil, Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging, P. Natl. Acad. Sci. USA , 89 (13), 1992, 5951--5955. doi:10.1073/pnas.89.13.5951 J. Engel Jr., Report of the ilae classification core group, Epilepsia , 47 (9), 2006, 1558--1568. doi:10.1111/j.1528-1167.2006.00215.x L. Lemieux, A. Salek-Haddadi, O. Josephs, P. Allen, N. Toms, C. Scott, K. Krakow, R. Turner and D. R. Fish, Event-related fmri with simultaneous and continuous eeg: description of the method and initial case r port, NeuroImage , 14 (3), 2001, 780--7. doi:10.1006/nimg.2001.0853 P. Federico, D. F. Abbott, R. S. Briellmann, A. S. Harvey and G. D. Jackson, Functional mri of the pre-ictal state, Brain , 128 (8), 2005, 1811-7. doi:10.1093/brain/awh533 C. S. Hawco, A. P. Bagshaw, Y. Lu, F. Dubeau and J. Gotman, bold changes occur prior to epileptic spikes seen on scalp eeg, NeuroImage , 35 (4), 2007, 1450--1458. doi:10.1016/j.neuroimage.2006.12.042 F. Moeller, H. R. Siebner, S. Wolff, H. Muhle, R. Boor, O. Granert, O. Jansen, U. Stephani and M. Siniatchkin, Changes in activity of striato-thalamo-cortical network precede generalized spike wave discharges, NeuroImage , 39 (4), 2008, 1839--1849. doi:10.1016/j.neuroimage.2007.10.058 V. Osharina, E. Ponchel, A. Aarabi, R. Grebe and F. Wallois, Local haemodynamic changes preceding interictal spikes: A simultaneous electrocorticography (ecog) and near-infrared spectroscopy (nirs) analysis in rats, NeuroImage , 50 (2), 2010, 600--607. doi:10.1016/j.neuroimage.2010.01.009 R. S. Fisher, W. Boas, W. Blume, C. Elger, P. Genton, P. Lee and J. Engel, Epileptic seizures and epilepsy: Definitions proposed by the international league against epilepsy (ilae) and the international bureau for epilepsy (ibe), Epilepsia , 46 (4), 2005, 470--472. doi:10.1111/j.0013-9580.2005.66104.x H. Berger, Electroencephalogram in humans, Arch. Psychiat. Nerven. , 87 , 1929, 527--570. C. M. Michel, M. M. Murray, G. Lantz, S. Gonzalez, L. Spinelli and R. G. de Peralta, eeg source imaging, Clin. Neurophysiol. , 115 (10), 2004, 2195--2222. doi:10.1016/j.clinph.2004.06.001 P. L. Nunez and R. B. Silberstein, On the relationship of synaptic activity to macroscopic measurements: Does co-registration of eeg with fmri make sense?, Brain Topogr. , 13 (2), 2000, 79--96. doi:10.1023/A:1026683200895 S. Ogawa, T. M. Lee, A. R. Kay and D. W. Tank, Brain magnetic resonance imaging with contrast dependent on blood oxygenation, P. Natl. Acad. Sci. USA , 87 (24), 1990, 9868--9872. doi:10.1073/pnas.87.24.9868 J. S. Gati, R. S. Menon, K. Ugurbil and B. K. Rutt, Experimental determination of the bold field strength dependence in vessels and tissue, Magn. Reson. Med. , 38 (2), 1997, 296--302. doi:10.1002/mrm.1910380220 P. A. Bandettini, E. C. Wong, R. S. Hinks, R. S. Tikofsky and J. S. Hyde, Time course EPI of human brain function during task activation, Magn. Reson. Med. , 25 (2), 1992, 390--397. K. K. Kwong, J. W. Belliveau, D. A. Chesler, I. E. Goldberg, R. M. Weisskoff, B. P. Poncelet, D. N. Kennedy, B. E. Hoppelm, M. S. Cohen and R. Turner, Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation, P. Natl. Acad. Sci. USA , 89 (12), 1992, 5675--5679. doi:10.1073/pnas.89.12.5675 J. Frahm, K. D. Merboldt and W. Hnicke, Functional mri of human brain activation at high spatial resolution, Magn. Reson. Med. , 29 (1), 1993, 139--144. P. A. Bandettini, A. Jesmanowicz, E. C. Wong and J. S. Hyde, Processing strategies for time-course data sets in functional MRI of the human brain, Magn. Reson. Med. , 30 (2), 1993, 161--173. K. J. Friston, P. Jezzard and R. Turner, Analysis of functional MRI time-series, Hum. Brain Mapp. , 1 (2), 1994, 153--171. B. Biswal, F. Z. Yetkin, V. M. Haughton and J. S. Hyde, Functional connectivity in the motor cortex of resting human brain using echo-planar mri, Mag. Reson. Med. , 34 (4), 1995, 537--541. doi:10.1002/mrm.1910340409 K. J. Friston, J. Ashburner, C. D. Frith, J. Poline, J. D. Heather and R. S. J. Frackowiak, Spatial registration and normalization of images, Hum. Brain Mapp. , 3 (3), 1995, 165--189. K. J. Friston, S. Williams, R. Howard, R. S. Frackowiak and R. Turner, Movement-related effects in fmri time-series, Magn. Reson. Med. , 35 (3), 1996, 346--355. G. H. Glover, T. Q. Li and D. Ress, Image-based method for retrospective correction of physiological motion effects in fmri: Retroicor, Magn. Reson. Med. , 44 (1), 2000, 162--167. doi:10.1002/1522-2594(200007)44:13.0.CO;2-E K. J. Friston, O. Josephs, G. Rees and R. Turner, Nonlinear event-related responses in fmri, Magn. Reson. Med. , 39 (1), 1998, 41--52. doi:10.1002/mrm.1910390109 K. Ugurbil, L. Toth and D. Kim, How accurate is magnetic resonance imaging of brain function?, Trends Neurosci. , 26 (2), 2003, 108--114. doi:10.1016/S0166-2236(02)00039-5 D. S. Kim, I. Ronen, C. Olman, S. G. Kim, K. Ugurbil and L. J. Toth, Spatial relationship between neuronal activity and bold functional mri, NeuroImage , 21 (3), 2004, 876--885. doi:10.1016/j.neuroimage.2003.10.018 A. Connelly, G. D. Jackson, R. S. Frackowiak, J. W. Belliveau, F. Vargha-Khadem and D. G. Gadian, Functional mapping of activated human primary cortex with a clinical mr imaging system, Radiology , 188 (1), 1993, 125--130. L. Allison, Hidden Markov Models, Technical Report , School of Computer and Software Engineering, Monash University, 2000. R. J. Elliott, L. Aggoun and J.B. Moore, Hidden Markov Models: Estimation and Control, Appl. Math.-Czech. , 2004. B. Bhavnagri, Discontinuities of plane functions projected from a surface with methods for finding these , Technical Report, 2009. B. Bhavnagri, Computer Vision using Shape Spaces , Technical Report,1996, University of Adelaide. B. Bhavnagri, A method for representing shape based on an equivalence relation on polygons, Pattern Recogn. , 27 (2), 1994, 247--260. doi:10.1016/0031-3203(94)90057-4 D. F. Abbott, A. B. Waites, A. S. Harvey and G. D. Jackson, Exploring epileptic seizure onset with fmri, NeuroImage , 36(S1) (344TH-PM), 2007. M. C. Mackey and L. Glass, Oscillation and chaos in physiological control systems, Science , 197 , 1977, 287--289. S. H. Strogatz, SYNC - The Emerging Science of Spontaneous Order , Theia, New York, 2003. J. W. Kim, J. A. Roberts and P. A. Robinson, Dynamics of epileptic seizures: Evolution, spreading, and suppression, J. Theor. Biol. , 257 (4), 2009, 527--532. doi:10.1016/j.jtbi.2008.12.009 Y. Kuramoto, T. Aoyagi, I. Nishikawa, T. Chawanya T and K. Okuda, Neural network model carrying phase information with application to collective dynamics, J. Theor. Phys. , 87 (5), 1992, 1119--1126. V. B. Mountcastle, The columnar organization of the neocortex, Brain , 120 (4), 1997, 701. doi:10.1093/brain/120.4.701 F. L. Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Epilepsies as dynamical diseases of brain systems: Basic models of the transition between normal and epileptic activity, Epilepsia , 44 (12), 2003, 72--83. F. H. Lopes da Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Dynamical diseases of brain systems: different routes to epileptic seizures, ieee T. Bio-Med. Eng. , 50 (5), 2003, 540. L.D. Iasemidis, Epileptic seizure prediction and control, ieee T. Bio-Med. Eng. , 50 (5), 2003, 549--558. L. D. Iasemidis, D. S. Shiau, W. Chaovalitwongse, J. C. Sackellares, P. M. Pardalos, J. C. Principe, P. R. Carney, A. Prasad, B. Veeramani, and K. Tsakalis, Adaptive epileptic seizure prediction system, ieee T. Bio-Med. Eng. , 50 (5), 2003, 616--627. K. Lehnertz, F. Mormann, T. Kreuz, R.G. Andrzejak, C. Rieke, P. David and C. E. Elger, Seizure prediction by nonlinear eeg analysis, ieee Eng. Med. Biol. , 22 (1), 2003, 57--63. doi:10.1109/MEMB.2003.1191451 K. Lehnertz, R. G. Andrzejak, J. Arnhold, T. Kreuz, F. Mormann, C. Rieke, G. Widman and C. E. Elger, Nonlinear eeg analysis in epilepsy: Its possible use for interictal focus localization, seizure anticipation, and prevention, J. Clin. Neurophysiol. , 18 (3), 2001, 209. B. Litt and K. Lehnertz, Seizure prediction and the preseizure period, Curr. Opin. Neurol. , 15 (2), 2002, 173. doi:10.1097/00019052-200204000-00008 B. Litt and J. Echauz, Prediction of epileptic seizures, Lancet Neurol. , 1 (1), 2002, 22--30. doi:10.1016/S1474-4422(02)00003-0 M. M{a}kiranta, J. Ruohonen, K Suominen, J. Niinim{a}ki, E. Sonkaj{a}rvi, V. Kiviniemi, T. Sepp{a}nen, S. Alahuhta, V. J{a}ntti and O. Tervonen, {bold} signal increase preceeds eeg spike activity--a dynamic penicillin induced focal epilepsy in deep anesthesia, NeuroImage , 27 (4), 2005, 715--724. doi:10.1016/j.neuroimage.2005.05.025 K. Lehnertz, F. Mormann, H. Osterhage, A. M{u}ller, J. Prusseit, A. Chernihovskyi, M. Staniek, D. Krug, S. Bialonski and C. E. Elger, State-of-the-art of seizure prediction, J. Clin. Neurophysiol. , 24 (2), 2007, 147. doi:10.1097/WNP.0b013e3180336f16 F. Mormann, T. Kreuz, C. Rieke, R. G. Andrzejak, A. Kraskov, P. David, C. E. Elger and K. Lehnertz, On the predictability of epileptic seizures, Clin. Neurophysiol. , 116 (3), 2005, 569--587. doi:10.1016/j.clinph.2004.08.025 F. Mormann, R. G. Andrzejak, C. E. Elger and K. Lehnertz, Seizure prediction: the long and winding road, Brain , 130 (2), 2007, 314--333. doi:10.1093/brain/awl241 Z. Rogowski, I. Gath and E. Bental, On the prediction of epileptic seizures, Biol. Cybern. , 42 (1), 1981, 9--15. Y. Salant, I. Gath, O. Henriksen, Prediction of epileptic seizures from two-channel eeg, Med. Biol. Eng. Comput. , 36 (5), 1998, 549--556. doi:10.1007/BF02524422 J. Gotman and D.J. Koffler, Interictal spiking increases after seizures but does not after decrease in medication, Evoked Potential , 72 (1), 1989, 7--15. J. Gotman and M. G. Marciani, Electroencephalographic spiking activity, drug levels, and seizure occurence in epileptic patients, Ann. Neurol. , 17 (6), 1985, 59--603. A. Katz, D. A. Marks, G. McCarthy and S. S. Spencer, Does interictal spiking change prior to seizures?, Electroen. Clin. Neuro. , 79 (2), 1991, 153--156. A. Granada, R. M. Hennig, B. Ronacher, A. Kramer and H. Herzel, Phase Response Curves: Elucidating the dynamics of couples oscillators, Method Enzymol. , 454 (A), 2009, 1--27. doi:10.1016/S0076-6879(08)03801-9 doi:10.1016/S0076-6879(08)03801-9 H. Kantz and T. Schreiber, Nonlinear time series analysis , 2004, Cambridge Univ Press. M. V. L. Bennett and R. S Zukin, Electrical coupling and neuronal synchronization in the mammalian brain, Neuron , 41 (4), 2004, 495 --511. doi:10.1016/S0896-6273(04)00043-1 L.D. Iasemidis, J. Chris Sackellares, H. P. Zaveri and W. J. Williams, Phase space topography and the Lyapunov exponent of electrocorticograms in partial seizures, Brain Topogr. , 2 (3), 1990, 187--201. doi:10.1007/BF01140588 M. Le Van Quyen, J. Martinerie, V. Navarro, M. Baulac and F. J. Varela, Characterizing neurodynamic changes before seizures, J. Clin. Neurophysiol. , 18 (3), 2001, 191. J. Martinerie, C. Adam, M. Le Van Quyen, M. Baulac, S. Clemenceau, B. Renault and F. J. Varela, Epileptic seizures can be anticipated by non-linear analysis, Nat. Med. , 4 (10), 1998, 1173--1176. doi:10.1038/2667 A. Pikovsky, M. Rosenblum, J. Kurths and R. C. Hilborn, Synchronization: A universal concept in nonlinear science, Amer. J. Phys. , 70 , 2002, 655. H. R. Wilson and J. D. Cowan, Excitatory and inhibitory interactions in localized populations of model neurons, Biophys. J. , 12 (1), 1972, 1--24. D. Cumin and C. P. Unsworth, Generalising the Kuramoto model for the study of neuronal synchronisation in the brain, Physica D , 226 (2), 2007, 181--196. doi:10.1016/j.physd.2006.12.004 F. K. Skinner, H. Bazzazi and S. A. Campbell, Two-cell to N-cell heterogeneous, inhibitory networks: Precise linking of multistable and coherent properties, J. Comput. Neurosci. , 18 (3), 2005, 343--352. doi:10.1007/s10827-005-0331-1 W. W. Lytton, Computer modelling of epilepsy, Nat. Rev. Neurosci. , 9 (8), 2008, 626--637. doi:10.1038/nrn2416 R. D. Traub, A. Bibbig, F. E. N. LeBeau, E. H. Buhl and M. A. Whittington, Cellular mechanisms of neuronal population oscillations in the hippocampus in vitro, Ann. Rev. , 2004. R. D. Traub, A. Draguhn, M. A. Whittington, T. Baldeweg, A. Bibbig, E. H. Buhl and D. Schmitz, Axonal gap junc ions between principal neurons: A novel source of network oscillations, and perhaps epileptogenesis., Rev. Neuroscience , 13 (1), 2002, 1. doi:10.1146/annurev.neuro.27.070203.144303 M. Scheffer, J. Bascompte, W. A. Brock, V. Brovkin, S. R. Carpenter, V. Dakos, H. Held, E. H. van Nes, M. Rietkerk and G. Sugihara, Early-warning signals for critical transitions, Nature , 461 (7260), 2009, 53--59. doi:10.1038/nature08227 K. Murphy, A Brief Introduction to Graphical Models and Bayesian Networks , 2008, http://www.cs.ubc.ca/murphyk/Bayes/bnintro.html . R. C. Bradley, An elementary

Model choice problems using approximate Bayesian computation with applications to pathogen transmission data sets

Analytically or computationally intractable likelihood functions can arise in complex statistical inferential problems making them inaccessible to standard Bayesian inferential methods. Approximate Bayesian computation (ABC) methods address such inferential problems by replacing direct likelihood evaluations with repeated sampling from the model. ABC methods have been predominantly applied to parameter estimation problems and less to model choice problems due to the added difficulty of handling multiple model spaces. The ABC algorithm proposed here addresses model choice problems by extending Fearnhead and Prangle (2012, Journal of the Royal Statistical Society, Series B 74, 1–28) where the posterior mean of the model parameters estimated through regression formed the summary statistics used in the discrepancy measure. An additional stepwise multinomial logistic regression is performed on the model indicator variable in the regression step and the estimated model probabilities are incorporated into the set of summary statistics for model choice purposes. A reversible jump Markov chain Monte Carlo step is also included in the algorithm to increase model diversity for thorough exploration of the model space. This algorithm was applied to a validating example to demonstrate the robustness of the algorithm across a wide range of true model probabilities. Its subsequent use in three pathogen transmission examples of varying complexity illustrates the utility of the algorithm in inferring preference of particular transmission models for the pathogens.

Alive SMC^2: Bayesian model selection for low-count time series models with intractable likelihoods

In this paper we present a new method for performing Bayesian parameter inference and model choice for low count time series models with intractable likelihoods. The method involves incorporating an alive particle filter within a sequential Monte Carlo (SMC) algorithm to create a novel pseudo-marginal algorithm, which we refer to as alive SMC^2. The advantages of this approach over competing approaches is that it is naturally adaptive, it does not involve between-model proposals required in reversible jump Markov chain Monte Carlo and does not rely on potentially rough approximations. The algorithm is demonstrated on Markov process and integer autoregressive moving average models applied to real biological datasets of hospital-acquired pathogen incidence, animal health time series and the cumulative number of poison disease cases in mule deer.