Bayesian algorithms with applications

Advances in algorithms for approximate sampling from a multivariable target function have led to solutions to challenging statistical inference problems that would otherwise not be considered by the applied scientist. Such sampling algorithms are particularly relevant to Bayesian statistics, since the target function is the posterior distribution of the unobservables given the observables. In this thesis we develop, adapt and apply Bayesian algorithms, whilst addressing substantive applied problems in biology and medicine as well as other applications. For an increasing number of high-impact research problems, the primary models of interest are often sufficiently complex that the likelihood function is computationally intractable. Rather than discard these models in favour of inferior alternatives, a class of Bayesian "likelihoodfree" techniques (often termed approximate Bayesian computation (ABC)) has emerged in the last few years, which avoids direct likelihood computation through repeated sampling of data from the model and comparing observed and simulated summary statistics. In Part I of this thesis we utilise sequential Monte Carlo (SMC) methodology to develop new algorithms for ABC that are more efficient in terms of the number of model simulations required and are almost black-box since very little algorithmic tuning is required. In addition, we address the issue of deriving appropriate summary statistics to use within ABC via a goodness-of-fit statistic and indirect inference. Another important problem in statistics is the design of experiments. That is, how one should select the values of the controllable variables in order to achieve some design goal. The presences of parameter and/or model uncertainty are computational obstacles when designing experiments but can lead to inefficient designs if not accounted for correctly. The Bayesian framework accommodates such uncertainties in a coherent way. If the amount of uncertainty is substantial, it can be of interest to perform adaptive designs in order to accrue information to make better decisions about future design points. This is of particular interest if the data can be collected sequentially. In a sense, the current posterior distribution becomes the new prior distribution for the next design decision. Part II of this thesis creates new algorithms for Bayesian sequential design to accommodate parameter and model uncertainty using SMC. The algorithms are substantially faster than previous approaches allowing the simulation properties of various design utilities to be investigated in a more timely manner. Furthermore the approach offers convenient estimation of Bayesian utilities and other quantities that are particularly relevant in the presence of model uncertainty. Finally, Part III of this thesis tackles a substantive medical problem. A neurological disorder known as motor neuron disease (MND) progressively causes motor neurons to no longer have the ability to innervate the muscle fibres, causing the muscles to eventually waste away. When this occurs the motor unit effectively ‘dies’. There is no cure for MND, and fatality often results from a lack of muscle strength to breathe. The prognosis for many forms of MND (particularly amyotrophic lateral sclerosis (ALS)) is particularly poor, with patients usually only surviving a small number of years after the initial onset of disease. Measuring the progress of diseases of the motor units, such as ALS, is a challenge for clinical neurologists. Motor unit number estimation (MUNE) is an attempt to directly assess underlying motor unit loss rather than indirect techniques such as muscle strength assessment, which generally is unable to detect progressions due to the body’s natural attempts at compensation. Part III of this thesis builds upon a previous Bayesian technique, which develops a sophisticated statistical model that takes into account physiological information about motor unit activation and various sources of uncertainties. More specifically, we develop a more reliable MUNE method by applying marginalisation over latent variables in order to improve the performance of a previously developed reversible jump Markov chain Monte Carlo sampler. We make other subtle changes to the model and algorithm to improve the robustness of the approach.

Polycaprolactone-based scaffold plus recombinant human bone morphogenic protein rhBMP-2) in a sheep thoracic spine fusion model

Adolescent idiopathic scoliosis is a complex three dimensional deformity affecting 2-3% of the general population. The resulting spinal deformity consists of coronal curvature, hypokyphosis of the thoracic spine and vertebral rotation in the axial plane with posterior elements turned into the curve concavity. The potential for curve progression is heightened during the adolescent growth spurt. Success of scoliosis deformity correction depends on solid bony fusion between adjacent vertebrae after the intervertebral (IV) discs have been surgically cleared and the disc spaces filled with graft material. Recently a bioactive and resorbable scaffold fabricated from medical grade polycaprolactone has been developed for bone regeneration at load bearing sites. Combined with rhBMP-2, this has been shown to be successful in acting as a bone graft substitute in a porcine lumbar interbody fusion model when compared to autologous bone graft alone. The study aimed to establish a large animal thoracic spine interbody fusion model, develop spine biodegradable scaffolds (PCL) in combination with biologics (rhBMP-2) and to establish a platform for research into spine tissue engineering constructs. Preliminary results demonstrate higher grades of radiologically evident bony fusion across all levels when comparing fusion scores between the 3 and 6 month postop groups at the PCL CaP coated scaffold level, which is observed to be a similar grade to autograft, while no fusion is seen at the scaffold only level. Results to date suggest that the combination of rhBMP-2 and scaffold engineering actively promotes bone formation, laying the basis of a viable tissue engineered constructs.

Biomechanical performance of polycaprolactone (PCL)-based scaffold with rhBMP-2 in a sheep thoracic spine fusion model

Adolescent idiopathic scoliosis is a complex three dimensional deformity affecting 2-3% of the general population. Resulting spine deformities include progressive coronal curvature, hypokyphosis, or frank lordosis in the thoracic spine and vertebral rotation in the axial plane with posterior elements turned into the curve concavity. The potential for curve progression is heightened during the adolescent growth spurt. Success of scoliosis deformity correction depends on solid bony fusion between adjacent vertebrae after the intervertebral discs have been surgically cleared and the disc spaces filled with graft material. Problems with bone graft harvest site morbidity as well as limited bone availability have led to the search for bone graft substitutes. Recently, a bioactive and resorbable scaffold fabricated from medical grade polycaprolactone (PCL) has been developed for bone regeneration at load bearing sites. Combined with recombinant human bone morphogenic protein–2 (rhBMP-2), this has been shown to be successful in acting as a bone graft substitute in acting as a bone graft substitute in a porcine lumbar interbody fusion model when compared to autologous bone graft. This in vivo sheep study intends to evaluate the suitability of a custom designed medical grade PCL scaffold in combination with rhBMP-2 as a bone graft substitute in the setting of mini–thoracotomy surgery as a platform for ongoing research to benefit patients with adolescent idiopathic scoliosis.

Uncertainty analysis of pollutant build-up modelling based on a Bayesian Weighted Least Squares approach

Reliable pollutant build-up prediction plays a critical role in the accuracy of urban stormwater quality modelling outcomes. However, water quality data collection is resource demanding compared to streamflow data monitoring, where a greater quantity of data is generally available. Consequently, available water quality data sets span only relatively short time scales unlike water quantity data. Therefore, the ability to take due consideration of the variability associated with pollutant processes and natural phenomena is constrained. This in turn gives rise to uncertainty in the modelling outcomes as research has shown that pollutant loadings on catchment surfaces and rainfall within an area can vary considerably over space and time scales. Therefore, the assessment of model uncertainty is an essential element of informed decision making in urban stormwater management. This paper presents the application of a range of regression approaches such as ordinary least squares regression, weighted least squares Regression and Bayesian Weighted Least Squares Regression for the estimation of uncertainty associated with pollutant build-up prediction using limited data sets. The study outcomes confirmed that the use of ordinary least squares regression with fixed model inputs and limited observational data may not provide realistic estimates. The stochastic nature of the dependent and independent variables need to be taken into consideration in pollutant build-up prediction. It was found that the use of the Bayesian approach along with the Monte Carlo simulation technique provides a powerful tool, which attempts to make the best use of the available knowledge in the prediction and thereby presents a practical solution to counteract the limitations which are otherwise imposed on water quality modelling.

Towards brain-based sensor fusion for navigating robots

Current state of the art robot mapping and navigation systems produce impressive performance under a narrow range of robot platform, sensor and environmental conditions, in contrast to animals such as rats that produce “good enough” maps that enable them to function under an incredible range of situations. In this paper we present a rat-inspired featureless sensor-fusion system that assesses the usefulness of multiple sensor modalities based on their utility and coherence for place recognition during a navigation task, without knowledge as to the type of sensor. We demonstrate the system on a Pioneer robot in indoor and outdoor environments with abrupt lighting changes. Through dynamic weighting of the sensors, the system is able to perform correct place recognition and mapping where the static sensor weighting approach fails.

Investigating effective wayfinding in airports : a Bayesian network approach

Effective Wayfinding is the successful interplay of human and environmental factors resulting in a person successfully moving from their current position to a desired location in a timely manner. To date this process has not been modelled to reflect this interplay. This paper proposes a complex modelling system approach of wayfinding by using Bayesian Networks to model this process, and applies the model to airports. The model suggests that human factors have a greater impact on effective wayfinding in airports than environmental factors. The greatest influences on human factors are found to be the level of spatial anxiety experienced by travellers and their cognitive and spatial skills. The model also predicted that the navigation pathway that a traveller must traverse has a larger impact on the effectiveness of an airport’s environment in promoting effective wayfinding than the terminal design.

Bayesian computation via empirical likelihood

Approximate Bayesian computation has become an essential tool for the analysis of complex stochastic models when the likelihood function is numerically unavailable. However, the well-established statistical method of empirical likelihood provides another route to such settings that bypasses simulations from the model and the choices of the approximate Bayesian computation parameters (summary statistics, distance, tolerance), while being convergent in the number of observations. Furthermore, bypassing model simulations may lead to significant time savings in complex models, for instance those found in population genetics. The Bayesian computation with empirical likelihood algorithm we develop in this paper also provides an evaluation of its own performance through an associated effective sample size. The method is illustrated using several examples, including estimation of standard distributions, time series, and population genetics models.

Exact and approximate Bayesian inference for low count time series models with intractable likelihoods

In this paper we present a new simulation methodology in order to obtain exact or approximate Bayesian inference for models for low-valued count time series data that have computationally demanding likelihood functions. The algorithm fits within the framework of particle Markov chain Monte Carlo (PMCMC) methods. The particle filter requires only model simulations and, in this regard, our approach has connections with approximate Bayesian computation (ABC). However, an advantage of using the PMCMC approach in this setting is that simulated data can be matched with data observed one-at-a-time, rather than attempting to match on the full dataset simultaneously or on a low-dimensional non-sufficient summary statistic, which is common practice in ABC. For low-valued count time series data we find that it is often computationally feasible to match simulated data with observed data exactly. Our particle filter maintains $N$ particles by repeating the simulation until $N+1$ exact matches are obtained. Our algorithm creates an unbiased estimate of the likelihood, resulting in exact posterior inferences when included in an MCMC algorithm. In cases where exact matching is computationally prohibitive, a tolerance is introduced as per ABC. A novel aspect of our approach is that we introduce auxiliary variables into our particle filter so that partially observed and/or non-Markovian models can be accommodated. We demonstrate that Bayesian model choice problems can be easily handled in this framework.

A context-fusion framework for enhancing context-awareness on mobile application

Mobile devices and smartphones have become a significant communication channel for everyday life. The sensing capabilities of mobile devices are expanding rapidly, and sensors embedded in these devices are cheaper and more powerful than before. It is evident that mobile devices have become the most suitable candidates to sense contextual information without needing extra tools. However, current research shows only a limited number of sensors are being explored and investigated. As a result, it still needs to be clarified what forms of contextual information extracted from mo- bile sensors are useful. Therefore, this research investigates the context sensing using current mobile sensors, the study follows experimental methods and sensor data is evaluated and synthesised, in order to deduce the value of various sensors and combinations of sensor for the use in context-aware mobile applications. This study aims to develop a context fusion framework that will enhance the context-awareness on mobile applications, as well as exploring innovative techniques for context sensing on smartphone devices.

Using a Bayesian network to model colonisation with Vancomycin Resistant Enterococcus (VRE)

Objective: Effective management of multi-resistant organisms is an important issue for hospitals both in Australia and overseas. This study investigates the utility of using Bayesian Network (BN) analysis to examine relationships between risk factors and colonization with Vancomycin Resistant Enterococcus (VRE). Design: Bayesian Network Analysis was performed using infection control data collected over a period of 36 months (2008-2010). Setting: Princess Alexandra Hospital (PAH), Brisbane. Outcome of interest: Number of new VRE Isolates Methods: A BN is a probabilistic graphical model that represents a set of random variables and their conditional dependencies via a directed acyclic graph (DAG). BN enables multiple interacting agents to be studied simultaneously. The initial BN model was constructed based on the infectious disease physician‟s expert knowledge and current literature. Continuous variables were dichotomised by using third quartile values of year 2008 data. BN was used to examine the probabilistic relationships between VRE isolates and risk factors; and to establish which factors were associated with an increased probability of a high number of VRE isolates. Software: Netica (version 4.16). Results: Preliminary analysis revealed that VRE transmission and VRE prevalence were the most influential factors in predicting a high number of VRE isolates. Interestingly, several factors (hand hygiene and cleaning) known through literature to be associated with VRE prevalence, did not appear to be as influential as expected in this BN model. Conclusions: This preliminary work has shown that Bayesian Network Analysis is a useful tool in examining clinical infection prevention issues, where there is often a web of factors that influence outcomes. This BN model can be restructured easily enabling various combinations of agents to be studied.

Establishment and characterisation of an open mini-thoracotomy surgical approach to an ovine thoracic spine fusion model

Background A large animal model is required for assessment of minimally invasive, tissue engineering based approaches to thoracic spine fusion, with relevance to deformity correction surgery for human adolescent idiopathic scoliosis. Here we develop a novel open mini–thoracotomy approach in an ovine model of thoracic interbody fusion which allows assessment of various fusion constructs, with a focus on novel, tissue engineering based interventions. Methods The open mini-thoracotomy surgical approach was developed through a series of mock surgeries, and then applied in a live sheep study. Customized scaffolds were manufactured to conform with intervertebral disc space clearances required of the study. Twelve male Merino sheep aged 4 to 6 years and weighing 35 – 45 kg underwent the abovementioned procedure and were divided into two groups of six sheep at survival timelines of 6 and 12 months. Each sheep underwent a 3-level discectomy (T6/7, T8/9 and T10/11) with randomly allocated implantation of a different graft substitute at each of the three levels; (i) polycaprolactone (PCL) based scaffold plus 0.54μg rhBMP-2, (ii) PCL-based scaffold alone or (iii) autograft. The sheep were closely monitored post- operatively for signs of pain (i.e. gait abnormalities/ teeth gnawing/ social isolation). Fusion assessments were conducted post-sacrifice using Computed Tomography and hard-tissue histology. All scientific work was undertaken in accordance with the study protocol has been approved by the Institute's committee on animal research. Results. All twelve sheep were successfully operated on and reached the allotted survival timelines, thereby demonstrating the feasibility of the surgical procedure and post-operative care. There were no significant complications and during the post-operative period the animals did not exhibit marked signs of distress according to the described assessment criteria. Computed Tomographic scanning demonstrated higher fusion grades in the rhBMP-2 plus PCL-based scaffold group in comparison to either PCL-based scaffold alone or autograft. These results were supported by histological evaluation of the respective groups. Conclusion. This novel open mini-thoracotomy surgical approach to the ovine thoracic spine represents a safe surgical method which can reproducibly form the platform for research into various spine tissue engineered constructs (TEC) and their fusion promoting properties.

Biological performance of a polycaprolactone-based scaffold plus recombinant human morphogenetic protein-2 (rhBMP-2) in an ovine thoracic interbody fusion model

Introduction. We develop a sheep thoracic spine interbody fusion model to study the suitability of polycaprolactone-based scaffold and recombinant human bone morphogenetic protein-2 (rhBMP-2) as a bone graft substitute within the thoracic spine. The surgical approach is a mini- open thoracotomy with relevance to minimally invasive deformity correction surgery for adolescent idiopathic scoliosis. To date there are no studies examining the use of this biodegradable implant in combination with biologics in a sheep thoracic spine model. Methods. In the present study, six sheep underwent a 3-level (T6/7, T8/9 and T10/11) discectomy with randomly allocated implantation of a different graft substitute at each of the three levels; (i) calcium phosphate (CaP) coated polycaprolactone based scaffold plus 0.54µg rhBMP-2, (ii) CaP coated PCL- based scaffold alone or (iii) autograft (mulched rib head). Fusion was assessed at six months post-surgery. Results. Computed Tomographic scanning demonstrated higher fusion grades in the rhBMP-2 plus PCL- based scaffold group in comparison to either PCL-based scaffold alone or autograft. These results were supported by histological evaluations of the respective groups. Biomechanical testing revealed significantly higher stiffness for the rhBMP-2 plus PCL- based scaffold group in all loading directions in comparison to the other two groups. Conclusions. The results of this study demonstrate that rhBMP-2 plus PCL-based scaffold is a viable bone graft substitute, providing an optimal environment for thoracic interbody spinal fusion in a large animal model.

A Bayesian approach to the determination of ignition delay

A novel in-cylinder pressure method for determining ignition delay has been proposed and demonstrated. This method proposes a new Bayesian statistical model to resolve the start of combustion, defined as being the point at which the band-pass in-cylinder pressure deviates from background noise and the combustion resonance begins. Further, it is demonstrated that this method is still accurate in situations where there is noise present. The start of combustion can be resolved for each cycle without the need for ad hoc methods such as cycle averaging. Therefore, this method allows for analysis of consecutive cycles and inter-cycle variability studies. Ignition delay obtained by this method and by the net rate of heat release have been shown to give good agreement. However, the use of combustion resonance to determine the start of combustion is preferable over the net rate of heat release method because it does not rely on knowledge of heat losses and will still function accurately in the presence of noise. Results for a six-cylinder turbo-charged common-rail diesel engine run with neat diesel fuel at full, three quarters and half load have been presented. Under these conditions the ignition delay was shown to increase as the load was decreased with a significant increase in ignition delay at half load, when compared with three quarter and full loads.

Combining elicited expert knowledge into Bayesian priors

Information that is elicited from experts can be treated as `data', so can be analysed using a Bayesian statistical model, to formulate a prior model. Typically methods for encoding a single expert's knowledge have been parametric, constrained by the extent of an expert's knowledge and energy regarding a target parameter. Interestingly these methods have often been deterministic, in that all elicited information is treated at `face value', without error. Here we sought a parametric and statistical approach for encoding assessments from multiple experts. Our recent work proposed and demonstrated the use of a flexible hierarchical model for this purpose. In contrast to previous mathematical approaches like linear or geometric pooling, our new approach accounts for several sources of variation: elicitation error, encoding error and expert diversity. Of interest are the practical, mathematical and philosophical interpretations of this form of hierarchical pooling (which is both statistical and parametric), and how it fits within the subjective Bayesian paradigm. Case studies from a bioassay and project management (on PhDs) are used to illustrate the approach.

Classifier selection using sequential error ratio criterion for multi-instance and multi-sample fusion

Classifier selection is a problem encountered by multi-biometric systems that aim to improve performance through fusion of decisions. A particular decision fusion architecture that combines multiple instances (n classifiers) and multiple samples (m attempts at each classifier) has been proposed in previous work to achieve controlled trade-off between false alarms and false rejects. Although analysis on text-dependent speaker verification has demonstrated better performance for fusion of decisions with favourable dependence compared to statistically independent decisions, the performance is not always optimal. Given a pool of instances, best performance with this architecture is obtained for certain combination of instances. Heuristic rules and diversity measures have been commonly used for classifier selection but it is shown that optimal performance is achieved for the `best combination performance' rule. As the search complexity for this rule increases exponentially with the addition of classifiers, a measure - the sequential error ratio (SER) - is proposed in this work that is specifically adapted to the characteristics of sequential fusion architecture. The proposed measure can be used to select a classifier that is most likely to produce a correct decision at each stage. Error rates for fusion of text-dependent HMM based speaker models using SER are compared with other classifier selection methodologies. SER is shown to achieve near optimal performance for sequential fusion of multiple instances with or without the use of multiple samples. The methodology applies to multiple speech utterances for telephone or internet based access control and to other systems such as multiple finger print and multiple handwriting sample based identity verification systems.