Understanding GPU programming for statistical computation: Studies in massively parallel massive mixtures

This article describes advances in statistical computation for large-scale data analysis in structured Bayesian mixture models via graphics processing unit (GPU) programming. The developments are partly motivated by computational challenges arising in fitting models of increasing heterogeneity to increasingly large datasets. An example context concerns common biological studies using high-throughput technologies generating many, very large datasets and requiring increasingly high-dimensional mixture models with large numbers of mixture components.We outline important strategies and processes for GPU computation in Bayesian simulation and optimization approaches, give examples of the benefits of GPU implementations in terms of processing speed and scale-up in ability to analyze large datasets, and provide a detailed, tutorial-style exposition that will benefit readers interested in developing GPU-based approaches in other statistical models. Novel, GPU-oriented approaches to modifying existing algorithms software design can lead to vast speed-up and, critically, enable statistical analyses that presently will not be performed due to compute time limitations in traditional computational environments. Supplementalmaterials are provided with all source code, example data, and details that will enable readers to implement and explore the GPU approach in this mixture modeling context. © 2010 American Statistical Association, Institute of Mathematical Statistics, and Interface Foundation of North America.

Programming stress-induced altruistic death in engineered bacteria.

Programmed death is often associated with a bacterial stress response. This behavior appears paradoxical, as it offers no benefit to the individual. This paradox can be explained if the death is 'altruistic': the killing of some cells can benefit the survivors through release of 'public goods'. However, the conditions where bacterial programmed death becomes advantageous have not been unambiguously demonstrated experimentally. Here, we determined such conditions by engineering tunable, stress-induced altruistic death in the bacterium Escherichia coli. Using a mathematical model, we predicted the existence of an optimal programmed death rate that maximizes population growth under stress. We further predicted that altruistic death could generate the 'Eagle effect', a counter-intuitive phenomenon where bacteria appear to grow better when treated with higher antibiotic concentrations. In support of these modeling insights, we experimentally demonstrated both the optimality in programmed death rate and the Eagle effect using our engineered system. Our findings fill a critical conceptual gap in the analysis of the evolution of bacterial programmed death, and have implications for a design of antibiotic treatment.

Metabolic programming and PDHK1 control CD4+ T cell subsets and inflammation.

Activation of CD4+ T cells results in rapid proliferation and differentiation into effector and regulatory subsets. CD4+ effector T cell (Teff) (Th1 and Th17) and Treg subsets are metabolically distinct, yet the specific metabolic differences that modify T cell populations are uncertain. Here, we evaluated CD4+ T cell populations in murine models and determined that inflammatory Teffs maintain high expression of glycolytic genes and rely on high glycolytic rates, while Tregs are oxidative and require mitochondrial electron transport to proliferate, differentiate, and survive. Metabolic profiling revealed that pyruvate dehydrogenase (PDH) is a key bifurcation point between T cell glycolytic and oxidative metabolism. PDH function is inhibited by PDH kinases (PDHKs). PDHK1 was expressed in Th17 cells, but not Th1 cells, and at low levels in Tregs, and inhibition or knockdown of PDHK1 selectively suppressed Th17 cells and increased Tregs. This alteration in the CD4+ T cell populations was mediated in part through ROS, as N-acetyl cysteine (NAC) treatment restored Th17 cell generation. Moreover, inhibition of PDHK1 modulated immunity and protected animals against experimental autoimmune encephalomyelitis, decreasing Th17 cells and increasing Tregs. Together, these data show that CD4+ subsets utilize and require distinct metabolic programs that can be targeted to control specific T cell populations in autoimmune and inflammatory diseases.

A general framework for aquatic biogeochemical models

One of the most of challenging steps in the development of coupled hydrodynamic-biogeochemical models is the combination of multiple, often incompatible computer codes that describe individual physical, chemical, biological and geological processes. This “coupling” is time-consuming, error-prone, and demanding in terms of scientific and programming expertise. The open source, Fortran-based Framework for Aquatic Biogeochemical Models addresses these problems by providing a consistent set of programming interfaces through which hydrodynamic and biogeochemical models communicate. Models are coded once to connect to FABM, after which arbitrary combinations of hydrodynamic and biogeochemical models can be made. Thus, a biogeochemical model code works unmodified within models of a chemostat, a vertically structured water column, and a three-dimensional basin. Moreover, complex biogeochemistry can be distributed over many compact, self-contained modules, coupled at run-time. By enabling distributed development and user-controlled coupling of biogeochemical models, FABM enables optimal use of the expertise of scientists, programmers and end-users.

Surrogate-Based Optimization Using Multifidelity Models with Variable Parameterization and Corrected Space Mapping

Surrogate-based-optimization methods provide a means to achieve high-fidelity design optimization at reduced computational cost by using a high-fidelity model in combination with lower-fidelity models that are less expensive to evaluate. This paper presents a provably convergent trust-region model-management methodology for variableparameterization design models: that is, models for which the design parameters are defined over different spaces. Corrected space mapping is introduced as a method to map between the variable-parameterization design spaces. It is then used with a sequential-quadratic-programming-like trust-region method for two aerospace-related design optimization problems. Results for a wing design problem and a flapping-flight problem show that the method outperforms direct optimization in the high-fidelity space. On the wing design problem, the new method achieves 76% savings in high-fidelity function calls. On a bat-flight design problem, it achieves approximately 45% time savings, although it converges to a different local minimum than did the benchmark.

Modelling the performance of self-compacting SIFCON of cement slurries containing limestone powder using genetic programming technique

The paper explores the potential of applicability of Genetic programming approach (GP), adopted in this investigation, to model the combined effects of five independent variables to predict the mini-slump, the plate cohesion meter, the induced bleeding test, the J-fiber penetration value, and the compressive strength at 7 and 28 days of self-compacting slurry infiltrated fiber concrete (SIFCON). The variables investigated were the proportions of limestone powder (LSP) and sand, the dosage rates of superplasticiser (SP) and viscosity modifying agent (VMA), and water-to-binder ratio (W/B). Twenty eight mixtures were made with 10-50% LSP as replacement of cement, 0.02-0.06% VMA by mass of cement, 0.6-1.2% SP and 50-150% sand (% mass of binder) and 0.42-0.48 W/B. The proposed genetic models of the self-compacting SIFCON offer useful modelling approach regarding the mix optimisation in predicting the fluidity, the cohesion, the bleeding, the penetration, and the compressive strength.

FPGA Prototyping of Emerging Manycore Architectures for Parallel Programming Research using Formic Boards

Performance evaluation of parallel software and architectural exploration of innovative hardware support face a common challenge with emerging manycore platforms: they are limited by the slow running time and the low accuracy of software simulators. Manycore FPGA prototypes are difficult to build, but they offer great rewards. Software running on such prototypes runs orders of magnitude faster than current simulators. Moreover, researchers gain significant architectural insight during the modeling process. We use the Formic FPGA prototyping board [1], which specifically targets scalable and cost-efficient multi-board prototyping, to build and test a 64-board model of a 512-core, MicroBlaze-based, non-coherent hardware prototype with a full network-on-chip in a 3D-mesh topology. We expand the hardware architecture to include the ARM Versatile Express platforms and build a 520-core heterogeneous prototype of 8 Cortex-A9 cores and 512 MicroBlaze cores. We then develop an MPI library for the prototype and evaluate it extensively using several bare-metal and MPI benchmarks. We find that our processor prototype is highly scalable, models faithfully single-chip multicore architectures, and is a very efficient platform for parallel programming research, being 50,000 times faster than software simulation.

Approximate credal network updating by linear programming with applications to decision making

Credal nets are probabilistic graphical models which extend Bayesian nets to cope with sets of distributions. An algorithm for approximate credal network updating is presented. The problem in its general formulation is a multilinear optimization task, which can be linearized by an appropriate rule for fixing all the local models apart from those of a single variable. This simple idea can be iterated and quickly leads to accurate inferences. A transformation is also derived to reduce decision making in credal networks based on the maximality criterion to updating. The decision task is proved to have the same complexity of standard inference, being NPPP-complete for general credal nets and NP-complete for polytrees. Similar results are derived for the E-admissibility criterion. Numerical experiments confirm a good performance of the method.

Towards Possibilistic Fuzzy Answer Set Programming

Fuzzy answer set programming (FASP) is a generalization of answer set programming to continuous domains. As it can not readily take uncertainty into account, however, FASP is not suitable as a basis for approximate reasoning and cannot easily be used to derive conclusions from imprecise information. To cope with this, we propose an extension of FASP based on possibility theory. The resulting framework allows us to reason about uncertain information in continuous domains, and thus also about information that is imprecise or vague. We propose a syntactic procedure, based on an immediate consequence operator, and provide a characterization in terms of minimal models, which allows us to straightforwardly implement our framework using existing FASP solvers.

Approximating Credal Network Inferences by Linear Programming

An algorithm for approximate credal network updating is presented. The problem in its general formulation is a multilinear optimization task, which can be linearized by an appropriate rule for fixing all the local models apart from those of a single variable. This simple idea can be iterated and quickly leads to very accurate inferences. The approach can also be specialized to classification with credal networks based on the maximality criterion. A complexity analysis for both the problem and the algorithm is reported together with numerical experiments, which confirm the good performance of the method. While the inner approximation produced by the algorithm gives rise to a classifier which might return a subset of the optimal class set, preliminary empirical results suggest that the accuracy of the optimal class set is seldom affected by the approximate probabilities

Multilinear and Integer Programming for Markov Decision Processes with Imprecise Probabilities

Markov Decision Processes (MDPs) are extensively used to encode sequences of decisions with probabilistic effects. Markov Decision Processes with Imprecise Probabilities (MDPIPs) encode sequences of decisions whose effects are modeled using sets of probability distributions. In this paper we examine the computation of Γ-maximin policies for MDPIPs using multilinear and integer programming. We discuss the application of our algorithms to “factored” models and to a recent proposal, Markov Decision Processes with Set-valued Transitions (MDPSTs), that unifies the fields of probabilistic and “nondeterministic” planning in artificial intelligence research. 

Genetic programming and bacterial algorithm for neural networks and fuzzy systems design

In the field of control systems it is common to use techniques based on model adaptation to carry out control for plants for which mathematical analysis may be intricate. Increasing interest in biologically inspired learning algorithms for control techniques such as Artificial Neural Networks and Fuzzy Systems is in progress. In this line, this paper gives a perspective on the quality of results given by two different biologically connected learning algorithms for the design of B-spline neural networks (BNN) and fuzzy systems (FS). One approach used is the Genetic Programming (GP) for BNN design and the other is the Bacterial Evolutionary Algorithm (BEA) applied for fuzzy rule extraction. Also, the facility to incorporate a multi-objective approach to the GP algorithm is outlined, enabling the designer to obtain models more adequate for their intended use.

B-spline and neuro-fuzzy models design with function and derivative equalities

The design of neuro-fuzzy models is still a complex problem, as it involves not only the determination of the model parameters, but also its structure. Of special importance is the incorporation of a priori information in the design process. In this paper two known design algorithms for B-spline models will be updated to account for function and derivatives equality restrictions, which are important when the neural model is used for performing single or multi-objective optimization on-line.

Design and implementation of a real-time data acquisition system for the identification of dynamic temperature models in a hydroponic greenhouse

This text describes a real data acquisition and identification system implemented in a soilless greenhouse located at the University of Algarve (south of Portugal). Using the Real Time Workshop, Simulink, Matlab and the C programming language a system was developed to perform real-time data acquisition from a set of sensors.

Training neuro-fuzzy models using evolution based algorithms

The normal design process for neural networks or fuzzy systems involve two different phases: the determination of the best topology, which can be seen as a system identification problem, and the determination of its parameters, which can be envisaged as a parameter estimation problem. This latter issue, the determination of the model parameters (linear weights and interior knots) is the simplest task and is usually solved using gradient or hybrid schemes. The former issue, the topology determination, is an extremely complex task, especially if dealing with real-world problems.