Advanced mean field methods:theory and practice

A major problem in modern probabilistic modeling is the huge computational complexity involved in typical calculations with multivariate probability distributions when the number of random variables is large. Because exact computations are infeasible in such cases and Monte Carlo sampling techniques may reach their limits, there is a need for methods that allow for efficient approximate computations. One of the simplest approximations is based on the mean field method, which has a long history in statistical physics. The method is widely used, particularly in the growing field of graphical models. Researchers from disciplines such as statistical physics, computer science, and mathematical statistics are studying ways to improve this and related methods and are exploring novel application areas. Leading approaches include the variational approach, which goes beyond factorizable distributions to achieve systematic improvements; the TAP (Thouless-Anderson-Palmer) approach, which incorporates correlations by including effective reaction terms in the mean field theory; and the more general methods of graphical models. Bringing together ideas and techniques from these diverse disciplines, this book covers the theoretical foundations of advanced mean field methods, explores the relation between the different approaches, examines the quality of the approximation obtained, and demonstrates their application to various areas of probabilistic modeling.

Mean field methods for classification with Gaussian processes

We discuss the Application of TAP mean field methods known from Statistical Mechanics of disordered systems to Bayesian classification with Gaussian processes. In contrast to previous applications, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.

Gaussian processes for classification: mean field algorithms:mean-field algorithms

We derive a mean field algorithm for binary classification with Gaussian processes which is based on the TAP approach originally proposed in Statistical Physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler 'naive' mean field theory and support vector machines (SVM) as limiting cases. For both mean field algorithms and support vectors machines, simulation results for three small benchmark data sets are presented. They show 1. that one may get state of the art performance by using the leave-one-out estimator for model selection and 2. the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The latter result is a taken as a strong support for the internal consistency of the mean field approach.

Gaussian processes and SVM: Mean field results and leave-one-out

In this chapter, we elaborate on the well-known relationship between Gaussian processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational problems arising in GP and SVM. The first one is the calculation of the posterior mean for GP classifiers using a `naive' mean field approach. The second one is a leave-one-out estimator for the generalization error of SVM based on a linear response method. Simulation results on a benchmark dataset show similar performances for the GP mean field algorithm and the SVM algorithm. The approximate leave-one-out estimator is found to be in very good agreement with the exact leave-one-out error.

A mean field theory of coded CDMA systems

We present a mean field theory of code-division multiple access (CDMA) systems with error-control coding. On the basis of the relation between the free energy and mutual information, we obtain an analytical expression of the maximum spectral efficiency of the coded CDMA system, from which a mean field description of the coded CDMA system is provided in terms of a bank of scalar Gaussian channels whose variances in general vary at different code symbol positions. Regular low-density parity-check (LDPC)-coded CDMA systems are also discussed as an example of the coded CDMA systems.

Flow field analysis of some mixing and conveying screw element regions, within a closely intermeshing, co-rotating twin-screw extruder.

Grafting of antioxidants and other modifiers onto polymers by reactive extrusion, has been performed successfully by the Polymer Processing and Performance Group at Aston University. Traditionally the optimum conditions for the grafting process have been established within a Brabender internal mixer. Transfer of this batch process to a continuous processor, such as an extruder, has, typically, been empirical. To have more confidence in the success of direct transfer of the process requires knowledge of, and comparison between, residence times, mixing intensities, shear rates and flow regimes in the internal mixer and in the continuous processor.The continuous processor chosen for the current work in the closely intermeshing, co-rotating twin-screw extruder (CICo-TSE). CICo-TSEs contain screw elements that convey material with a self-wiping action and are widely used for polymer compounding and blending. Of the different mixing modules contained within the CICo-TSE, the trilobal elements, which impose intensive mixing, and the mixing discs, which impose extensive mixing, are of importance when establishing the intensity of mixing. In this thesis, the flow patterns within the various regions of the single-flighted conveying screw elements and within both the trilobal element and mixing disc zones of a Betol BTS40 CICo-TSE, have been modelled using the computational fluid dynamics package Polyflow. A major obstacle encountered when solving the flow problem within all of these sets of elements, arises from both the complex geometry and the time-dependent flow boundaries as the elements rotate about their fixed axes. Simulation of the time dependent boundaries was overcome by selecting a number of sequential 2D and 3D geometries, used to represent partial mixing cycles. The flow fields were simulated using the ideal rheological properties of polypropylene and characterised in terms of velocity vectors, shear stresses generated and a parameter known as the mixing efficiency. The majority of the large 3D simulations were performed on the Cray J90 supercomputer situated at the Rutherford-Appleton laboratories, with pre- and postprocessing operations achieved via a Silicon Graphics Indy workstation. A mechanical model was constructed consisting of various CICo-TSE elements rotating within a transparent outer barrel. A technique has been developed using coloured viscous clays whereby the flow patterns and mixing characteristics within the CICo-TSE may be visualised. In order to test and verify the simulated predictions, the patterns observed within the mechanical model were compared with the flow patterns predicted by the computational model. The flow patterns within the single-flighted conveying screw elements in particular, showed good agreement between the experimental and simulated results.

Generalized mean field approximation for parallel dynamics of the Ising model

The dynamics of the non-equilibrium Ising model with parallel updates is investigated using a generalized mean field approximation that incorporates multiple two-site correlations at any two time steps, which can be obtained recursively. The proposed method shows significant improvement in predicting local system properties compared to other mean field approximation techniques, particularly in systems with symmetric interactions. Results are also evaluated against those obtained from Monte Carlo simulations. The method is also employed to obtain parameter values for the kinetic inverse Ising modeling problem, where couplings and local field values of a fully connected spin system are inferred from data. © 2014 IOP Publishing Ltd and SISSA Medialab srl.

Mean field methods for Gaussian process classifiers

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

Mean field evolution in an open quantum system

In this thesis, we consider N quantum particles coupled to collective thermal quantum environments. The coupling is energy conserving and scaled in the mean field way. There is no direct interaction between the particles, they only interact via the common reservoir. It is well known that an initially disentangled state of the N particles will remain disentangled at times in the limit N -> [infinity]. In this thesis, we evaluate the η-body reduced density matrix (tracing over the reservoirs and the N - η remaining particles). We identify the main disentangled part of the reduced density matrix and obtain the first order correction term in 1/N. We show that this correction term is entangled. We also estimate the speed of convergence of the reduced density matrix as N -> [infinity]. Our model is exactly solvable and it is not based on numerical approximation.

Numerical detection of the Gardner transition in a mean-field glass former.

Recent theoretical advances predict the existence, deep into the glass phase, of a novel phase transition, the so-called Gardner transition. This transition is associated with the emergence of a complex free energy landscape composed of many marginally stable sub-basins within a glass metabasin. In this study, we explore several methods to detect numerically the Gardner transition in a simple structural glass former, the infinite-range Mari-Kurchan model. The transition point is robustly located from three independent approaches: (i) the divergence of the characteristic relaxation time, (ii) the divergence of the caging susceptibility, and (iii) the abnormal tail in the probability distribution function of cage order parameters. We show that the numerical results are fully consistent with the theoretical expectation. The methods we propose may also be generalized to more realistic numerical models as well as to experimental systems.

An introduction to mathematical models of coagulation-fragmentation processes: a discrete deterministic mean-field approach

We summarise the properties and the fundamental mathematical results associated with basic models which describe coagulation and fragmentation processes in a deterministic manner and in which cluster size is a discrete quantity (an integer multiple of some basic unit size). In particular, we discuss Smoluchowski's equation for aggregation, the Becker-Döring model of simultaneous aggregation and fragmentation, and more general models involving coagulation and fragmentation.

Variational mean-field approach to the double-exchange model

It has been recently shown that the double exchange Hamiltonian, with weak antiferromagnetic interactions, has a richer variety of first- and second-order transitions than previously anticipated, and that such transitions are consistent with the magnetic properties of manganites. Here we present a thorough discussion of the variational mean-field approach that leads to these results. We also show that the effect of the Berry phase turns out to be crucial to produce first-order paramagnetic-ferromagnetic transitions near half filling with transition temperatures compatible with the experimental situation. The computation relies on two crucial facts: the use of a mean-field ansatz that retains the complexity of a system of electrons with off-diagonal disorder, not fully taken into account by the mean-field techniques, and the small but significant antiferromagnetic superexchange interaction between the localized spins.

Phase-field analysis of finite-strain plates and shells including element subdivision

With the theme of fracture of finite-strain plates and shells based on a phase-field model of crack regularization, we introduce a new staggered algorithm for elastic and elasto-plastic materials. To account for correct fracture behavior in bending, two independent phase-fields are used, corresponding to the lower and upper faces of the shell. This is shown to provide a realistic behavior in bending-dominated problems, here illustrated in classical beam and plate problems. Finite strain behavior for both elastic and elasto-plastic constitutive laws is made compatible with the phase-field model by use of a consistent updated-Lagrangian algorithm. To guarantee sufficient resolution in the definition of the crack paths, a local remeshing algorithm based on the phase- field values at the lower and upper shell faces is introduced. In this local remeshing algorithm, two stages are used: edge-based element subdivision and node repositioning. Five representative numerical examples are shown, consisting of a bi-clamped beam, two versions of a square plate, the Keesecker pressurized cylinder problem, the Hexcan problem and the Muscat-Fenech and Atkins plate. All problems were successfully solved and the proposed solution was found to be robust and efficient.

Confinement-deconfinement transition and spin correlations in a generalized Kitaev model

We present a spin model, namely, the Kitaev model augmented by a loop term and perturbed by an Ising Hamiltonian, and show that it exhibits both confinement-deconfinement transitions from spin liquid to antiferromagnetic/spin-chain/ferromagnetic phases and topological quantum phase transitions between gapped and gapless spin-liquid phases. We develop a fermionic resonating-valence-bonds (RVB) mean-field theory to chart out the phase diagram of the model and estimate the stability of its spin-liquid phases, which might be relevant for attempts to realize the model in optical lattices and other spin systems. We present an analytical mean-field theory to study the confinement-deconfinement transition for large coefficient of the loop term and show that this transition is first order within such mean-field analysis in this limit. We also conjecture that in some other regimes, the confinement-deconfinement transitions in the model, predicted to be first order within the mean-field theory, may become second order via a defect condensation mechanism. Finally, we present a general classification of the perturbations to the Kitaev model on the basis of their effect on it's spin correlation functions and derive a necessary and sufficient condition, within the regime of validity of perturbation theory, for the spin correlators to exhibit a long-ranged power-law behavior in the presence of such perturbations. Our results reproduce those of Tikhonov et al. [Phys. Rev. Lett. 106, 067203 (2011)] as a special case.