A mean field theory of coded CDMA systems

We present a mean field theory of code-division multiple access (CDMA) systems with error-control coding. On the basis of the relation between the free energy and mutual information, we obtain an analytical expression of the maximum spectral efficiency of the coded CDMA system, from which a mean field description of the coded CDMA system is provided in terms of a bank of scalar Gaussian channels whose variances in general vary at different code symbol positions. Regular low-density parity-check (LDPC)-coded CDMA systems are also discussed as an example of the coded CDMA systems.

Advanced mean field methods:theory and practice

A major problem in modern probabilistic modeling is the huge computational complexity involved in typical calculations with multivariate probability distributions when the number of random variables is large. Because exact computations are infeasible in such cases and Monte Carlo sampling techniques may reach their limits, there is a need for methods that allow for efficient approximate computations. One of the simplest approximations is based on the mean field method, which has a long history in statistical physics. The method is widely used, particularly in the growing field of graphical models. Researchers from disciplines such as statistical physics, computer science, and mathematical statistics are studying ways to improve this and related methods and are exploring novel application areas. Leading approaches include the variational approach, which goes beyond factorizable distributions to achieve systematic improvements; the TAP (Thouless-Anderson-Palmer) approach, which incorporates correlations by including effective reaction terms in the mean field theory; and the more general methods of graphical models. Bringing together ideas and techniques from these diverse disciplines, this book covers the theoretical foundations of advanced mean field methods, explores the relation between the different approaches, examines the quality of the approximation obtained, and demonstrates their application to various areas of probabilistic modeling.

Gaussian processes for classification: mean field algorithms:mean-field algorithms

We derive a mean field algorithm for binary classification with Gaussian processes which is based on the TAP approach originally proposed in Statistical Physics of disordered systems. The theory also yields an approximate leave-one-out estimator for the generalization error which is computed with no extra computational cost. We show that from the TAP approach, it is possible to derive both a simpler 'naive' mean field theory and support vector machines (SVM) as limiting cases. For both mean field algorithms and support vectors machines, simulation results for three small benchmark data sets are presented. They show 1. that one may get state of the art performance by using the leave-one-out estimator for model selection and 2. the built-in leave-one-out estimators are extremely precise when compared to the exact leave-one-out estimate. The latter result is a taken as a strong support for the internal consistency of the mean field approach.

Mean field methods for classification with Gaussian processes

We discuss the Application of TAP mean field methods known from Statistical Mechanics of disordered systems to Bayesian classification with Gaussian processes. In contrast to previous applications, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.

Gaussian processes and SVM: Mean field results and leave-one-out

In this chapter, we elaborate on the well-known relationship between Gaussian processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational problems arising in GP and SVM. The first one is the calculation of the posterior mean for GP classifiers using a `naive' mean field approach. The second one is a leave-one-out estimator for the generalization error of SVM based on a linear response method. Simulation results on a benchmark dataset show similar performances for the GP mean field algorithm and the SVM algorithm. The approximate leave-one-out estimator is found to be in very good agreement with the exact leave-one-out error.

Generalized mean field approximation for parallel dynamics of the Ising model

The dynamics of the non-equilibrium Ising model with parallel updates is investigated using a generalized mean field approximation that incorporates multiple two-site correlations at any two time steps, which can be obtained recursively. The proposed method shows significant improvement in predicting local system properties compared to other mean field approximation techniques, particularly in systems with symmetric interactions. Results are also evaluated against those obtained from Monte Carlo simulations. The method is also employed to obtain parameter values for the kinetic inverse Ising modeling problem, where couplings and local field values of a fully connected spin system are inferred from data. © 2014 IOP Publishing Ltd and SISSA Medialab srl.

Mean field methods for Gaussian process classifiers

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Variational mean-field algorithm for efficient inference in large systems of stochastic differential equations

This work introduces a Gaussian variational mean-field approximation for inference in dynamical systems which can be modeled by ordinary stochastic differential equations. This new approach allows one to express the variational free energy as a functional of the marginal moments of the approximating Gaussian process. A restriction of the moment equations to piecewise polynomial functions, over time, dramatically reduces the complexity of approximate inference for stochastic differential equation models and makes it comparable to that of discrete time hidden Markov models. The algorithm is demonstrated on state and parameter estimation for nonlinear problems with up to 1000 dimensional state vectors and compares the results empirically with various well-known inference methodologies.

The charge effect on the hindrance factors for diffusion and convection of a solute in pores II

The diffusion and convection of a solute suspended in a fluid across porous membranes are known to be reduced compared to those in a bulk solution, owing to the fluid mechanical interaction between the solute and the pore wall as well as steric restriction. If the solute and the pore wall are electrically charged, the electrostatic interaction between them could affect the hindrance to diffusion and convection. In this study, the transport of charged spherical solutes through charged circular cylindrical pores filled with an electrolyte solution containing small ions was studied numerically by using a fluid mechanical and electrostatic model. Based on a mean field theory, the electrostatic interaction energy between the solute and the pore wall was estimated from the Poisson-Boltzmann equation, and the charge effect on the solute transport was examined for the solute and pore wall of like charge. The results were compared with those obtained from the linearized form of the Poisson-Boltzmann equation, i.e.the Debye-Hückel equation. © 2012 The Japan Society of Fluid Mechanics and IOP Publishing Ltd.

Charge effects on hindrance factors for diffusion and convection of solute in pores I

The transport of a spherical solute through a long circular cylindrical pore filled with an electrolyte solution is studied numerically, in the presence of constant surface charge on the solute and the pore wall. Fluid dynamic analyses were carried out to calculate the flow field around the solute in the pore to evaluate the drag coefficients exerted on the solute. Electrical potentials around the solute in the electrolyte solution were computed based on a mean-field theory to provide the interaction energy between the charged solute and the pore wall. Combining the results of the fluid dynamic and electrostatic analyses, we estimated the rate of the diffusive and convective transport of the solute across the pore. Although the present estimates of the drag coefficients on the solute suggest more than 10% difference from existing studies, depending on the radius ratio of the solute relative to the pore and the radial position of the solute center in the pore, this difference leads to a minor effect on the hindrance factors. It was found that even at rather large ion concentrations, the repulsive electrostatic interaction between the charged solute and the pore wall of like charge could significantly reduce the transport rate of the solute.

Charge effects on the hindered transport of macromolecules across the endothelial surface glycocalyx layer

A fluid mechanical and electrostatic model for the transport of solute molecules across the vascular endothelial surface glycocalyx layer (EGL) was developed to study the charge effect on the diffusive and convective transport of the solutes. The solute was assumed to be a spherical particle with a constant surface charge density, and the EGL was represented as an array of periodically arranged circular cylinders of like charge, with a constant surface charge density. By combining the fluid mechanical analyses for the flow around a solute suspended in an electrolyte solution and the electrostatic analyses for the free energy of the interaction between the solute and cylinders based on a mean field theory, we estimated the transport coefficients of the solute across the EGL. Both of diffusive and convective transports are reduced compared to those for an uncharged system, due to the stronger exclusion of the solute that results from the repulsive electrostatic interaction. The model prediction for the reflection coefficient for serum albumin agreed well with experimental observations if the charge density in the EGL is ranged from approximately -10 to -30 mEq/l.

TAP for parity check error correcting codes

We employ the methods presented in the previous chapter for decoding corrupted codewords, encoded using sparse parity check error correcting codes. We show the similarity between the equations derived from the TAP approach and those obtained from belief propagation, and examine their performance as practical decoding methods.

The TAP approach to intensive and extensive connectivity systems

The Thouless-Anderson-Palmer (TAP) approach was originally developed for analysing the Sherrington-Kirkpatrick model in the study of spin glass models and has been employed since then mainly in the context of extensively connected systems whereby each dynamical variable interacts weakly with the others. Recently, we extended this method for handling general intensively connected systems where each variable has only O(1) connections characterised by strong couplings. However, the new formulation looks quite different with respect to existing analyses and it is only natural to question whether it actually reproduces known results for systems of extensive connectivity. In this chapter, we apply our formulation of the TAP approach to an extensively connected system, the Hopfield associative memory model, showing that it produces identical results to those obtained by the conventional formulation.

Synchronization processes in nonlinear systems and their relation to cortical oscillatory dynamics

This thesis was focused on theoretical models of synchronization to cortical dynamics as measured by magnetoencephalography (MEG). Dynamical systems theory was used in both identifying relevant variables for brain coordination and also in devising methods for their quantification. We presented a method for studying interactions of linear and chaotic neuronal sources using MEG beamforming techniques. We showed that such sources can be accurately reconstructed in terms of their location, temporal dynamics and possible interactions. Synchronization in low-dimensional nonlinear systems was studied to explore specific correlates of functional integration and segregation. In the case of interacting dissimilar systems, relevant coordination phenomena involved generalized and phase synchronization, which were often intermittent. Spatially-extended systems were then studied. For locally-coupled dissimilar systems, as in the case of cortical columns, clustering behaviour occurred. Synchronized clusters emerged at different frequencies and their boundaries were marked through oscillation death. The macroscopic mean field revealed sharp spectral peaks at the frequencies of the clusters and broader spectral drops at their boundaries. These results question existing models of Event Related Synchronization and Desynchronization. We re-examined the concept of the steady-state evoked response following an AM stimulus. We showed that very little variability in the AM following response could be accounted by system noise. We presented a methodology for detecting local and global nonlinear interactions from MEG data in order to account for residual variability. We found crosshemispheric nonlinear interactions of ongoing cortical rhythms concurrent with the stimulus and interactions of these rhythms with the following AM responses. Finally, we hypothesized that holistic spatial stimuli would be accompanied by the emergence of clusters in primary visual cortex resulting in frequency-specific MEG oscillations. Indeed, we found different frequency distributions in induced gamma oscillations for different spatial stimuli, which was suggestive of temporal coding of these spatial stimuli. Further, we addressed the bursting character of these oscillations, which was suggestive of intermittent nonlinear dynamics. However, we did not observe the characteristic-3/2 power-law scaling in the distribution of interburst intervals. Further, this distribution was only seldom significantly different to the one obtained in surrogate data, where nonlinear structure was destroyed. In conclusion, the work presented in this thesis suggests that advances in dynamical systems theory in conjunction with developments in magnetoencephalography may facilitate a mapping between levels of description int he brain. this may potentially represent a major advancement in neuroscience.

The single-phase induction motor:a critical appraisal of the rotating-field and cross-field theories with particular reference to skin effect

The purpose of this thesis is twofold: to examine the validity of the rotating-field and cross-field theories of the single-phase induction motor when applied to a cage rotor machine; and to examine the extent to which skin effect is likely to modify the characteristics of a cage rotor machine. A mathematical analysis is presented for a single-phase induction motor in which the rotor parameters are modified by skin effect. Although this is based on the usual type of ideal machine, a new form of model rotor allows approximations for skin effect phenomena to be included as an integral part of the analysis. Performance equations appropriate to the rotating-field and cross-field theories are deduced, and the corresponding explanations for the steady-state mode of operation are critically examined. The evaluation of the winding currents and developed torque is simplified by the introduction of new dimensionless factors which are functions of the resistance/reactance ratios of the rotor and the speed. Tables of the factors are included for selected numerical values of the parameter ratios, and these are used to deduce typical operating characteristics for both cage and wound rotor machines. It is shown that a qualitative explanation of the mode of operation of a cage rotor machine is obtained from either theory; but the operating characteristics must be deduced from the performance equations of the rotating-field theory, because of the restrictions on the values of the rotor parameters imposed by skin effect.