Efficient Bayesian inference for learning in the Ising linear perceptron and signal detection in CDMA

Efficient new Bayesian inference technique is employed for studying critical properties of the Ising linear perceptron and for signal detection in code division multiple access (CDMA). The approach is based on a recently introduced message passing technique for densely connected systems. Here we study both critical and non-critical regimes. Results obtained in the non-critical regime give rise to a highly efficient signal detection algorithm in the context of CDMA; while in the critical regime one observes a first-order transition line that ends in a continuous phase transition point. Finite size effects are also studied. © 2006 Elsevier B.V. All rights reserved.

Improved message passing for inference in densely connected systems

An improved inference method for densely connected systems is presented. The approach is based on passing condensed messages between variables, representing macroscopic averages of microscopic messages. We extend previous work that showed promising results in cases where the solution space is contiguous to cases where fragmentation occurs. We apply the method to the signal detection problem of Code Division Multiple Access (CDMA) for demonstrating its potential. A highly efficient practical algorithm is also derived on the basis of insight gained from the analysis. © EDP Sciences.

Bayesian inference for wind field retrieval

In many problems in spatial statistics it is necessary to infer a global problem solution by combining local models. A principled approach to this problem is to develop a global probabilistic model for the relationships between local variables and to use this as the prior in a Bayesian inference procedure. We show how a Gaussian process with hyper-parameters estimated from Numerical Weather Prediction Models yields meteorologically convincing wind fields. We use neural networks to make local estimates of wind vector probabilities. The resulting inference problem cannot be solved analytically, but Markov Chain Monte Carlo methods allow us to retrieve accurate wind fields.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

A basis function approach to Bayesian inference in diffusion processes

In this paper, we present a framework for Bayesian inference in continuous-time diffusion processes. The new method is directly related to the recently proposed variational Gaussian Process approximation (VGPA) approach to Bayesian smoothing of partially observed diffusions. By adopting a basis function expansion (BF-VGPA), both the time-dependent control parameters of the approximate GP process and its moment equations are projected onto a lower-dimensional subspace. This allows us both to reduce the computational complexity and to eliminate the time discretisation used in the previous algorithm. The new algorithm is tested on an Ornstein-Uhlenbeck process. Our preliminary results show that BF-VGPA algorithm provides a reasonably accurate state estimation using a small number of basis functions.

A comparison of variational and Markov chain Monte Carlo methods for inference in partially observed stochastic dynamic systems

In recent work we have developed a novel variational inference method for partially observed systems governed by stochastic differential equations. In this paper we provide a comparison of the Variational Gaussian Process Smoother with an exact solution computed using a Hybrid Monte Carlo approach to path sampling, applied to a stochastic double well potential model. It is demonstrated that the variational smoother provides us a very accurate estimate of mean path while conditional variance is slightly underestimated. We conclude with some remarks as to the advantages and disadvantages of the variational smoother. © 2008 Springer Science + Business Media LLC.

Variational inference for Student-t MLP models

This paper presents a novel methodology to infer parameters of probabilistic models whose output noise is a Student-t distribution. The method is an extension of earlier work for models that are linear in parameters to nonlinear multi-layer perceptrons (MLPs). We used an EM algorithm combined with variational approximation, the evidence procedure, and an optimisation algorithm. The technique was tested on two regression applications. The first one is a synthetic dataset and the second is gas forward contract prices data from the UK energy market. The results showed that forecasting accuracy is significantly improved by using Student-t noise models.

Data assimilation for precipitation nowcasting using Bayesian inference

This work introduces a new variational Bayes data assimilation method for the stochastic estimation of precipitation dynamics using radar observations for short term probabilistic forecasting (nowcasting). A previously developed spatial rainfall model based on the decomposition of the observed precipitation field using a basis function expansion captures the precipitation intensity from radar images as a set of ‘rain cells’. The prior distributions for the basis function parameters are carefully chosen to have a conjugate structure for the precipitation field model to allow a novel variational Bayes method to be applied to estimate the posterior distributions in closed form, based on solving an optimisation problem, in a spirit similar to 3D VAR analysis, but seeking approximations to the posterior distribution rather than simply the most probable state. A hierarchical Kalman filter is used to estimate the advection field based on the assimilated precipitation fields at two times. The model is applied to tracking precipitation dynamics in a realistic setting, using UK Met Office radar data from both a summer convective event and a winter frontal event. The performance of the model is assessed both traditionally and using probabilistic measures of fit based on ROC curves. The model is shown to provide very good assimilation characteristics, and promising forecast skill. Improvements to the forecasting scheme are discussed

The informal economy in the former Soviet Union and in Central and Eastern Europe

Guest editors' introduction

Workplace crime and the informal economy in Ukraine: Employee and employer perspectives

Purpose – The purpose of this paper is to explore the criminal workplace activities of both employers and employees in Ukrainian enterprises. It challenges traditional definitions of corruption and suggests that the practices that can be observed fit into the category of organised crime because of the country's economic framework. The paper also explores how the practices are partially a legacy of Soviet economic processes. Design/methodology/approach – A total of 700 household surveys were completed in three cities, Kyiv (where 450 surveys were completed), Uzhgorod (150) and Kharkiv (100). To complement these, approximately 25 in-depth interviews were undertaken with workers in each region. Furthermore, ethnographic observations and “kitchen table” interviews also played an important role in the research. Although the research was oriented towards those working in informal economies, business owners (both formal and informal) were also interviewed. Findings – As well as revealing the endemic nature of corruption in Ukrainian workplaces and the high levels of informal activity undertaken by workers, the research found that many people wish for their workplace to become more regulated. Research limitations/implications – Further interviews could have been carried out with state officials and in more locations. The implications are multiple but mainly they demonstrate the difficulty that those charged with economic reform in Ukraine must face. Originality/value – It is one of the first studies to explore these issues in Ukraine using a variety of research methods.

Everyday tactics and spaces of power: the role of informal economies in post-Soviet Ukraine

Utilising de Certeau's concepts of daily life and his delineation between strategies and tactics as everyday practices this paper examines the role of informal economies in post-Ukraine. Based on 700 household surveys and seventy-five in-depth interviews, conducted in three Ukrainian cities, the paper argues that individuals/households have developed a wide range of tactics in response to the economic marginalisation the country has endured since the collapse of the Soviet Union. Firstly, the paper details the importance of informal economies in contemporary Ukraine while highlighting that many such practices are operated out of necessity due to low wage and pension rates and high levels of corruption. This challenges state-produced statistics on the scale of economic marginalisation currently experienced in the country. By exploring a variety of these tactics the paper then examines how unequal power relations shape the spaces in which these practices operate in and how they can be simultaneously sites of exploitation and resistance to economic marginalisation. The paper concludes pessimistically by suggesting that the way in which these economic spaces are shaped precludes the development of state policies which might benefit the economically marginalised.

The end of transition?:The role of informal economies in the post-Soviet space

This book challenges the accepted notion that the transition from the command economy to market based systems is complete across the post-Soviet space. While it is noted that different political economies have developed in such states, such as Russia’s ‘managed democracy’, events such as Ukraine gaining ‘market economy status’ by the European Union and acceding to the World Trade Organisation in 2008 are taken as evidence that the reform period is over. Such thinking is based on numerous assumptions; specifically that economic transition has defined start and end points, that the formal economy now has primacy over other forms of economic practices and that national economic growth leads to the ‘trickle down’ of wealth to those marginalised by the transition process. Based on extensive ethnographic and quantitative research, conducted in Ukraine and Russia between 2004 - 2007, this book questions these assumptions by stating that the economies that operate across post-Soviet spaces are far from the textbook idea of a market economy. Through this the whole notion of ‘transition’ is problematised and the importance of informal economies to everyday life is demonstrated. Using case studies of various sectors, such as entrepreneurial behaviour and the higher education system, it is also shown how corruption has invaded almost all sectors of the post-Soviet every day.

Beyond the formal/informal economy binary hierarchy

Purpose – This paper aims to evaluate critically the conventional binary hierarchical representation of the formal/informal economy dualism which reads informal employment as a residual and marginal sphere that has largely negative consequences for economic development and needs to be deterred. Design/methodology/approach – To contest this depiction, the results of 600 household interviews conducted in Ukraine during 2005/2006 on the extent and nature of their informal employment are reported. Findings – Informal employment is revealed to be an extensively used form of work and, through a richer and more textured understanding of the multiple roles that different forms of informal employment play, a form of work that positively contributes to economic and social development, acting both as an important seedbed for enterprise creation and development and as a primary vehicle through which community self-help is delivered in contemporary Ukraine. Research limitations/implications – This survey reveals that depicting informal employment as a hindrance to development and deterring engagement in this sphere results in state authorities destroying the entrepreneurial endeavour and active citizenship that other public policies are seeking to nurture. The paper concludes by addressing how this public policy paradox might start to be resolved. Originality/value – This paper is one of the first to document the role of informal employment in nurturing enterprise creation and development as well as community exchange.

Evaluating the motives of informal entrepreneurs: some lessons from Ukraine

To understand entrepreneurs' motivations, it has become increasingly common to distinguish between those driven by necessity (or pushed) and those driven by opportunity (or pulled) into entrepreneurship. Until now, entrepreneurs operating wholly or partially in the informal economy have been widely assumed to be necessity-driven, pushed into this enterprise as a survival strategy in the absence of alternatives. To evaluate whether this is indeed the case, this paper reports one of the first surveys of informal entrepreneurs' motives. Reporting face-to-face interviews conducted in Ukraine during 2005–06 with 298 informal entrepreneurs, the finding is although most identified themselves as necessity entrepreneurs when initially asked whether they were either pushed or pulled, subsequent questions reveal in the vast majority of cases, there were not only both push and pull factors driving their original decision to start-up informal enterprises, but also a clear shift among these entrepreneurs as their business became established away from necessity-oriented motivations and toward more opportunity-oriented motivations. The outcome is a call for a transcendence of a static either/or approach and the adoption of a dynamic both/and approach that recognizes the coexistence of necessity- and opportunity-drivers as well as the fluidity of entrepreneurs' motivations.