New variational Bayesian approaches for statistical data mining : with applications to profiling and differentiating habitual consumption behaviour of customers in the wireless telecommunication industry

This thesis investigates profiling and differentiating customers through the use of statistical data mining techniques. The business application of our work centres on examining individuals’ seldomly studied yet critical consumption behaviour over an extensive time period within the context of the wireless telecommunication industry; consumption behaviour (as oppose to purchasing behaviour) is behaviour that has been performed so frequently that it become habitual and involves minimal intentions or decision making. Key variables investigated are the activity initialised timestamp and cell tower location as well as the activity type and usage quantity (e.g., voice call with duration in seconds); and the research focuses are on customers’ spatial and temporal usage behaviour. The main methodological emphasis is on the development of clustering models based on Gaussian mixture models (GMMs) which are fitted with the use of the recently developed variational Bayesian (VB) method. VB is an efficient deterministic alternative to the popular but computationally demandingMarkov chainMonte Carlo (MCMC) methods. The standard VBGMMalgorithm is extended by allowing component splitting such that it is robust to initial parameter choices and can automatically and efficiently determine the number of components. The new algorithm we propose allows more effective modelling of individuals’ highly heterogeneous and spiky spatial usage behaviour, or more generally human mobility patterns; the term spiky describes data patterns with large areas of low probability mixed with small areas of high probability. Customers are then characterised and segmented based on the fitted GMM which corresponds to how each of them uses the products/services spatially in their daily lives; this is essentially their likely lifestyle and occupational traits. Other significant research contributions include fitting GMMs using VB to circular data i.e., the temporal usage behaviour, and developing clustering algorithms suitable for high dimensional data based on the use of VB-GMM.

On variational data assimilation in continuous time

Variational data assimilation in continuous time is revisited. The central techniques applied in this paper are in part adopted from the theory of optimal nonlinear control. Alternatively, the investigated approach can be considered as a continuous time generalization of what is known as weakly constrained four-dimensional variational assimilation (4D-Var) in the geosciences. The technique allows to assimilate trajectories in the case of partial observations and in the presence of model error. Several mathematical aspects of the approach are studied. Computationally, it amounts to solving a two-point boundary value problem. For imperfect models, the trade-off between small dynamical error (i.e. the trajectory obeys the model dynamics) and small observational error (i.e. the trajectory closely follows the observations) is investigated. This trade-off turns out to be trivial if the model is perfect. However, even in this situation, allowing for minute deviations from the perfect model is shown to have positive effects, namely to regularize the problem. The presented formalism is dynamical in character. No statistical assumptions on dynamical or observational noise are imposed.

Variational second order flow estimation for PIV sequences

[EN] We present in this paper a variational approach to accurately estimate simultaneously the velocity field and its derivatives directly from PIV image sequences. Our method differs from other techniques that have been presented in the literature in the fact that the energy minimization used to estimate the particles motion depends on a second order Taylor development of the flow. In this way, we are not only able to compute the motion vector field, but we also obtain an accurate estimation of their derivatives. Hence, we avoid the use of numerical schemes to compute the derivatives from the estimated flow that usually yield to numerical amplification of the inherent uncertainty on the estimated flow. The performance of our approach is illustrated with the estimation of the motion vector field and the vorticity on both synthetic and real PIV datasets.

Spectral and variational characterizations of solutions to semilinear eigenvalue problems

Die vorliegende Arbeit befaßt sich mit einer Klasse von nichtlinearen Eigenwertproblemen mit Variationsstrukturin einem reellen Hilbertraum. Die betrachteteEigenwertgleichung ergibt sich demnach als Euler-Lagrange-Gleichung eines stetig differenzierbarenFunktionals, zusätzlich sei der nichtlineare Anteil desProblems als ungerade und definit vorausgesetzt.Die wichtigsten Ergebnisse in diesem abstrakten Rahmen sindKriterien für die Existenz spektral charakterisierterLösungen, d.h. von Lösungen, deren Eigenwert gerade miteinem vorgegeben variationellen Eigenwert eines zugehörigen linearen Problems übereinstimmt. Die Herleitung dieserKriterien basiert auf einer Untersuchung kontinuierlicher Familien selbstadjungierterEigenwertprobleme und erfordert Verallgemeinerungenspektraltheoretischer Konzepte.Neben reinen Existenzsätzen werden auch Beziehungen zwischenspektralen Charakterisierungen und denLjusternik-Schnirelman-Niveaus des Funktionals erörtert.Wir betrachten Anwendungen auf semilineareDifferentialgleichungen (sowieIntegro-Differentialgleichungen) zweiter Ordnung. Diesliefert neue Informationen über die zugehörigenLösungsmengen im Hinblick auf Knoteneigenschaften. Diehergeleiteten Methoden eignen sich besonders für eindimensionale und radialsymmetrische Probleme, während einTeil der Resultate auch ohne Symmetrieforderungen gültigist.

Spatial and temporal characterisation of ground deformation recorded by geodetic techniques

A critical point in the analysis of ground displacements time series is the development of data driven methods that allow the different sources that generate the observed displacements to be discerned and characterised. A widely used multivariate statistical technique is the Principal Component Analysis (PCA), which allows reducing the dimensionality of the data space maintaining most of the variance of the dataset explained. Anyway, PCA does not perform well in finding the solution to the so-called Blind Source Separation (BSS) problem, i.e. in recovering and separating the original sources that generated the observed data. This is mainly due to the assumptions on which PCA relies: it looks for a new Euclidean space where the projected data are uncorrelated. The Independent Component Analysis (ICA) is a popular technique adopted to approach this problem. However, the independence condition is not easy to impose, and it is often necessary to introduce some approximations. To work around this problem, I use a variational bayesian ICA (vbICA) method, which models the probability density function (pdf) of each source signal using a mix of Gaussian distributions. This technique allows for more flexibility in the description of the pdf of the sources, giving a more reliable estimate of them. Here I present the application of the vbICA technique to GPS position time series. First, I use vbICA on synthetic data that simulate a seismic cycle (interseismic + coseismic + postseismic + seasonal + noise) and a volcanic source, and I study the ability of the algorithm to recover the original (known) sources of deformation. Secondly, I apply vbICA to different tectonically active scenarios, such as the 2009 L'Aquila (central Italy) earthquake, the 2012 Emilia (northern Italy) seismic sequence, and the 2006 Guerrero (Mexico) Slow Slip Event (SSE).

Space-time ocean wave measurement using variational stereo vision systems

Remote sensing imaging systems for the measurement of oceanic sea states have recently attracted renovated attention. Imaging technology is economical, non-invasive and enables a better understanding of the space-time dynamics of ocean waves over an area rather than at selected point locations of previous monitoring methods (buoys, wave gauges, etc.). We present recent progress in space-time measurement of ocean waves using stereo vision systems on offshore platforms. Both traditional disparity-based systems and modern elevation-based ones are presented in a variational optimization framework: the main idea is to pose the stereoscopic reconstruction problem of the surface of the ocean in a variational setting and design an energy functional whose minimizer is the desired temporal sequence of wave heights. The functional combines photometric observations as well as spatial and temporal smoothness priors. Disparity methods estimate the disparity between images as an intermediate step toward retrieving the depth of the waves with respect to the cameras, whereas elevation methods estimate the ocean surface displacements directly in 3-D space. Both techniques are used to measure ocean waves from real data collected at offshore platforms in the Black Sea (Crimean Peninsula, Ukraine) and the Northern Adriatic Sea (Venice coast, Italy). Then, the statistical and spectral properties of the resulting observed waves are analyzed. We show the advantages and disadvantages of the presented stereo vision systems and discuss the improvement of their performance in critical issues such as the robustness of the camera calibration in spite of undesired variations of the camera parameters.

Joint 4-D variational stereo reconstruction and camera calibration refinement for oceanic sea state measurements

Validating modern oceanographic theories using models produced through stereo computer vision principles has recently emerged. Space-time (4-D) models of the ocean surface may be generated by stacking a series of 3-D reconstructions independently generated for each time instant or, in a more robust manner, by simultaneously processing several snapshots coherently in a true ?4-D reconstruction.? However, the accuracy of these computer-vision-generated models is subject to the estimations of camera parameters, which may be corrupted under the influence of natural factors such as wind and vibrations. Therefore, removing the unpredictable errors of the camera parameters is necessary for an accurate reconstruction. In this paper, we propose a novel algorithm that can jointly perform a 4-D reconstruction as well as correct the camera parameter errors introduced by external factors. The technique is founded upon variational optimization methods to benefit from their numerous advantages: continuity of the estimated surface in space and time, robustness, and accuracy. The performance of the proposed algorithm is tested using synthetic data produced through computer graphics techniques, based on which the errors of the camera parameters arising from natural factors can be simulated.

Application of the homogenization techniques to the determination of the effective flexure constants of plate structural systems

A Mindlin plate with periodically distributed ribs patterns is analyzed by using homogenization techniques based on asymptotic expansion methods. The stiffness matrix of the homogenized plate is found to be dependent on the geometrical characteristics of the periodical cell, i.e. its skewness, plan shape, thickness variation etc. and on the plate material elastic constants. The computation of this plate stiffness matrix is carried out by averaging over the cell domain some solutions of different periodical boundary value problems. These boundary value problems are defined in variational form by linear first order differential operators on the cell domain and the boundary conditions of the variational equation correspond to a periodic structural problem. The elements of the stiffness matrix of homogenized plate are obtained by linear combinations of the averaged solution functions of the above mentioned boundary value problems. Finally, an illustrative example of application of this homogenization technique to hollowed plates and plate structures with ribs patterns regularly arranged over its area is shown. The possibility of using in the profesional practice the present procedure to the actual analysis of floors of typical buildings is also emphasized.

Approximate Bayesian techniques for inference in stochastic dynamical systems

This thesis is concerned with approximate inference in dynamical systems, from a variational Bayesian perspective. When modelling real world dynamical systems, stochastic differential equations appear as a natural choice, mainly because of their ability to model the noise of the system by adding a variant of some stochastic process to the deterministic dynamics. Hence, inference in such processes has drawn much attention. Here two new extended frameworks are derived and presented that are based on basis function expansions and local polynomial approximations of a recently proposed variational Bayesian algorithm. It is shown that the new extensions converge to the original variational algorithm and can be used for state estimation (smoothing). However, the main focus is on estimating the (hyper-) parameters of these systems (i.e. drift parameters and diffusion coefficients). The new methods are numerically validated on a range of different systems which vary in dimensionality and non-linearity. These are the Ornstein-Uhlenbeck process, for which the exact likelihood can be computed analytically, the univariate and highly non-linear, stochastic double well and the multivariate chaotic stochastic Lorenz '63 (3-dimensional model). The algorithms are also applied to the 40 dimensional stochastic Lorenz '96 system. In this investigation these new approaches are compared with a variety of other well known methods such as the ensemble Kalman filter / smoother, a hybrid Monte Carlo sampler, the dual unscented Kalman filter (for jointly estimating the systems states and model parameters) and full weak-constraint 4D-Var. Empirical analysis of their asymptotic behaviour as a function of observation density or length of time window increases is provided.

Variational inference for diffusion processes

Diffusion processes are a family of continuous-time continuous-state stochastic processes that are in general only partially observed. The joint estimation of the forcing parameters and the system noise (volatility) in these dynamical systems is a crucial, but non-trivial task, especially when the system is nonlinear and multimodal. We propose a variational treatment of diffusion processes, which allows us to compute type II maximum likelihood estimates of the parameters by simple gradient techniques and which is computationally less demanding than most MCMC approaches. We also show how a cheap estimate of the posterior over the parameters can be constructed based on the variational free energy.

On a Variational Approach to some Quasilinear Problems

We prove some multiplicity results concerning quasilinear elliptic equations with natural growth conditions. Techniques of nonsmooth critical point theory are employed.

Variational mean-field approach to the double-exchange model

It has been recently shown that the double exchange Hamiltonian, with weak antiferromagnetic interactions, has a richer variety of first- and second-order transitions than previously anticipated, and that such transitions are consistent with the magnetic properties of manganites. Here we present a thorough discussion of the variational mean-field approach that leads to these results. We also show that the effect of the Berry phase turns out to be crucial to produce first-order paramagnetic-ferromagnetic transitions near half filling with transition temperatures compatible with the experimental situation. The computation relies on two crucial facts: the use of a mean-field ansatz that retains the complexity of a system of electrons with off-diagonal disorder, not fully taken into account by the mean-field techniques, and the small but significant antiferromagnetic superexchange interaction between the localized spins.