Time Series Input Selection using Multiple Kernel Learning

In this paper we study the relevance of multiple kernel learning (MKL) for the automatic selection of time series inputs. Recently, MKL has gained great attention in the machine learning community due to its flexibility in modelling complex patterns and performing feature selection. In general, MKL constructs the kernel as a weighted linear combination of basis kernels, exploiting different sources of information. An efficient algorithm wrapping a Support Vector Regression model for optimizing the MKL weights, named SimpleMKL, is used for the analysis. In this sense, MKL performs feature selection by discarding inputs/kernels with low or null weights. The approach proposed is tested with simulated linear and nonlinear time series (AutoRegressive, Henon and Lorenz series).

Multiple Kernel Learning of Environmental Data. Case study: Analysis and Mapping of Wind Fields

The paper presents the Multiple Kernel Learning (MKL) approach as a modelling and data exploratory tool and applies it to the problem of wind speed mapping. Support Vector Regression (SVR) is used to predict spatial variations of the mean wind speed from terrain features (slopes, terrain curvature, directional derivatives) generated at different spatial scales. Multiple Kernel Learning is applied to learn kernels for individual features and thematic feature subsets, both in the context of feature selection and optimal parameters determination. An empirical study on real-life data confirms the usefulness of MKL as a tool that enhances the interpretability of data-driven models.

Learning wind fields with multiple kernels

This paper presents multiple kernel learning (MKL) regression as an exploratory spatial data analysis and modelling tool. The MKL approach is introduced as an extension of support vector regression, where MKL uses dedicated kernels to divide a given task into sub-problems and to treat them separately in an effective way. It provides better interpretability to non-linear robust kernel regression at the cost of a more complex numerical optimization. In particular, we investigate the use of MKL as a tool that allows us to avoid using ad-hoc topographic indices as covariables in statistical models in complex terrains. Instead, MKL learns these relationships from the data in a non-parametric fashion. A study on data simulated from real terrain features confirms the ability of MKL to enhance the interpretability of data-driven models and to aid feature selection without degrading predictive performances. Here we examine the stability of the MKL algorithm with respect to the number of training data samples and to the presence of noise. The results of a real case study are also presented, where MKL is able to exploit a large set of terrain features computed at multiple spatial scales, when predicting mean wind speed in an Alpine region.

Multi-source composite kernels for urban image classification

This letter presents advanced classification methods for very high resolution images. Efficient multisource information, both spectral and spatial, is exploited through the use of composite kernels in support vector machines. Weighted summations of kernels accounting for separate sources of spectral and spatial information are analyzed and compared to classical approaches such as pure spectral classification or stacked approaches using all the features in a single vector. Model selection problems are addressed, as well as the importance of the different kernels in the weighted summation.

Kernel-based Mapping of Meteorological Fields in Complex Orography

Due to the advances in sensor networks and remote sensing technologies, the acquisition and storage rates of meteorological and climatological data increases every day and ask for novel and efficient processing algorithms. A fundamental problem of data analysis and modeling is the spatial prediction of meteorological variables in complex orography, which serves among others to extended climatological analyses, for the assimilation of data into numerical weather prediction models, for preparing inputs to hydrological models and for real time monitoring and short-term forecasting of weather.In this thesis, a new framework for spatial estimation is proposed by taking advantage of a class of algorithms emerging from the statistical learning theory. Nonparametric kernel-based methods for nonlinear data classification, regression and target detection, known as support vector machines (SVM), are adapted for mapping of meteorological variables in complex orography.With the advent of high resolution digital elevation models, the field of spatial prediction met new horizons. In fact, by exploiting image processing tools along with physical heuristics, an incredible number of terrain features which account for the topographic conditions at multiple spatial scales can be extracted. Such features are highly relevant for the mapping of meteorological variables because they control a considerable part of the spatial variability of meteorological fields in the complex Alpine orography. For instance, patterns of orographic rainfall, wind speed and cold air pools are known to be correlated with particular terrain forms, e.g. convex/concave surfaces and upwind sides of mountain slopes.Kernel-based methods are employed to learn the nonlinear statistical dependence which links the multidimensional space of geographical and topographic explanatory variables to the variable of interest, that is the wind speed as measured at the weather stations or the occurrence of orographic rainfall patterns as extracted from sequences of radar images. Compared to low dimensional models integrating only the geographical coordinates, the proposed framework opens a way to regionalize meteorological variables which are multidimensional in nature and rarely show spatial auto-correlation in the original space making the use of classical geostatistics tangled.The challenges which are explored during the thesis are manifolds. First, the complexity of models is optimized to impose appropriate smoothness properties and reduce the impact of noisy measurements. Secondly, a multiple kernel extension of SVM is considered to select the multiscale features which explain most of the spatial variability of wind speed. Then, SVM target detection methods are implemented to describe the orographic conditions which cause persistent and stationary rainfall patterns. Finally, the optimal splitting of the data is studied to estimate realistic performances and confidence intervals characterizing the uncertainty of predictions.The resulting maps of average wind speeds find applications within renewable resources assessment and opens a route to decrease the temporal scale of analysis to meet hydrological requirements. Furthermore, the maps depicting the susceptibility to orographic rainfall enhancement can be used to improve current radar-based quantitative precipitation estimation and forecasting systems and to generate stochastic ensembles of precipitation fields conditioned upon the orography.

Learning about Bayesian networks for forensic interpretation : An example based on the "the problem of multiple propositions"

Both, Bayesian networks and probabilistic evaluation are gaining more and more widespread use within many professional branches, including forensic science. Notwithstanding, they constitute subtle topics with definitional details that require careful study. While many sophisticated developments of probabilistic approaches to evaluation of forensic findings may readily be found in published literature, there remains a gap with respect to writings that focus on foundational aspects and on how these may be acquired by interested scientists new to these topics. This paper takes this as a starting point to report on the learning about Bayesian networks for likelihood ratio based, probabilistic inference procedures in a class of master students in forensic science. The presentation uses an example that relies on a casework scenario drawn from published literature, involving a questioned signature. A complicating aspect of that case study - proposed to students in a teaching scenario - is due to the need of considering multiple competing propositions, which is an outset that may not readily be approached within a likelihood ratio based framework without drawing attention to some additional technical details. Using generic Bayesian networks fragments from existing literature on the topic, course participants were able to track the probabilistic underpinnings of the proposed scenario correctly both in terms of likelihood ratios and of posterior probabilities. In addition, further study of the example by students allowed them to derive an alternative Bayesian network structure with a computational output that is equivalent to existing probabilistic solutions. This practical experience underlines the potential of Bayesian networks to support and clarify foundational principles of probabilistic procedures for forensic evaluation.

How multiple repetitions influence the processing of self-, famous and unknown names and faces: an ERP study.

Because we live in an extremely complex social environment, people require the ability to memorize hundreds or thousands of social stimuli. The aim of this study was to investigate the effect of multiple repetitions on the processing of names and faces varying in terms of pre-experimental familiarity. We measured both behavioral and electrophysiological responses to self-, famous and unknown names and faces in three phases of the experiment (in every phase, each type of stimuli was repeated a pre-determined number of times). We found that the negative brain potential in posterior scalp sites observed approximately 170 ms after the stimulus onset (N170) was insensitive to pre-experimental familiarity but showed slight enhancement with each repetition. The negative wave in the inferior-temporal regions observed at approximately 250 ms (N250) was affected by both pre-experimental (famous>unknown) and intra-experimental familiarity (the more repetitions, the larger N250). In addition, N170 and N250 for names were larger in the left inferior-temporal region, whereas right-hemispheric or bilateral patterns of activity for faces were observed. The subsequent presentations of famous and unknown names and faces were also associated with higher amplitudes of the positive waveform in the central-parietal sites analyzed in the 320-900 ms time-window (P300). In contrast, P300 remained unchanged after the subsequent presentations of self-name and self-face. Moreover, the P300 for unknown faces grew more quickly than for unknown names. The latter suggests that the process of learning faces is more effective than learning names, possibly because faces carry more semantic information.

Role of disease activity for decision making ability in early multiple sclerosis

Background and purpose: Decision making (DM) has been defined as the process through which a person forms preferences, selects and executes actions, and evaluates the outcome related to a selected choice. This ability represents an important factor for adequate behaviour in everyday life. DM impairment in multiple sclerosis (MS) has been previously reported. The purpose of the present study was to assess DM in patients with MS at the earliest clinically detectable time point of the disease. Methods: Patients with definite (n=109) or possible (clinically isolated syndrome, CIS; n=56) MS, a short disease duration (mean 2.3 years) and a minor neurological disability (mean EDSS 1.8) were compared to 50 healthy controls aged 18 to 60 years (mean age 32.2) using the Iowa Gambling Task (IGT). Subjects had to select a card from any of 4 decks (A/B [disadvantageous]; C/D [advantageous]). The game consisted of 100 trials then grouped in blocks of 20 cards for data analysis. Skill in DM was assessed by means of a learning index (LI) defined as the difference between the averaged last three block indexes and first two block indexes (LI=[(BI-3+BI-4+BI-5)/3-(BI-1+B2)/2]). Non parametric tests were used for statistical analysis. Results: LI was higher in the control group (0.24, SD 0.44) than in the MS group (0.21, SD 0.38), however without reaching statistical significance (p=0.7). Interesting differences were detected when MS patients were grouped according to phenotype. A trend to a difference between MS subgroups and controls was observed for LI (p=0.06), which became significant between MS subgroups (p=0.03). CIS patients who confirmed MS diagnosis by presenting a second relapse after study entry showed a dysfunction in the IGT in comparison to the other CIS (p=0.01) and definite MS (p=0.04) patients. In the opposite, CIS patients characterised by not entirely fulfilled McDonald criteria at inclusion and absence of relapse during the study showed an normal learning pattern on the IGT. Finally, comparing MS patients who developed relapses after study entry, those who remained clinically stable and controls, we observed impaired performances only in relapsing patients in comparison to stable patients (p=0.008) and controls (p=0.03). Discussion: These results raise the assumption of a sustained role for both MS relapsing activity and disease heterogeneity (i.e. infra-clinical severity or activity of MS) in the impaired process of decision making.

Imaging Properties of MS Lesions Correlate with Attention Deficits in Early Multiple Sclerosis

Introduction: Cognitive impairment affects 40-65% of multiple sclerosis (MS) patients, often since early stages of the disease (relapsing remitting MS, RRMS). Frequently affected functions are memory, attention or executive abilities but the most sensitive measure of cognitive deficits in early MS is the information processing speed (Amato, 2008). MRI has been extensively exploited to investigate the substrate of cognitive dysfunction in MS but the underlying physiopathological mechanisms remain unclear. White matter lesion load, whole-brain atrophy and cortical lesions' number play a role but correlations are in some cases modest (Rovaris, 2006; Calabrese, 2009). In this study, we aimed at characterizing and correlating the T1 relaxation times of cortical and sub-cortical lesions with cognitive deficits detected by neuropsychological tests in a group of very early RR MS patients. Methods: Ten female patients with very early RRMS (age: 31.6 ±4.7y; disease duration: 3.8 ±1.9y; EDSS disability score: 1.8 ±0.4) and 10 age- and gender-matched healthy volunteers (mean age: 31.2 ±5.8y) were included in the study. All participants underwent the following neuropsychological tests: Rao's Brief Repeatable Battery of Neuropsychological tests (BRB-N), Stockings of Cambridge, Trail Making Test (TMT, part A and B), Boston Naming Test, Hooper Visual Organization Test and copy of the Rey-Osterrieth Complex Figure. Within 2 weeks from neuropsychological assessment, participants underwent brain MRI at 3T (Magnetom Trio a Tim System, Siemens, Germany) using a 32-channel head coil. The imaging protocol included 3D sequences with 1x1x1.2 mm3 resolution and 256x256x160 matrix, except for axial 2D-FLAIR: -DIR (T2-weighted, suppressing both WM and CSF; Pouwels, 2006) -MPRAGE (T1-weighted; Mugler, 1991) -MP2RAGE (T1-weighted with T1 maps; Marques, 2010) -FLAIR SPACE (only for patient 4-10, T2-weighted; Mugler, 2001) -2D Axial FLAIR (0.9x0.9x2.5 mm3, 256x256x44 matrix). Lesions were identified by one experienced neurologist and radiologist using all contrasts, manually contoured and assigned to regional locations (cortical or sub-cortical). Lesion number, volume and T1 relaxation time were calculated for lesions in each contrast and in a merged mask representing the union of the lesions from all contrasts. T1 relaxation times of lesions were normalized with the mean T1 value in corresponding control regions of the healthy subjects. Statistical analysis was performed using GraphPad InStat software. Cognitive scores were compared between patients and controls with paired t-tests; p values ≤ 0.05 were considered significant. Spearmann correlation tests were performed between the cognitive tests, which differed significantly between patients and controls, and lesions' i) number ii) volume iii) T1 relaxation time iv) disease duration and v) years of study. Results: Cortical and sub-cortical lesions count, T1 values and volume are reported in Table 1 (A and B). All early RRMS patients showed cortical lesions (CLs) and the majority consisted of CLs type I (lesions with a cortical component extending to the sub-cortical tissue). The rest of cortical lesions were characterized as type II (intra-cortical lesions). No type III/IV lesions (large sub-pial lesions) were detected. RRMS patients were slightly less educated (13.5±2.5y vs. 16.3±1.8y of study, p=0.02) than the controls. Signs of cortical dysfunction (i.e. impaired learning, language, visuo-spatial skills or gnosis) were rare in all patients. However, patients showed on average lower scores on measures of visual attention and information processing speed (TMT-part A: p=0.01; TMT-part B: p=0.006; PASAT-included in the BRB-N: p=0.04). The T1 relaxation values of CLs type I negatively correlated with the TMT-part A score (r=0.78, p<0.01). The correlations of TMT-part B score and PASAT score with T1 relaxation time of lesions as well and the correlation between TMT-part A, TMT-part B and PASAT score with lesions' i) number ii) volume iii) disease duration and iv) years of study did not reach significance. In order to preclude possible influences from partial volume effects on the T1 values, the correlation between lesion volume and T1 value of CLs type I was calculated; no correlation was found, suggesting that partial volume effects did not affect the statistics. Conclusions: The present pilot study reports for the first time the presence and the T1 characteristics at 3 T of cortical lesions in very early RRMS (< 6 y disease duration). It also shows that CLS type I represents the most frequent cortical lesion type in this cohort of RRMS patients. In addition, it reveals a negative correlation between the attentional test TMT-part A and the T1 properties of cortical lesions type I. In other words, lower attention deficits are concomitant with longer T1-relaxation time in cortical lesions. In respect to this last finding, it could be speculated that long relaxation time correspond to a certain degree of tissue loss that is enough to stimulate compensatory mechanisms. This hypothesis is in line with previous fMRI studies showing functional compensatory mechanisms to help maintaining normal or sub-normal attention performances in RR MS patients (Penner, 2003).

Preserved decision making ability in early multiple sclerosis.

BACKGROUND: The purpose of this study was to assess decision making in patients with multiple sclerosis (MS) at the earliest clinically detectable time point of the disease. METHODS: Patients with definite MS (n = 109) or with clinically isolated syndrome (CIS, n = 56), a disease duration of 3 months to 5 years, and no or only minor neurological impairment (Expanded Disability Status Scale [EDSS] score 0-2.5) were compared to 50 healthy controls using the Iowa Gambling Task (IGT). RESULTS: The performance of definite MS, CIS patients, and controls was comparable for the two main outcomes of the IGT (learning index: p = 0.7; total score: p = 0.6). The IGT learning index was influenced by the educational level and the co-occurrence of minor depression. CIS and MS patients developing a relapse during an observation period of 15 months dated from IGT testing demonstrated a lower learning index in the IGT than patients who had no exacerbation (p = 0.02). When controlling for age, gender and education, the difference between relapsing and non-relapsing patients was at the limit of significance (p = 0.06). CONCLUSION: Decision making in a task mimicking real life decisions is generally preserved in early MS patients as compared to controls. A possible consequence of MS relapsing activity in the impairment of decision making ability is also suspected in the early phase of MS.

Kernel-based methods for change detection in remote sensing images

Nowadays, the joint exploitation of images acquired daily by remote sensing instruments and of images available from archives allows a detailed monitoring of the transitions occurring at the surface of the Earth. These modifications of the land cover generate spectral discrepancies that can be detected via the analysis of remote sensing images. Independently from the origin of the images and of type of surface change, a correct processing of such data implies the adoption of flexible, robust and possibly nonlinear method, to correctly account for the complex statistical relationships characterizing the pixels of the images. This Thesis deals with the development and the application of advanced statistical methods for multi-temporal optical remote sensing image processing tasks. Three different families of machine learning models have been explored and fundamental solutions for change detection problems are provided. In the first part, change detection with user supervision has been considered. In a first application, a nonlinear classifier has been applied with the intent of precisely delineating flooded regions from a pair of images. In a second case study, the spatial context of each pixel has been injected into another nonlinear classifier to obtain a precise mapping of new urban structures. In both cases, the user provides the classifier with examples of what he believes has changed or not. In the second part, a completely automatic and unsupervised method for precise binary detection of changes has been proposed. The technique allows a very accurate mapping without any user intervention, resulting particularly useful when readiness and reaction times of the system are a crucial constraint. In the third, the problem of statistical distributions shifting between acquisitions is studied. Two approaches to transform the couple of bi-temporal images and reduce their differences unrelated to changes in land cover are studied. The methods align the distributions of the images, so that the pixel-wise comparison could be carried out with higher accuracy. Furthermore, the second method can deal with images from different sensors, no matter the dimensionality of the data nor the spectral information content. This opens the doors to possible solutions for a crucial problem in the field: detecting changes when the images have been acquired by two different sensors.

Decoding sequence learning from single-trial intracranial EEG in humans.

We propose and validate a multivariate classification algorithm for characterizing changes in human intracranial electroencephalographic data (iEEG) after learning motor sequences. The algorithm is based on a Hidden Markov Model (HMM) that captures spatio-temporal properties of the iEEG at the level of single trials. Continuous intracranial iEEG was acquired during two sessions (one before and one after a night of sleep) in two patients with depth electrodes implanted in several brain areas. They performed a visuomotor sequence (serial reaction time task, SRTT) using the fingers of their non-dominant hand. Our results show that the decoding algorithm correctly classified single iEEG trials from the trained sequence as belonging to either the initial training phase (day 1, before sleep) or a later consolidated phase (day 2, after sleep), whereas it failed to do so for trials belonging to a control condition (pseudo-random sequence). Accurate single-trial classification was achieved by taking advantage of the distributed pattern of neural activity. However, across all the contacts the hippocampus contributed most significantly to the classification accuracy for both patients, and one fronto-striatal contact for one patient. Together, these human intracranial findings demonstrate that a multivariate decoding approach can detect learning-related changes at the level of single-trial iEEG. Because it allows an unbiased identification of brain sites contributing to a behavioral effect (or experimental condition) at the level of single subject, this approach could be usefully applied to assess the neural correlates of other complex cognitive functions in patients implanted with multiple electrodes.