Support Vector Machines: Training and Applications

The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Labs. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and Multi-Layer Perceptron classifiers. An interesting property of this approach is that it is an approximate implementation of the Structural Risk Minimization (SRM) induction principle. The derivation of Support Vector Machines, its relationship with SRM, and its geometrical insight, are discussed in this paper. Training a SVM is equivalent to solve a quadratic programming problem with linear and box constraints in a number of variables equal to the number of data points. When the number of data points exceeds few thousands the problem is very challenging, because the quadratic form is completely dense, so the memory needed to store the problem grows with the square of the number of data points. Therefore, training problems arising in some real applications with large data sets are impossible to load into memory, and cannot be solved using standard non-linear constrained optimization algorithms. We present a decomposition algorithm that can be used to train SVM's over large data sets. The main idea behind the decomposition is the iterative solution of sub-problems and the evaluation of, and also establish the stopping criteria for the algorithm. We present previous approaches, as well as results and important details of our implementation of the algorithm using a second-order variant of the Reduced Gradient Method as the solver of the sub-problems. As an application of SVM's, we present preliminary results we obtained applying SVM to the problem of detecting frontal human faces in real images.

Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Development, implementation and evaluation of an activated sludge supervisory system for the Granollers WWTP

La present tesi pretén recollir l'experiència viscuda en desenvolupar un sistema supervisor intel·ligent per a la millora de la gestió de plantes depuradores d'aigües residuals., implementar-lo en planta real (EDAR Granollers) i avaluar-ne el funcionament dia a dia amb situacions típiques de la planta. Aquest sistema supervisor combina i integra eines de control clàssic de les plantes depuradores (controlador automàtic del nivell d'oxigen dissolt al reactor biològic, ús de models descriptius del procés...) amb l'aplicació d'eines del camp de la intel·ligència artificial (sistemes basats en el coneixement, concretament sistemes experts i sistemes basats en casos, i xarxes neuronals). Aquest document s'estructura en 9 capítols diferents. Hi ha una primera part introductòria on es fa una revisió de l'estat actual del control de les EDARs i s'explica el perquè de la complexitat de la gestió d'aquests processos (capítol 1). Aquest capítol introductori juntament amb el capítol 2, on es pretén explicar els antecedents d'aquesta tesi, serveixen per establir els objectius d'aquest treball (capítol 3). A continuació, el capítol 4 descriu les peculiaritats i especificitats de la planta que s'ha escollit per implementar el sistema supervisor. Els capítols 5 i 6 del present document exposen el treball fet per a desenvolupar el sistema basat en regles o sistema expert (capítol 6) i el sistema basat en casos (capítol 7). El capítol 8 descriu la integració d'aquestes dues eines de raonament en una arquitectura multi nivell distribuïda. Finalment, hi ha una darrer capítol que correspon a la avaluació (verificació i validació), en primer lloc, de cadascuna de les eines per separat i, posteriorment, del sistema global en front de situacions reals que es donin a la depuradora

The influence of bottom topography on the decay of modeled Agulhas rings

The influence of a large meridional submarine ridge on the decay of Agulhas rings is investigated with a 1 and 2-layer setup of the isopycnic primitive-equation ocean model MICOM. In the single-layer case we show that the SSH decay of the ring is primarily governed by bottom friction and secondly by the radiation of Rossby waves. When a topographic ridge is present, the effect of the ridge on SSH decay and loss of tracer from the ring is negligible. However, the barotropic ring cannot pass the ridge due to energy and vorticity constraints. In the case of a two-layer ring the initial SSH decay is governed by a mixed barotropic–baroclinic instability of the ring. Again, radiation of barotropic Rossby waves is present. When the ring passes the topographic ridge, it shows a small but significant stagnation of SSH decay, agreeing with satellite altimetry observations. This is found to be due to a reduction of the growth rate of the m = 2 instability, to conversions of kinetic energy to the upper layer, and to a decrease in Rossby-wave radiation. The energy transfer is related to the fact that coherent structures in the lower layer cannot pass the steep ridge due to energy constraints. Furthermore, the loss of tracer from the ring through filamentation is less than for a ring moving over a flat bottom, related to a decrease in propagation speed of the ring. We conclude that ridges like the Walvis Ridge tend to stabilize a multi-layer ring and reduce its decay.

QUAGMIRE v1.3: a quasi-geostrophic model for investigating rotating fluids experiments

QUAGMIRE is a quasi-geostrophic numerical model for performing fast, high-resolution simulations of multi-layer rotating annulus laboratory experiments on a desktop personal computer. The model uses a hybrid finite-difference/spectral approach to numerically integrate the coupled nonlinear partial differential equations of motion in cylindrical geometry in each layer. Version 1.3 implements the special case of two fluid layers of equal resting depths. The flow is forced either by a differentially rotating lid, or by relaxation to specified streamfunction or potential vorticity fields, or both. Dissipation is achieved through Ekman layer pumping and suction at the horizontal boundaries, including the internal interface. The effects of weak interfacial tension are included, as well as the linear topographic beta-effect and the quadratic centripetal beta-effect. Stochastic forcing may optionally be activated, to represent approximately the effects of random unresolved features. A leapfrog time stepping scheme is used, with a Robert filter. Flows simulated by the model agree well with those observed in the corresponding laboratory experiments.

The turnover of organic carbon in subsoils. Part 2. Modelling carbon turnover

A new model, RothPC-1, is described for the turnover of organic C in the top metre of soil. RothPC-1 is a version of RothC-26.3, an earlier model for the turnover of C in topsoils. In RothPC-1 two extra parameters are used to model turnover in the top metre of soil: one, p, which moves organic C down the profile by an advective process, and the other, s, which slows decomposition with depth. RothPC-1 is parameterized and tested using measurements (described in Part 1, this issue) of total organic C and radiocarbon on soil profiles from the Rothamsted long-term field experiments, collected over a period of more than 100 years. RothPC-1 gives fits to measurements of organic C and radiocarbon in the 0-23, 23-46, 46-69 and 69-92 cm layers of soil that are almost all within (or close to) measurement error in two areas of regenerating woodland (Geescroft and Broadbalk Wildernesses) and an area of cultivated land from the Broadbalk Continuous Wheat Experiment. The fits to old grassland (the Park Grass Experiment) are less close. Two other sites that provide the requisite pre- and post-bomb data are also fitted; a prairie Chernozem from Russia and an annual grassland from California. Roth-PC-1 gives a close fit to measurements of organic C and radiocarbon down the Chernozem profile, provided that allowance is made for soil age; with the annual grassland the fit is acceptable in the upper part of the profile, but not in the clay-rich Bt horizon below. Calculations suggest that treating the top metre of soil as a homogeneous unit will greatly overestimate the effects of global warming in accelerating the decomposition of soil C and hence on the enhanced release of CO2 from soil organic matter; more realistic estimates will be obtained from multi-layer models such as RothPC-1.

A contour-advective semi-lagrangian numerical algorithm for simulating fine-scale conservative dynamical fields

This paper describes a novel numerical algorithm for simulating the evolution of fine-scale conservative fields in layer-wise two-dimensional flows, the most important examples of which are the earth's atmosphere and oceans. the algorithm combines two radically different algorithms, one Lagrangian and the other Eulerian, to achieve an unexpected gain in computational efficiency. The algorithm is demonstrated for multi-layer quasi-geostrophic flow, and results are presented for a simulation of a tilted stratospheric polar vortex and of nearly-inviscid quasi-geostrophic turbulence. the turbulence results contradict previous arguments and simulation results that have suggested an ultimate two-dimensional, vertically-coherent character of the flow. Ongoing extensions of the algorithm to the generally ageostrophic flows characteristic of planetary fluid dynamics are outlined.

A new Snow-SVAT to simulate the accumulation and ablation of seasonal snow cover beneath a forest canopy

A new snow-soil-vegetation-atmosphere transfer (Snow-SVAT) scheme, which simulates the accumulation and ablation of the snow cover beneath a forest canopy, is presented. The model was formulated by coupling a canopy optical and thermal radiation model to a physically-based multi-layer snow model. This canopy radiation model is physically-based yet requires few parameters, so can be used when extensive in-situ field measurements are not available. Other forest effects such as the reduction of wind speed, interception of snow on the canopy and the deposition of litter were incorporated within this combined model, SNOWCAN, which was tested with data taken as part of the Boreal Ecosystem-Atmosphere Study (BOREAS) international collaborative experiment. Snow depths beneath four different canopy types and at an open site were simulated. Agreement between observed and simulated snow depths was generally good, with correlation coefficients ranging between r^2=0.94 and r^2=0.98 for all sites where automatic measurements were available. However, the simulated date of total snowpack ablation generally occurred later than the observed date. A comparison between simulated solar radiation and limited measurements of sub-canopy radiation at one site indicates that the model simulates the sub-canopy downwelling solar radiation early in the season to within measurement uncertainty.

Chlorine tolerant, multilayer reverse-somosis membranes with high permeate flux and high salt rejection

A new class of high molecular weight polyethersulfone ionomers is described in which the ionic content can be varied, at will, over a very wide and fully-controllable range. A novel type of coating process enables these materials to be deposited from alcohol-type solvents as cohesive but very thin (50 – 250 nm) films on porous support-membranes, giving high-flux membranes (3.3 – 5.0 L m-2 h-1 bar-1) with very good, though not outstanding salt rejection (typically 92 - 96%). A secondary layer, of formaldehyde-cross-linked polyvinyl alcohol, can be deposited from aqueous solution on the surface of the ionomer membrane, and this layer increases salt rejection to greater than 99% without serious loss of water permeability. The final multi-layer membrane shows excellent chlorine tolerance in reverse-osmosis operation.

Statistical downscaling of river flows

An extensive statistical ‘downscaling’ study is done to relate large-scale climate information from a general circulation model (GCM) to local-scale river flows in SW France for 51 gauging stations ranging from nival (snow-dominated) to pluvial (rainfall-dominated) river-systems. This study helps to select the appropriate statistical method at a given spatial and temporal scale to downscale hydrology for future climate change impact assessment of hydrological resources. The four proposed statistical downscaling models use large-scale predictors (derived from climate model outputs or reanalysis data) that characterize precipitation and evaporation processes in the hydrological cycle to estimate summary flow statistics. The four statistical models used are generalized linear (GLM) and additive (GAM) models, aggregated boosted trees (ABT) and multi-layer perceptron neural networks (ANN). These four models were each applied at two different spatial scales, namely at that of a single flow-gauging station (local downscaling) and that of a group of flow-gauging stations having the same hydrological behaviour (regional downscaling). For each statistical model and each spatial resolution, three temporal resolutions were considered, namely the daily mean flows, the summary statistics of fortnightly flows and a daily ‘integrated approach’. The results show that flow sensitivity to atmospheric factors is significantly different between nival and pluvial hydrological systems which are mainly influenced, respectively, by shortwave solar radiations and atmospheric temperature. The non-linear models (i.e. GAM, ABT and ANN) performed better than the linear GLM when simulating fortnightly flow percentiles. The aggregated boosted trees method showed higher and less variable R2 values to downscale the hydrological variability in both nival and pluvial regimes. Based on GCM cnrm-cm3 and scenarios A2 and A1B, future relative changes of fortnightly median flows were projected based on the regional downscaling approach. The results suggest a global decrease of flow in both pluvial and nival regimes, especially in spring, summer and autumn, whatever the considered scenario. The discussion considers the performance of each statistical method for downscaling flow at different spatial and temporal scales as well as the relationship between atmospheric processes and flow variability.

On simple adaptive momentum

Simple Adaptive Momentum [1] was introduced as a simple means of speeding the training of multi-layer perceptrons (MLPs) by changing the momentum term depending on the angle between the current and previous changes in the weights of the MLP. In the original paper. the weight changes of the whole network are used in determining this angle. This paper considers adapting the momentum term using certain subsets of these weights. This idea was inspired by the author's object oriented approach to programming MLPs. successfully used in teaching students: this approach is also described. It is concluded that the angle is best determined using the weight changes in each layer separately.

Neural and wavelet network models for financial distress classification

This work analyzes the use of linear discriminant models, multi-layer perceptron neural networks and wavelet networks for corporate financial distress prediction. Although simple and easy to interpret, linear models require statistical assumptions that may be unrealistic. Neural networks are able to discriminate patterns that are not linearly separable, but the large number of parameters involved in a neural model often causes generalization problems. Wavelet networks are classification models that implement nonlinear discriminant surfaces as the superposition of dilated and translated versions of a single "mother wavelet" function. In this paper, an algorithm is proposed to select dilation and translation parameters that yield a wavelet network classifier with good parsimony characteristics. The models are compared in a case study involving failed and continuing British firms in the period 1997-2000. Problems associated with over-parameterized neural networks are illustrated and the Optimal Brain Damage pruning technique is employed to obtain a parsimonious neural model. The results, supported by a re-sampling study, show that both neural and wavelet networks may be a valid alternative to classical linear discriminant models.

Comparison of extrasystolic ECG signal classifiers using discrete wavelet transforms

This work compares and contrasts results of classifying time-domain ECG signals with pathological conditions taken from the MITBIH arrhythmia database. Linear discriminant analysis and a multi-layer perceptron were used as classifiers. The neural network was trained by two different methods, namely back-propagation and a genetic algorithm. Converting the time-domain signal into the wavelet domain reduced the dimensionality of the problem at least 10-fold. This was achieved using wavelets from the db6 family as well as using adaptive wavelets generated using two different strategies. The wavelet transforms used in this study were limited to two decomposition levels. A neural network with evolved weights proved to be the best classifier with a maximum of 99.6% accuracy when optimised wavelet-transform ECG data wits presented to its input and 95.9% accuracy when the signals presented to its input were decomposed using db6 wavelets. The linear discriminant analysis achieved a maximum classification accuracy of 95.7% when presented with optimised and 95.5% with db6 wavelet coefficients. It is shown that the much simpler signal representation of a few wavelet coefficients obtained through an optimised discrete wavelet transform facilitates the classification of non-stationary time-variant signals task considerably. In addition, the results indicate that wavelet optimisation may improve the classification ability of a neural network. (c) 2005 Elsevier B.V. All rights reserved.

Reversible ageing effects in cryogenically cooled infrared filter radiometers

In this paper we report the observation of drifts in the responsivity of cryogenically cooled InSb detector-based infrared filter radiometers which have very strong wavelength dependence. These drifts can result in the increase or decrease of the response of the filter radiometers by over 5%. The origin of these variations was investigated and was shown to arise due to a thin film of ice formed on the multi-layer bandpass filter used to define the spectral response of the filter radiometer. The thin layer of ice interacts with the characteristics of the filter (which itself consists of a number of thin layers) and modifies the filter spectral transmission thus modifying the response of the filter radiometer of which the filter is part of. These observations are particularly relevant to space instruments which use infrared filter radiometers for earth observation. Debris from the spacecraft engines is known to accumulate on cold surfaces of instruments carried on board. The deposition of this debris on cold filters can modify the spectral response of the instruments, which use these filters to define a spectral response. Crown Copyright (c) 2004 Published by Elsevier B.V. All rights reserved.

Improvements in or relating to the manufacture of multilayer interference filters

This invention relates to the manufacture of multi-layer interference filters for use with infra-red radiation, especially at wavelengths beyond 3.8 microns. A method of manufacturing a multi-layer interference filter comprising the steps of forming on a substrate successive layers of lead telluride and another material in alternation, under conditions in which sufficient oxygen is included in the lead telluride layers to reduce the apparent free charge carrier concentration therein, so that the resulting filter exhibits enhanced transparency to radiation of wavelengths greater than 3.8 microns and enhanced natural absorption to radiation of wavelength less than 3.8 microns.