Model selection approaches for non-linear system identification: a review

The identification of non-linear systems using only observed finite datasets has become a mature research area over the last two decades. A class of linear-in-the-parameter models with universal approximation capabilities have been intensively studied and widely used due to the availability of many linear-learning algorithms and their inherent convergence conditions. This article presents a systematic overview of basic research on model selection approaches for linear-in-the-parameter models. One of the fundamental problems in non-linear system identification is to find the minimal model with the best model generalisation performance from observational data only. The important concepts in achieving good model generalisation used in various non-linear system-identification algorithms are first reviewed, including Bayesian parameter regularisation and models selective criteria based on the cross validation and experimental design. A significant advance in machine learning has been the development of the support vector machine as a means for identifying kernel models based on the structural risk minimisation principle. The developments on the convex optimisation-based model construction algorithms including the support vector regression algorithms are outlined. Input selection algorithms and on-line system identification algorithms are also included in this review. Finally, some industrial applications of non-linear models are discussed.

On the spectra of certain integro-differential-delay problems with applications in neurodynamics

We investigate the spectrum of certain integro-differential-delay equations (IDDEs) which arise naturally within spatially distributed, nonlocal, pattern formation problems. Our approach is based on the reformulation of the relevant dispersion relations with the use of the Lambert function. As a particular application of this approach, we consider the case of the Amari delay neural field equation which describes the local activity of a population of neurons taking into consideration the finite propagation speed of the electric signal. We show that if the kernel appearing in this equation is symmetric around some point a= 0 or consists of a sum of such terms, then the relevant dispersion relation yields spectra with an infinite number of branches, as opposed to finite sets of eigenvalues considered in previous works. Also, in earlier works the focus has been on the most rightward part of the spectrum and the possibility of an instability driven pattern formation. Here, we numerically survey the structure of the entire spectra and argue that a detailed knowledge of this structure is important within neurodynamical applications. Indeed, the Amari IDDE acts as a filter with the ability to recognise and respond whenever it is excited in such a way so as to resonate with one of its rightward modes, thereby amplifying such inputs and dampening others. Finally, we discuss how these results can be generalised to the case of systems of IDDEs.

Probability density estimation with tunable kernels using orthogonal forward regression

A generalized or tunable-kernel model is proposed for probability density function estimation based on an orthogonal forward regression procedure. Each stage of the density estimation process determines a tunable kernel, namely, its center vector and diagonal covariance matrix, by minimizing a leave-one-out test criterion. The kernel mixing weights of the constructed sparse density estimate are finally updated using the multiplicative nonnegative quadratic programming algorithm to ensure the nonnegative and unity constraints, and this weight-updating process additionally has the desired ability to further reduce the model size. The proposed tunable-kernel model has advantages, in terms of model generalization capability and model sparsity, over the standard fixed-kernel model that restricts kernel centers to the training data points and employs a single common kernel variance for every kernel. On the other hand, it does not optimize all the model parameters together and thus avoids the problems of high-dimensional ill-conditioned nonlinear optimization associated with the conventional finite mixture model. Several examples are included to demonstrate the ability of the proposed novel tunable-kernel model to effectively construct a very compact density estimate accurately.

Sting jets in simulations of a real cyclone by two mesoscale models

The existence of sting jets as a potential source of damaging surface winds during the passage of extratropical cyclones has recently been recognized However, there are still very few published studies on the subject Furthermore, although ills known that other models are capable of reproducing sting jets, in the published literature only one numerical model [the Met Office Unified Model (MetUM)] has been used to numerically analyze these phenomena This article alms to improve our understanding of the processes that contribute to the development of sting jets and show that model differences affect the evolution of modeled sting jets A sting jet event during the passage of a cyclone over the United Kingdom on 26 February 2002 has been simulated using two mesoscale models namely the MetUM and the Consortium for Small Scale Modeling (COSMO) model to compare their performance Given the known critical importance of vertical resolution in the simulation of sting jets the vertical resolution of both models has been enhanced with respect to their operational versions Both simulations have been verified against surface measurements of maximum gusts, satellite imagery and Met Office operational synoptic analyses, as well as operational analyses from the ECMWF It is shown that both models are capable of reproducing sting jets with similar, though not identical. features Through the comparison of the results from these two models, the relevance of physical mechanisms, such as evaporative cooling and the release of conditional symmetric instability, in the generation and evolution of sting jets is also discussed

Computer generation and quantitative morphometric analysis of virtual neurons

An important goal in computational neuroanatomy is the complete and accurate simulation of neuronal morphology. We are developing computational tools to model three-dimensional dendritic structures based on sets of stochastic rules. This paper reports an extensive, quantitative anatomical characterization of simulated motoneurons and Purkinje cells. We used several local and global algorithms implemented in the L-Neuron and ArborVitae programs to generate sets of virtual neurons. Parameters statistics for all algorithms were measured from experimental data, thus providing a compact and consistent description of these morphological classes. We compared the emergent anatomical features of each group of virtual neurons with those of the experimental database in order to gain insights on the plausibility of the model assumptions, potential improvements to the algorithms, and non-trivial relations among morphological parameters. Algorithms mainly based on local constraints (e.g., branch diameter) were successful in reproducing many morphological properties of both motoneurons and Purkinje cells (e.g. total length, asymmetry, number of bifurcations). The addition of global constraints (e.g., trophic factors) improved the angle-dependent emergent characteristics (average Euclidean distance from the soma to the dendritic terminations, dendritic spread). Virtual neurons systematically displayed greater anatomical variability than real cells, suggesting the need for additional constraints in the models. For several emergent anatomical properties, a specific algorithm reproduced the experimental statistics better than the others did. However, relative performances were often reversed for different anatomical properties and/or morphological classes. Thus, combining the strengths of alternative generative models could lead to comprehensive algorithms for the complete and accurate simulation of dendritic morphology.

The Prony fitting method for digital deconvolution problem

The Prony fitting theory is applied in this paper to solve the deconvolution problem. There are two cases in deconvolution in which unstable solution is easy to appear. They are: (1)the frequency band of known kernel is more narraw than that of the unknown kernel; (2) there exists noise. These two cases are studied thoroughly and the effectiveness of Prony fitting method is showed. Finally, this method is simulated in computer. The simulation results are compared with those obtained by using FFT method directly.

Generalised Dirichelt-to-Neumann map in time dependent domains

We study the heat, linear Schrodinger and linear KdV equations in the domain l(t) < x < ∞, 0 < t < T, with prescribed initial and boundary conditions and with l(t) a given differentiable function. For the first two equations, we show that the unknown Neumann or Dirichlet boundary value can be computed as the solution of a linear Volterra integral equation with an explicit weakly singular kernel. This integral equation can be derived from the formal Fourier integral representation of the solution. For the linear KdV equation we show that the two unknown boundary values can be computed as the solution of a system of linear Volterra integral equations with explicit weakly singular kernels. The derivation in this case makes crucial use of analyticity and certain invariance properties in the complex spectral plane. The above Volterra equations are shown to admit a unique solution.

Using zero-norm constraint for sparse probability density function estimation

A new sparse kernel probability density function (pdf) estimator based on zero-norm constraint is constructed using the classical Parzen window (PW) estimate as the target function. The so-called zero-norm of the parameters is used in order to achieve enhanced model sparsity, and it is suggested to minimize an approximate function of the zero-norm. It is shown that under certain condition, the kernel weights of the proposed pdf estimator based on the zero-norm approximation can be updated using the multiplicative nonnegative quadratic programming algorithm. Numerical examples are employed to demonstrate the efficacy of the proposed approach.

Assimilation impacts on Arctic Ocean circulation, heat and freshwater

We investigate the Arctic basin circulation, freshwater content (FWC) and heat budget by using a high-resolution global coupled ice–ocean model implemented with a state-of-the-art data assimilation scheme. We demonstrate that, despite a very sparse dataset, by assimilating hydrographic data in and near the Arctic basin, the initial warm bias and drift in the control run is successfully corrected, reproducing a much more realistic vertical and horizontal structure to the cyclonic boundary current carrying the Atlantic Water (AW) along the Siberian shelves in the reanalysis run. The Beaufort Gyre structure and FWC and variability are also more accurately reproduced. Small but important changes in the strait exchange flows are found which lead to more balanced budgets in the reanalysis run. Assimilation fluxes dominate the basin budgets over the first 10 years (P1: 1987–1996) of the reanalysis for both heat and FWC, after which the drifting Arctic upper water properties have been restored to realistic values. For the later period (P2: 1997–2004), the Arctic heat budget is almost balanced without assimilation contributions, while the freshwater budget shows reduced assimilation contributions compensating largely for surface salinity damping, which was extremely strong in this run. A downward trend in freshwater export at the Canadian Straits and Fram Strait is found in period P2, associated with Beaufort Gyre recharge. A detailed comparison with observations and previous model studies at the individual Arctic straits is also included.

Contested H2O: science, policy and politics in water resources management in Chile

This paper critically explores the politics that mediate the use of environmental science assessments as the basis of resource management policy. Drawing on recent literature in the political ecology tradition that has emphasised the politicised nature of the production and use of scientific knowledge in environmental management, the paper analyses a hydrological assessment in a small river basin in Chile, undertaken in response to concerns over the possible overexploitation of groundwater resources. The case study illustrates the limitations of an approach based predominantly on hydrogeological modelling to ascertain the effects of increased groundwater abstraction. In particular, it identifies the subjective ways in which the assessment was interpreted and used by the state water resources agency to underpin water allocation decisions in accordance with its own interests, and the role that a desocialised assessment played in reproducing unequal patterns of resource use and configuring uneven waterscapes. Nevertheless, as Chile’s ‘neoliberal’ political-economic framework privileges the role of science and technocracy, producing other forms of environmental knowledge to complement environmental science is likely to be contentious. In conclusion, the paper considers the potential of mobilising the concept of the hydrosocial cycle to further critically engage with environmental science.

A process for analysis of microarray comparative genomics hybridisation studies for bacterial genomes

Background: Microarray based comparative genomic hybridisation (CGH) experiments have been used to study numerous biological problems including understanding genome plasticity in pathogenic bacteria. Typically such experiments produce large data sets that are difficult for biologists to handle. Although there are some programmes available for interpretation of bacterial transcriptomics data and CGH microarray data for looking at genetic stability in oncogenes, there are none specifically to understand the mosaic nature of bacterial genomes. Consequently a bottle neck still persists in accurate processing and mathematical analysis of these data. To address this shortfall we have produced a simple and robust CGH microarray data analysis process that may be automated in the future to understand bacterial genomic diversity. Results: The process involves five steps: cleaning, normalisation, estimating gene presence and absence or divergence, validation, and analysis of data from test against three reference strains simultaneously. Each stage of the process is described and we have compared a number of methods available for characterising bacterial genomic diversity, for calculating the cut-off between gene presence and absence or divergence, and shown that a simple dynamic approach using a kernel density estimator performed better than both established, as well as a more sophisticated mixture modelling technique. We have also shown that current methods commonly used for CGH microarray analysis in tumour and cancer cell lines are not appropriate for analysing our data. Conclusion: After carrying out the analysis and validation for three sequenced Escherichia coli strains, CGH microarray data from 19 E. coli O157 pathogenic test strains were used to demonstrate the benefits of applying this simple and robust process to CGH microarray studies using bacterial genomes.

A variational method to retrieve the extinction profile in liquid clouds using multiple field-of-view lidar

Liquid clouds play a profound role in the global radiation budget but it is difficult to remotely retrieve their vertical profile. Ordinary narrow field-of-view (FOV) lidars receive a strong return from such clouds but the information is limited to the first few optical depths. Wideangle multiple-FOV lidars can isolate radiation scattered multiple times before returning to the instrument, often penetrating much deeper into the cloud than the singly-scattered signal. These returns potentially contain information on the vertical profile of extinction coefficient, but are challenging to interpret due to the lack of a fast radiative transfer model for simulating them. This paper describes a variational algorithm that incorporates a fast forward model based on the time-dependent two-stream approximation, and its adjoint. Application of the algorithm to simulated data from a hypothetical airborne three-FOV lidar with a maximum footprint width of 600m suggests that this approach should be able to retrieve the extinction structure down to an optical depth of around 6, and total opticaldepth up to at least 35, depending on the maximum lidar FOV. The convergence behavior of Gauss-Newton and quasi-Newton optimization schemes are compared. We then present results from an application of the algorithm to observations of stratocumulus by the 8-FOV airborne “THOR” lidar. It is demonstrated how the averaging kernel can be used to diagnose the effective vertical resolution of the retrieved profile, and therefore the depth to which information on the vertical structure can be recovered. This work enables exploitation of returns from spaceborne lidar and radar subject to multiple scattering more rigorously than previously possible.

Validation of three new measure-correlate-predict models for the long-term prospection of the wind resource

The estimation of the long-term wind resource at a prospective site based on a relatively short on-site measurement campaign is an indispensable task in the development of a commercial wind farm. The typical industry approach is based on the measure-correlate-predict �MCP� method where a relational model between the site wind velocity data and the data obtained from a suitable reference site is built from concurrent records. In a subsequent step, a long-term prediction for the prospective site is obtained from a combination of the relational model and the historic reference data. In the present paper, a systematic study is presented where three new MCP models, together with two published reference models �a simple linear regression and the variance ratio method�, have been evaluated based on concurrent synthetic wind speed time series for two sites, simulating the prospective and the reference site. The synthetic method has the advantage of generating time series with the desired statistical properties, including Weibull scale and shape factors, required to evaluate the five methods under all plausible conditions. In this work, first a systematic discussion of the statistical fundamentals behind MCP methods is provided and three new models, one based on a nonlinear regression and two �termed kernel methods� derived from the use of conditional probability density functions, are proposed. All models are evaluated by using five metrics under a wide range of values of the correlation coefficient, the Weibull scale, and the Weibull shape factor. Only one of all models, a kernel method based on bivariate Weibull probability functions, is capable of accurately predicting all performance metrics studied.

Initialisation and predictability of the AMOC over the last 50 years in a climate model

The mechanisms involved in Atlantic meridional overturning circulation (AMOC) decadal variability and predictability over the last 50 years are analysed in the IPSL–CM5A–LR model using historical and initialised simulations. The initialisation procedure only uses nudging towards sea surface temperature anomalies with a physically based restoring coefficient. When compared to two independent AMOC reconstructions, both the historical and nudged ensemble simulations exhibit skill at reproducing AMOC variations from 1977 onwards, and in particular two maxima occurring respectively around 1978 and 1997. We argue that one source of skill is related to the large Mount Agung volcanic eruption starting in 1963, which reset an internal 20-year variability cycle in the North Atlantic in the model. This cycle involves the East Greenland Current intensity, and advection of active tracers along the subpolar gyre, which leads to an AMOC maximum around 15 years after the Mount Agung eruption. The 1997 maximum occurs approximately 20 years after the former one. The nudged simulations better reproduce this second maximum than the historical simulations. This is due to the initialisation of a cooling of the convection sites in the 1980s under the effect of a persistent North Atlantic oscillation (NAO) positive phase, a feature not captured in the historical simulations. Hence we argue that the 20-year cycle excited by the 1963 Mount Agung eruption together with the NAO forcing both contributed to the 1990s AMOC maximum. These results support the existence of a 20-year cycle in the North Atlantic in the observations. Hindcasts following the CMIP5 protocol are launched from a nudged simulation every 5 years for the 1960–2005 period. They exhibit significant correlation skill score as compared to an independent reconstruction of the AMOC from 4-year lead-time average. This encouraging result is accompanied by increased correlation skills in reproducing the observed 2-m air temperature in the bordering regions of the North Atlantic as compared to non-initialized simulations. To a lesser extent, predicted precipitation tends to correlate with the nudged simulation in the tropical Atlantic. We argue that this skill is due to the initialisation and predictability of the AMOC in the present prediction system. The mechanisms evidenced here support the idea of volcanic eruptions as a pacemaker for internal variability of the AMOC. Together with the existence of a 20-year cycle in the North Atlantic they propose a novel and complementary explanation for the AMOC variations over the last 50 years.

NeMo: a platform for neural modelling of spiking neurons using GPUs

Simulating spiking neural networks is of great interest to scientists wanting to model the functioning of the brain. However, large-scale models are expensive to simulate due to the number and interconnectedness of neurons in the brain. Furthermore, where such simulations are used in an embodied setting, the simulation must be real-time in order to be useful. In this paper we present NeMo, a platform for such simulations which achieves high performance through the use of highly parallel commodity hardware in the form of graphics processing units (GPUs). NeMo makes use of the Izhikevich neuron model which provides a range of realistic spiking dynamics while being computationally efficient. Our GPU kernel can deliver up to 400 million spikes per second. This corresponds to a real-time simulation of around 40 000 neurons under biologically plausible conditions with 1000 synapses per neuron and a mean firing rate of 10 Hz.