Data based constructive identification - overcoming the curse of dimensionality

The modelling of a nonlinear stochastic dynamical processes from data involves solving the problems of data gathering, preprocessing, model architecture selection, learning or adaptation, parametric evaluation and model validation. For a given model architecture such as associative memory networks, a common problem in non-linear modelling is the problem of "the curse of dimensionality". A series of complementary data based constructive identification schemes, mainly based on but not limited to an operating point dependent fuzzy models, are introduced in this paper with the aim to overcome the curse of dimensionality. These include (i) a mixture of experts algorithm based on a forward constrained regression algorithm; (ii) an inherent parsimonious delaunay input space partition based piecewise local lineal modelling concept; (iii) a neurofuzzy model constructive approach based on forward orthogonal least squares and optimal experimental design and finally (iv) the neurofuzzy model construction algorithm based on basis functions that are Bézier Bernstein polynomial functions and the additive decomposition. Illustrative examples demonstrate their applicability, showing that the final major hurdle in data based modelling has almost been removed.

Computer intensive methods in control and signal processing: the curse of dimensionality

This volume is based upon the 2nd IEEE European Workshop on Computer-Intensive Methods in Control and Signal Processing, held in Prague, August 1996.

Quantile regression without the curse of unsmoothness

We consider quantile regression models and investigate the induced smoothing method for obtaining the covariance matrix of the regression parameter estimates. We show that the difference between the smoothed and unsmoothed estimating functions in quantile regression is negligible. The detailed and simple computational algorithms for calculating the asymptotic covariance are provided. Intensive simulation studies indicate that the proposed method performs very well. We also illustrate the algorithm by analyzing the rainfall–runoff data from Murray Upland, Australia.

A Feature-Based Solution to Forward Problem in Electrical Capacitance Tomography of Conductive Materials

A new feature-based technique is introduced to solve the nonlinear forward problem (FP) of the electrical capacitance tomography with the target application of monitoring the metal fill profile in the lost foam casting process. The new technique is based on combining a linear solution to the FP and a correction factor (CF). The CF is estimated using an artificial neural network (ANN) trained using key features extracted from the metal distribution. The CF adjusts the linear solution of the FP to account for the nonlinear effects caused by the shielding effects of the metal. This approach shows promising results and avoids the curse of dimensionality through the use of features and not the actual metal distribution to train the ANN. The ANN is trained using nine features extracted from the metal distributions as input. The expected sensors readings are generated using ANSYS software. The performance of the ANN for the training and testing data was satisfactory, with an average root-mean-square error equal to 2.2%.

A new landscape of arts-based buildings and comparative cultural policies on the island of Ireland: the curse of Jocasta's necklace

Much current cultural policy research focuses on activity traditionally viewed as arts practice: visual arts, music, literature and dance. Architecture’s role in the discussion of cultural policy is, however, less certain and thus less frequently interrogated. The study presented here both addresses this dearth of in-depth research while also contributing to the interdisciplinary discussion of cultural policy in wider terms. In seeking to better understand how architectural culture is regulated and administered in a specific case study, it unpacks how the complicated relationships of nominal and explicit policies on both sides of the Irish/Northern Irish border contributed to the significant expansion of arts-based buildings 1995-2008. It contrasts political and cultural motivations behind these projects during a period of significant economic growth, investment and inward immigration. Data has been gathered from both official published policies as well as interviews with elite actors in the decision-making field and architects who produced the buildings of interest in both countries. With the sizeable number of arts-based buildings now completed in both Republic of Ireland and Northern Ireland, one must wonder if this necklace of buildings is, like Jocasta’s, a thing of both beauty and redolent with a potential future curse. It is the goal of this project to contribute to the larger applied and critical discussion of these issues and to engage with future policy design, administration and, certainly, evaluation.

A new landscape of arts-based buildings and comparative cultural policies on the island of Ireland: the curse of Jocasta's necklace

Much current cultural policy research focuses on activity traditionally viewed as arts practice: visual arts, music, literature and dance. Architecture’s role in the discussion of cultural policy is, however, less certain and thus less frequently interrogated. The study presented here both addresses this dearth of in-depth research while also contributing to the interdisciplinary discussion of cultural policy in wider terms. In seeking to better understand how architectural culture is regulated and administered in a specific case study, it unpacks how the complicated relationships of nominal and explicit policies on both sides of the Irish/Northern Irish border contributed to the significant expansion of arts-based buildings 1995-2008. It contrasts political and cultural motivations behind these projects during a period of significant economic growth, investment and inward immigration. Data has been gathered from both official published policies as well as interviews with elite actors in the decision-making field and architects who produced the buildings of interest in both countries. With the sizeable number of arts-based buildings now completed in both Republic of Ireland and Northern Ireland, one must wonder if this necklace of buildings is, like Jocasta’s, a thing of both beauty and redolent with a potential future curse. It is the goal of this project to contribute to the larger applied and critical discussion of these issues and to engage with future policy design, administration and, certainly, evaluation.

Urban Detection From Hyperspectral Images Using Dimension-Reduction Model and Fusion of Multiple Segmentations Based on Stuctural and Textural Features

Ce mémoire de maîtrise présente une nouvelle approche non supervisée pour détecter et segmenter les régions urbaines dans les images hyperspectrales. La méthode proposée n ́ecessite trois étapes. Tout d’abord, afin de réduire le coût calculatoire de notre algorithme, une image couleur du contenu spectral est estimée. A cette fin, une étape de réduction de dimensionalité non-linéaire, basée sur deux critères complémentaires mais contradictoires de bonne visualisation; à savoir la précision et le contraste, est réalisée pour l’affichage couleur de chaque image hyperspectrale. Ensuite, pour discriminer les régions urbaines des régions non urbaines, la seconde étape consiste à extraire quelques caractéristiques discriminantes (et complémentaires) sur cette image hyperspectrale couleur. A cette fin, nous avons extrait une série de paramètres discriminants pour décrire les caractéristiques d’une zone urbaine, principalement composée d’objets manufacturés de formes simples g ́eométriques et régulières. Nous avons utilisé des caractéristiques texturales basées sur les niveaux de gris, la magnitude du gradient ou des paramètres issus de la matrice de co-occurrence combinés avec des caractéristiques structurelles basées sur l’orientation locale du gradient de l’image et la détection locale de segments de droites. Afin de réduire encore la complexité de calcul de notre approche et éviter le problème de la ”malédiction de la dimensionnalité” quand on décide de regrouper des données de dimensions élevées, nous avons décidé de classifier individuellement, dans la dernière étape, chaque caractéristique texturale ou structurelle avec une simple procédure de K-moyennes et ensuite de combiner ces segmentations grossières, obtenues à faible coût, avec un modèle efficace de fusion de cartes de segmentations. Les expérimentations données dans ce rapport montrent que cette stratégie est efficace visuellement et se compare favorablement aux autres méthodes de détection et segmentation de zones urbaines à partir d’images hyperspectrales.

Using Recurrent Networks for Dimensionality Reduction

This report explores how recurrent neural networks can be exploited for learning high-dimensional mappings. Since recurrent networks are as powerful as Turing machines, an interesting question is how recurrent networks can be used to simplify the problem of learning from examples. The main problem with learning high-dimensional functions is the curse of dimensionality which roughly states that the number of examples needed to learn a function increases exponentially with input dimension. This thesis proposes a way of avoiding this problem by using a recurrent network to decompose a high-dimensional function into many lower dimensional functions connected in a feedback loop.

Convergence Rates of Approximation by Translates

In this paper we consider the problem of approximating a function belonging to some funtion space Φ by a linear comination of n translates of a given function G. Ussing a lemma by Jones (1990) and Barron (1991) we show that it is possible to define function spaces and functions G for which the rate of convergence to zero of the erro is 0(1/n) in any number of dimensions. The apparent avoidance of the "curse of dimensionality" is due to the fact that these function spaces are more and more constrained as the dimension increases. Examples include spaces of the Sobolev tpe, in which the number of weak derivatives is required to be larger than the number of dimensions. We give results both for approximation in the L2 norm and in the Lc norm. The interesting feature of these results is that, thanks to the constructive nature of Jones" and Barron"s lemma, an iterative procedure is defined that can achieve this rate.

A sovereign wealth fund to lift Germany's curse of excess savings. CEPS Policy Brief No. 280, 28 August 2012

As an alternative to the present system of intermediation of the German savings surplus, this paper suggests that the risk-adjusted rate of return could be improved by creating a sovereign wealth fund for Germany (designated DESWF), which could invest excess German savings globally. Such a DESWF would offer German savers a secure vehicle paying a guaranteed positive minimum real interest rate, with a top-up when real investment returns allowed. The vehicle would invest the funds in a portfolio that is highly diversified by geography and asset classes. Positive real returns can be expected in the long run based on positive real global growth. Since, in this case, a significant amount of funds would flow outside the euro area, the euro would depreciate, which would help crisis countries presently struggling to revive growth through exports and to close their external deficits so as to recoup their international credit-worthiness. Target imbalances would gradually disappear and German claims abroad would move from nominal claims on the ECB to diversified real and nominal claims on various private and public foreign entities in a variety of asset classes.

Neurofuzzy mixture of experts network parallel learning and model construction algorithms

A connection between a fuzzy neural network model with the mixture of experts network (MEN) modelling approach is established. Based on this linkage, two new neuro-fuzzy MEN construction algorithms are proposed to overcome the curse of dimensionality that is inherent in the majority of associative memory networks and/or other rule based systems. The first construction algorithm employs a function selection manager module in an MEN system. The second construction algorithm is based on a new parallel learning algorithm in which each model rule is trained independently, for which the parameter convergence property of the new learning method is established. As with the first approach, an expert selection criterion is utilised in this algorithm. These two construction methods are equivalent in their effectiveness in overcoming the curse of dimensionality by reducing the dimensionality of the regression vector, but the latter has the additional computational advantage of parallel processing. The proposed algorithms are analysed for effectiveness followed by numerical examples to illustrate their efficacy for some difficult data based modelling problems.

Neurofuzzy design and model construction of nonlinear dynamical processes from data

A common problem in many data based modelling algorithms such as associative memory networks is the problem of the curse of dimensionality. In this paper, a new two-stage neurofuzzy system design and construction algorithm (NeuDeC) for nonlinear dynamical processes is introduced to effectively tackle this problem. A new simple preprocessing method is initially derived and applied to reduce the rule base, followed by a fine model detection process based on the reduced rule set by using forward orthogonal least squares model structure detection. In both stages, new A-optimality experimental design-based criteria we used. In the preprocessing stage, a lower bound of the A-optimality design criterion is derived and applied as a subset selection metric, but in the later stage, the A-optimality design criterion is incorporated into a new composite cost function that minimises model prediction error as well as penalises the model parameter variance. The utilisation of NeuDeC leads to unbiased model parameters with low parameter variance and the additional benefit of a parsimonious model structure. Numerical examples are included to demonstrate the effectiveness of this new modelling approach for high dimensional inputs.

Variable selection algorithm for the construction of MIMO operating point dependent neurofuzzy networks

An input variable selection procedure is introduced for the identification and construction of multi-input multi-output (MIMO) neurofuzzy operating point dependent models. The algorithm is an extension of a forward modified Gram-Schmidt orthogonal least squares procedure for a linear model structure which is modified to accommodate nonlinear system modeling by incorporating piecewise locally linear model fitting. The proposed input nodes selection procedure effectively tackles the problem of the curse of dimensionality associated with lattice-based modeling algorithms such as radial basis function neurofuzzy networks, enabling the resulting neurofuzzy operating point dependent model to be widely applied in control and estimation. Some numerical examples are given to demonstrate the effectiveness of the proposed construction algorithm.