Exam 1999

Exam and solutions in LaTex

M-estimator, and D-optimality model construction using orthogonal forward regression

This correspondence introduces a new orthogonal forward regression (OFR) model identification algorithm using D-optimality for model structure selection and is based on an M-estimators of parameter estimates. M-estimator is a classical robust parameter estimation technique to tackle bad data conditions such as outliers. Computationally, The M-estimator can be derived using an iterative reweighted least squares (IRLS) algorithm. D-optimality is a model structure robustness criterion in experimental design to tackle ill-conditioning in model Structure. The orthogonal forward regression (OFR), often based on the modified Gram-Schmidt procedure, is an efficient method incorporating structure selection and parameter estimation simultaneously. The basic idea of the proposed approach is to incorporate an IRLS inner loop into the modified Gram-Schmidt procedure. In this manner, the OFR algorithm for parsimonious model structure determination is extended to bad data conditions with improved performance via the derivation of parameter M-estimators with inherent robustness to outliers. Numerical examples are included to demonstrate the effectiveness of the proposed algorithm.

Experimental design and model construction algorithms for radial basis function networks

New construction algorithms for radial basis function (RBF) network modelling are introduced based on the A-optimality and D-optimality experimental design criteria respectively. We utilize new cost functions, based on experimental design criteria, for model selection that simultaneously optimizes model approximation, parameter variance (A-optimality) or model robustness (D-optimality). The proposed approaches are based on the forward orthogonal least-squares (OLS) algorithm, such that the new A-optimality- and D-optimality-based cost functions are constructed on the basis of an orthogonalization process that gains computational advantages and hence maintains the inherent computational efficiency associated with the conventional forward OLS approach. The proposed approach enhances the very popular forward OLS-algorithm-based RBF model construction method since the resultant RBF models are constructed in a manner that the system dynamics approximation capability, model adequacy and robustness are optimized simultaneously. The numerical examples provided show significant improvement based on the D-optimality design criterion, demonstrating that there is significant room for improvement in modelling via the popular RBF neural network.

Parameter estimation based on stacked regression and evolutionary algorithms

A new parameter-estimation algorithm, which minimises the cross-validated prediction error for linear-in-the-parameter models, is proposed, based on stacked regression and an evolutionary algorithm. It is initially shown that cross-validation is very important for prediction in linear-in-the-parameter models using a criterion called the mean dispersion error (MDE). Stacked regression, which can be regarded as a sophisticated type of cross-validation, is then introduced based on an evolutionary algorithm, to produce a new parameter-estimation algorithm, which preserves the parsimony of a concise model structure that is determined using the forward orthogonal least-squares (OLS) algorithm. The PRESS prediction errors are used for cross-validation, and the sunspot and Canadian lynx time series are used to demonstrate the new algorithms.

Adaptive recursive control of a soil testing load frame

Recursive Learning Control (RLC) has the potential to significantly reduce the tracking error in many repetitive trajectory applications. This paper presents an application of RLC to a soil testing load frame where non-adaptive techniques struggle with the highly nonlinear nature of soil. The main purpose of the controller is to apply a sinusoidal force reference trajectory on a soil sample with a high degree of accuracy and repeatability. The controller uses a feedforward control structure, recursive least squares adaptation algorithm and RLC to compensate for periodic errors. Tracking error is reduced and stability is maintained across various soil sample responses.

A neurofuzzy classifier for two class problems

A neurofuzzy classifier identification algorithm is introduced for two class problems. The initial fuzzy base construction is based on fuzzy clustering utilizing a Gaussian mixture model (GMM) and the analysis of covariance (ANOVA) decomposition. The expectation maximization (EM) algorithm is applied to determine the parameters of the fuzzy membership functions. Then neurofuzzy model is identified via the supervised subspace orthogonal least square (OLS) algorithm. Finally a logistic regression model is applied to produce the class probability. The effectiveness of the proposed neurofuzzy classifier has been demonstrated using a real data set.

Adaptive noise cancellation with fast tunable RBF network

This paper describes a novel adaptive noise cancellation system with fast tunable radial basis function (RBF). The weight coefficients of the RBF network are adapted by the multi-innovation recursive least square (MRLS) algorithm. If the RBF network performs poorly despite of the weight adaptation, an insignificant node with little contribution to the overall performance is replaced with a new node without changing the model size. Otherwise, the RBF network structure remains unchanged and only the weight vector is adapted. The simulation results show that the proposed approach can well cancel the noise in both stationary and nonstationary ANC systems.

Online modeling with tunable RBF network

In this paper, we propose a novel online modeling algorithm for nonlinear and nonstationary systems using a radial basis function (RBF) neural network with a fixed number of hidden nodes. Each of the RBF basis functions has a tunable center vector and an adjustable diagonal covariance matrix. A multi-innovation recursive least square (MRLS) algorithm is applied to update the weights of RBF online, while the modeling performance is monitored. When the modeling residual of the RBF network becomes large in spite of the weight adaptation, a node identified as insignificant is replaced with a new node, for which the tunable center vector and diagonal covariance matrix are optimized using the quantum particle swarm optimization (QPSO) algorithm. The major contribution is to combine the MRLS weight adaptation and QPSO node structure optimization in an innovative way so that it can track well the local characteristic in the nonstationary system with a very sparse model. Simulation results show that the proposed algorithm has significantly better performance than existing approaches.

Nonlinear identification using orthogonal forward regression with nested optimal regularization

An efficient data based-modeling algorithm for nonlinear system identification is introduced for radial basis function (RBF) neural networks with the aim of maximizing generalization capability based on the concept of leave-one-out (LOO) cross validation. Each of the RBF kernels has its own kernel width parameter and the basic idea is to optimize the multiple pairs of regularization parameters and kernel widths, each of which is associated with a kernel, one at a time within the orthogonal forward regression (OFR) procedure. Thus, each OFR step consists of one model term selection based on the LOO mean square error (LOOMSE), followed by the optimization of the associated kernel width and regularization parameter, also based on the LOOMSE. Since like our previous state-of-the-art local regularization assisted orthogonal least squares (LROLS) algorithm, the same LOOMSE is adopted for model selection, our proposed new OFR algorithm is also capable of producing a very sparse RBF model with excellent generalization performance. Unlike our previous LROLS algorithm which requires an additional iterative loop to optimize the regularization parameters as well as an additional procedure to optimize the kernel width, the proposed new OFR algorithm optimizes both the kernel widths and regularization parameters within the single OFR procedure, and consequently the required computational complexity is dramatically reduced. Nonlinear system identification examples are included to demonstrate the effectiveness of this new approach in comparison to the well-known approaches of support vector machine and least absolute shrinkage and selection operator as well as the LROLS algorithm.

A new adaptive multiple modelling approach for non-linear and non-stationary systems

This paper proposes a novel adaptive multiple modelling algorithm for non-linear and non-stationary systems. This simple modelling paradigm comprises K candidate sub-models which are all linear. With data available in an online fashion, the performance of all candidate sub-models are monitored based on the most recent data window, and M best sub-models are selected from the K candidates. The weight coefficients of the selected sub-model are adapted via the recursive least square (RLS) algorithm, while the coefficients of the remaining sub-models are unchanged. These M model predictions are then optimally combined to produce the multi-model output. We propose to minimise the mean square error based on a recent data window, and apply the sum to one constraint to the combination parameters, leading to a closed-form solution, so that maximal computational efficiency can be achieved. In addition, at each time step, the model prediction is chosen from either the resultant multiple model or the best sub-model, whichever is the best. Simulation results are given in comparison with some typical alternatives, including the linear RLS algorithm and a number of online non-linear approaches, in terms of modelling performance and time consumption.

Improving the performance of equalization in communication systems

In this paper, research on exploring the potential of several popular equalization techniques while overcoming their disadvantages has been conducted. First, extensive literature survey on equalization is conducted. The focus has been placed on several popular linear equalization algorithm such as the conventional least-mean-square (LMS) algorithm, the recursive least squares (RLS) algorithm, the fi1tered-X LMS algorithm and their development. The approach in analysing the performance of the filtered-X LMS Algorithm, a heuristic method based on linear time-invariant operator theory is provided to analyse the robust perfonnance of the filtered-X structure. It indicates that the extra filter could enhance the stability margin of the corresponding non filtered X structure. To overcome the slow convergence problem while keeping the simplicity of the LMS based algorithms, an H2 optimal initialization is proposed.

Terrestrial avifauna of the Gippsland Plain and Strzlecki Ranges, Victoria, Australia: insights from Atlas data.

The rate and spatial scale at which natural environments are being modified by human land-uses mean that a regional or national perspective is necessary to understand the status of the native biota. Here, we outline a landscape-based approach for using data from the ‘New Atlas of Australian Birds’ to examine the distribution and status of avifauna at a regional scale. We use data from two bioregions in south-east Australia – the Gippsland Plain and the Strzelecki Ranges (collectively termed the greater Gippsland Plains) – to demonstrate this approach. Records were compiled for 57 landscape units, each 10′ latitude by 10′ longitude (~270 km2) across the study region. A total of 165 terrestrial bird species was recorded from 1870 ‘area searches’, with a further 24 species added from incidental observations and other surveys. Of these, 108 species were considered ‘typical’ of the greater Gippsland Plain in that they currently or historically occur regularly in the study region. An index of species ‘occurrence’, combining reporting rate and breadth of distribution, was used to identify rare, common, widespread and restricted species. Ordination of the dataset highlighted assemblages of birds that had similar spatial distributions. A complementarity analysis identified a subset of 14 landscape units that together contained records from at least three different landscape units for each of the 108 ‘typical’ species. When compared with the 40 most common ‘typical’ species, the 40 least common species were more likely to be forest specialists, nest on the ground and, owing to the prevalence of raptors in the least common group, take prey on the wing. The future status of the terrestrial avifauna of the greater Gippsland Plains will depend on the extent to which effective restoration actions can be undertaken to ensure adequate representation of habitats for all species, especially for the large number of species of conservation concern.

Improving the performance of the LMS and RLS algorithms for adaptive equalizer

In this paper, we present the experiment results of three adaptive equalization algorithms: least-mean-square (LMS) algorithm, discrete cosine transform-least mean square (DCT-LMS) algorithm, and recursive least square (RLS) algorithm. Based on the experiments, we obtained that the convergence rate of LMS is slow; the convergence rate of RLS is great faster while the computational price is expensive; the performance of that two parameters of DCT-LMS are between the previous two algorithms, but still not good enough. Therefore we will propose an algorithm based on H2 in a coming paper to solve the problems.

Application of fast and robust equalization in communication technology

In this paper, the authors explore the potential of several popular equalization techniques while overcoming their disadvantages. First, extensive literature survey on equalization is conducted. The focus is on popular linear equalization algorithms such as the conventional least-mean-square (LMS) algorithm , The recursive least-squares (RLS) algorithm, the filtered-X LMS algorithm and their development. To overcome the slow convergence problem while keeping the simplicity of the LMS based algorithms, an H2 optimal initialization is proposed.

Position estimation and tracking of an autonomous mobile sensor using received signal strength

In this paper, an algorithm for approximating the path of a moving autonomous mobile sensor with an unknown position location using Received Signal Strength (RSS) measurements is proposed. Using a Least Squares (LS) estimation method as an input, a Maximum-Likelihood (ML) approach is used to determine the location of the unknown mobile sensor. For the mobile sensor case, as the sensor changes position the characteristics of the RSS measurements also change; therefore the proposed method adapts the RSS measurement model by dynamically changing the pass loss value alpha to aid in position estimation. Secondly, a Recursive Least-Squares (RLS) algorithm is used to estimate the path of a moving mobile sensor using the Maximum-Likelihood position estimation as an input. The performance of the proposed algorithm is evaluated via simulation and it is shown that this method can accurately determine the position of the mobile sensor, and can efficiently track the position of the mobile sensor during motion.