Backward elimination model construction for regression and classification using leave-one-out criteria

A fundamental principle in practical nonlinear data modeling is the parsimonious principle of constructing the minimal model that explains the training data well. Leave-one-out (LOO) cross validation is often used to estimate generalization errors by choosing amongst different network architectures (M. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974). Based upon the minimization of LOO criteria of either the mean squares of LOO errors or the LOO misclassification rate respectively, we present two backward elimination algorithms as model post-processing procedures for regression and classification problems. The proposed backward elimination procedures exploit an orthogonalization procedure to enable the orthogonality between the subspace as spanned by the pruned model and the deleted regressor. Subsequently, it is shown that the LOO criteria used in both algorithms can be calculated via some analytic recursive formula, as derived in this contribution, without actually splitting the estimation data set so as to reduce computational expense. Compared to most other model construction methods, the proposed algorithms are advantageous in several aspects; (i) There are no tuning parameters to be optimized through an extra validation data set; (ii) The procedure is fully automatic without an additional stopping criteria; and (iii) The model structure selection is directly based on model generalization performance. The illustrative examples on regression and classification are used to demonstrate that the proposed algorithms are viable post-processing methods to prune a model to gain extra sparsity and improved generalization.

Chronic hypoxia in the human neuroblastoma SH-SY5Y causes reduced expression of the putative alpha-secretases, ADAM10 and TACE, without altering their mRNA levels

Alzheimer's disease is more frequent following an ischemic or hypoxic episode, with levels of beta-amyloid peptides elevated in brains from patients. Similar increases are found after experimental ischemia in animals. It is possible that increased beta-amyloid deposition arises from alterations in amyloid precursor protein (APP) metabolism, indeed, we have shown that exposing cells of neuronal origin to chronic hypoxia decreased the secretion of soluble APP (sAPPalpha) derived by action of alpha-secretase on APP, coinciding with a decrease in protein levels of ADAM10, a disintegrin metalloprotease which is thought to be the major alpha-secretase. In the current study, we extended those observations to determine whether the expression of ADAM10 and another putative alpha-secretase, TACE, as well as the beta-secretase, BACE1 were regulated by chronic hypoxia at the level of protein and mRNA. Using Western blotting and RT-PCR, we demonstrate that after 48 h chronic hypoxia, such that sAPPalpha secretion is decreased by over 50%, protein levels of ADAM10 and TACE and by approximately 60% and 40% respectively with no significant decrease in BACE1 levels. In contrast, no change in the expression of the mRNA for these proteins could be detected. Thus, we conclude that under CH the level of the putative alpha-secretases, ADAM10 and TACE are regulated by post-translational mechanisms, most probably proteolysis, rather than at the level of transcription.

Diacylglycerol-induced protection against injury during ischemia and reperfusion can be achieved without activation of protein kinase C in the rat heart: Comparative studies with ischemic preconditioning

The role of protein kinase C (PKC) activation in ischemic preconditioning remains controversial. Since diacylglycerol is the endogenous activator of PKC and as such might be expected cardioprotective, we have investigated whether: (i) the diacylglycerol analog 1,2-dioctanoyl-sn-glycerol (DOG) can protect against injury during ischemia and reperfusion; (ii) any effect is mediated via PKC activation; and (iii) the outcome is influenced by the time of administration. Isolated rat hearts were perfused with buffer at 37°C and paced at 400 bpm. In Study 1, hearts (n=6/group) were subjected to one of the following: (1) 36 min aerobic perfusion (controls); (2) 20 min aerobic perfusion plus ischemic preconditioning (3 min ischemia/3 min reperfusion+5 min ischemia/5 min reperfusion); (3) aerobic perfusion with buffer containing DOG (10 μM) given as a substitute for ischemic preconditioning; (4) aerobic perfusion with DOG (10 μM) during the last 2 min of aerobic perfusion. All hearts then were subjected to 35 min of global ischemia and 40 min reperfusion. A further group (5) were perfused with DOG (10 μM) for the first 2 min of reperfusion. Ischemic preconditioning improved postischemic recovery of LVDP from 24±3% in controls to 71±2% (P<0.05). Recovery of LVDP also was enhanced by DOG when given just before ischemia (54±4%), however, DOG had no effect on the recovery of LVDP when used as a substitute for ischemic preconditioning (22±5%) or when given during reperfusion (29±6%). In Study 2, the first four groups of study were repeated (n=4–5/group) without imposing the periods of ischemia and reperfusion, instead hearts were taken for the measurement of PKC activity (pmol/min/mg protein±SEM). PKC activity after 36 min in groups (1), (2), (3) and (4) was: 332±102, 299±63, 521±144, and 340±113 and the membrane:cytosolic PKC activity ratio was: 5.6±1.5, 5.3±1.8, 6.6±2.7, and 3.9±2.1 (P=NS in each instance). In conclusion, DOG is cardioprotective but under the conditions of the present study is less cardioprotective than ischemic preconditioning, furthermore the protection does not appear to necessitate PKC activation prior to ischemia.

Investigation of liquid MALDI and optimization for instrument tuning and quantitative measurements

The success of Matrix-assisted laser desorption / ionisation (MALDI) in fields such as proteomics has partially but not exclusively been due to the development of improved data acquisition and sample preparation techniques. This has been required to overcome some of the short comings of the commonly used solid-state MALDI matrices such as - cyano-4-hydroxycinnamic acid (CHCA) and 2,5-dihydroxybenzoic acid (DHB). Solid state matrices form crystalline samples with highly inhomogeneous topography and morphology which results in large fluctuations in analyte signal intensity from spot to spot and positions within the spot. This means that efficient tuning of the mass spectrometer can be impeded and the use of MALDI MS for quantitative measurements is severely impeded. Recently new MALDI liquid matrices have been introduced which promise to be an effective alternative to crystalline matrices. Generally the liquid matrices comprise either ionic liquid matrices (ILMs) or a usually viscous liquid matrix which is doped with a UV lightabsorbing chromophore [1-3]. The advantages are that the droplet surface is smooth and relatively uniform with the analyte homogeneously distributed within. They have the ability to replenish a sampling position between shots negating the need to search for sample hot-spots. Also the liquid nature of the matrix allows for the use of additional additives to change the environment to which the analyte is added.

Neural network enhanced generalised minimum variance self-tuning controller for nonlinear discrete-time systems

A neural network enhanced self-tuning controller is presented, which combines the attributes of neural network mapping with a generalised minimum variance self-tuning control (STC) strategy. In this way the controller can deal with nonlinear plants, which exhibit features such as uncertainties, nonminimum phase behaviour, coupling effects and may have unmodelled dynamics, and whose nonlinearities are assumed to be globally bounded. The unknown nonlinear plants to be controlled are approximated by an equivalent model composed of a simple linear submodel plus a nonlinear submodel. A generalised recursive least squares algorithm is used to identify the linear submodel and a layered neural network is used to detect the unknown nonlinear submodel in which the weights are updated based on the error between the plant output and the output from the linear submodel. The procedure for controller design is based on the equivalent model therefore the nonlinear submodel is naturally accommodated within the control law. Two simulation studies are provided to demonstrate the effectiveness of the control algorithm.

Filtering techniques in self-tuning

In the last few years a state-space formulation has been introduced into self-tuning control. This has not only allowed for a wider choice of possible control actions, but has also provided an insight into the theory underlying—and hidden by—that used in the polynomial description. This paper considers many of the self-tuning algorithms, both state-space and polynomial, presently in use, and by starting from first principles develops the observers which are, effectively, used in each case. At any specific time instant the state estimator can be regarded as taking one of two forms. In the first case the most recently available output measurement is excluded, and here an optimal and conditionally stable observer is obtained. In the second case the present output signal is included, and here it is shown that although the observer is once again conditionally stable, it is no longer optimal. This result is of significance, as many of the popular self-tuning controllers lie in the second, rather than first, category.

Self-tuning regulators—a state space approach

This paper employs a state space system description to provide a pole placement scheme via state feedback. It is shown that when a recursive least squares estimation scheme is used, the feedback employed can be expressed simply in terms of the estimated system parameters. To complement the state feedback approach, a method employing both state feedback and linear output feedback is discussed. Both methods arc then compared with the previous output polynomial type feedback schemes.

Simplified parameter adaptive control

A simple parameter adaptive controller design methodology is introduced in which steady-state servo tracking properties provide the major control objective. This is achieved without cancellation of process zeros and hence the underlying design can be applied to non-minimum phase systems. As with other self-tuning algorithms, the design (user specified) polynomials of the proposed algorithm define the performance capabilities of the resulting controller. However, with the appropriate definition of these polynomials, the synthesis technique can be shown to admit different adaptive control strategies, e.g. self-tuning PID and self-tuning pole-placement controllers. The algorithm can therefore be thought of as an embodiment of other self-tuning design techniques. The performances of some of the resulting controllers are illustrated using simulation examples and the on-line application to an experimental apparatus.

An implicit pole-assignment self-tuning regulator

A new self-tuning implicit pole-assignment algorithm is presented which, through the use of a pole compression factor and different RLS model and control structures, overcomes stability and convergence problems encountered in previously available algorithms. Computational requirements of the technique are much reduced when compared to explicit pole-assignment schemes, whereas the inherent robustness of the strategy is retained.

Auto-weighted minimum-variance self-tuning controller

A self-tuning controller which automatically assigns weightings to control and set-point following is introduced. This discrete-time single-input single-output controller is based on a generalized minimum-variance control strategy. The automatic on-line selection of weightings is very convenient, especially when the system parameters are unknown or slowly varying with respect to time, which is generally considered to be the type of systems for which self-tuning control is useful. This feature also enables the controller to overcome difficulties with non-minimum phase systems.

A neural network enhanced generalized minimum variance self-tuning proportional, integral and derivative control algorithm for complex dynamic systems

A neural network enhanced proportional, integral and derivative (PID) controller is presented that combines the attributes of neural network learning with a generalized minimum-variance self-tuning control (STC) strategy. The neuro PID controller is structured with plant model identification and PID parameter tuning. The plants to be controlled are approximated by an equivalent model composed of a simple linear submodel to approximate plant dynamics around operating points, plus an error agent to accommodate the errors induced by linear submodel inaccuracy due to non-linearities and other complexities. A generalized recursive least-squares algorithm is used to identify the linear submodel, and a layered neural network is used to detect the error agent in which the weights are updated on the basis of the error between the plant output and the output from the linear submodel. The procedure for controller design is based on the equivalent model, and therefore the error agent is naturally functioned within the control law. In this way the controller can deal not only with a wide range of linear dynamic plants but also with those complex plants characterized by severe non-linearity, uncertainties and non-minimum phase behaviours. Two simulation studies are provided to demonstrate the effectiveness of the controller design procedure.

Self-tuning proportional, integral and derivative controller based on genetic algorithm least squares

A self-tuning proportional, integral and derivative control scheme based on genetic algorithms (GAs) is proposed and applied to the control of a real industrial plant. This paper explores the improvement in the parameter estimator, which is an essential part of an adaptive controller, through the hybridization of recursive least-squares algorithms by making use of GAs and the possibility of the application of GAs to the control of industrial processes. Both the simulation results and the experiments on a real plant show that the proposed scheme can be applied effectively.

Complete robot-eye calibration without special calibration objects

A robot mounted camera is useful in many machine vision tasks as it allows control over view direction and position. In this paper we report a technique for calibrating both the robot and the camera using only a single corresponding point. All existing head-eye calibration systems we have encountered rely on using pre-calibrated robots, pre- calibrated cameras, special calibration objects or combinations of these. Our method avoids using large scale non-linear optimizations by recovering the parameters in small dependent groups. This is done by performing a series of planned, but initially uncalibrated robot movements. Many of the kinematic parameters are obtained using only camera views in which the calibration feature is at, or near the image center, thus avoiding errors which could be introduced by lens distortion. The calibration is shown to be both stable and accurate. The robotic system we use consists of camera with pan-tilt capability mounted on a Cartesian robot, providing a total of 5 degrees of freedom.

Automatic tuning of PID controllers using model reference adaptive control techniques

This paper discusses the application of model reference adaptive control concepts to the automatic tuning of PID controllers. The effectiveness of the proposed method is shown through simulated applications. The gradient approach and simulated examples are provided.