Nonlinear identification using orthogonal forward regression with nested optimal regularization

An efficient data based-modeling algorithm for nonlinear system identification is introduced for radial basis function (RBF) neural networks with the aim of maximizing generalization capability based on the concept of leave-one-out (LOO) cross validation. Each of the RBF kernels has its own kernel width parameter and the basic idea is to optimize the multiple pairs of regularization parameters and kernel widths, each of which is associated with a kernel, one at a time within the orthogonal forward regression (OFR) procedure. Thus, each OFR step consists of one model term selection based on the LOO mean square error (LOOMSE), followed by the optimization of the associated kernel width and regularization parameter, also based on the LOOMSE. Since like our previous state-of-the-art local regularization assisted orthogonal least squares (LROLS) algorithm, the same LOOMSE is adopted for model selection, our proposed new OFR algorithm is also capable of producing a very sparse RBF model with excellent generalization performance. Unlike our previous LROLS algorithm which requires an additional iterative loop to optimize the regularization parameters as well as an additional procedure to optimize the kernel width, the proposed new OFR algorithm optimizes both the kernel widths and regularization parameters within the single OFR procedure, and consequently the required computational complexity is dramatically reduced. Nonlinear system identification examples are included to demonstrate the effectiveness of this new approach in comparison to the well-known approaches of support vector machine and least absolute shrinkage and selection operator as well as the LROLS algorithm.

Estimation of Gaussian process regression model using probability distance measures

A new class of parameter estimation algorithms is introduced for Gaussian process regression (GPR) models. It is shown that the integration of the GPR model with probability distance measures of (i) the integrated square error and (ii) Kullback–Leibler (K–L) divergence are analytically tractable. An efficient coordinate descent algorithm is proposed to iteratively estimate the kernel width using golden section search which includes a fast gradient descent algorithm as an inner loop to estimate the noise variance. Numerical examples are included to demonstrate the effectiveness of the new identification approaches.

Tensor regression based on linked multiway parameter analysis

Classical regression methods take vectors as covariates and estimate the corresponding vectors of regression parameters. When addressing regression problems on covariates of more complex form such as multi-dimensional arrays (i.e. tensors), traditional computational models can be severely compromised by ultrahigh dimensionality as well as complex structure. By exploiting the special structure of tensor covariates, the tensor regression model provides a promising solution to reduce the model’s dimensionality to a manageable level, thus leading to efficient estimation. Most of the existing tensor-based methods independently estimate each individual regression problem based on tensor decomposition which allows the simultaneous projections of an input tensor to more than one direction along each mode. As a matter of fact, multi-dimensional data are collected under the same or very similar conditions, so that data share some common latent components but can also have their own independent parameters for each regression task. Therefore, it is beneficial to analyse regression parameters among all the regressions in a linked way. In this paper, we propose a tensor regression model based on Tucker Decomposition, which identifies not only the common components of parameters across all the regression tasks, but also independent factors contributing to each particular regression task simultaneously. Under this paradigm, the number of independent parameters along each mode is constrained by a sparsity-preserving regulariser. Linked multiway parameter analysis and sparsity modeling further reduce the total number of parameters, with lower memory cost than their tensor-based counterparts. The effectiveness of the new method is demonstrated on real data sets.

Probabilistic wind power forecasts with an inverse power curve transformation and censored regression

Forecasting wind power is an important part of a successful integration of wind power into the power grid. Forecasts with lead times longer than 6 h are generally made by using statistical methods to post-process forecasts from numerical weather prediction systems. Two major problems that complicate this approach are the non-linear relationship between wind speed and power production and the limited range of power production between zero and nominal power of the turbine. In practice, these problems are often tackled by using non-linear non-parametric regression models. However, such an approach ignores valuable and readily available information: the power curve of the turbine's manufacturer. Much of the non-linearity can be directly accounted for by transforming the observed power production into wind speed via the inverse power curve so that simpler linear regression models can be used. Furthermore, the fact that the transformed power production has a limited range can be taken care of by employing censored regression models. In this study, we evaluate quantile forecasts from a range of methods: (i) using parametric and non-parametric models, (ii) with and without the proposed inverse power curve transformation and (iii) with and without censoring. The results show that with our inverse (power-to-wind) transformation, simpler linear regression models with censoring perform equally or better than non-linear models with or without the frequently used wind-to-power transformation.

Tests of sunspot number sequences: 3. Effects of regression procedures on the calibration of historic sunspot data

We use sunspot group observations from the Royal Greenwich Observatory (RGO) to investigate the effects of intercalibrating data from observers with different visual acuities. The tests are made by counting the number of groups RB above a variable cut-off threshold of observed total whole-spot area (uncorrected for foreshortening) to simulate what a lower acuity observer would have seen. The synthesised annual means of RB are then re-scaled to the full observed RGO group number RA using a variety of regression techniques. It is found that a very high correlation between RA and RB (rAB > 0.98) does not prevent large errors in the intercalibration (for example sunspot maximum values can be over 30 % too large even for such levels of rAB). In generating the backbone sunspot number (RBB), Svalgaard and Schatten (2015, this issue) force regression fits to pass through the scatter plot origin which generates unreliable fits (the residuals do not form a normal distribution) and causes sunspot cycle amplitudes to be exaggerated in the intercalibrated data. It is demonstrated that the use of Quantile-Quantile (“Q  Q”) plots to test for a normal distribution is a useful indicator of erroneous and misleading regression fits. Ordinary least squares linear fits, not forced to pass through the origin, are sometimes reliable (although the optimum method used is shown to be different when matching peak and average sunspot group numbers). However, other fits are only reliable if non-linear regression is used. From these results it is entirely possible that the inflation of solar cycle amplitudes in the backbone group sunspot number as one goes back in time, relative to related solar-terrestrial parameters, is entirely caused by the use of inappropriate and non-robust regression techniques to calibrate the sunspot data.

The specific role of relationship life events in the onset of depression during pregnancy and the postpartum

Background The precipitating role of life events in the onset of depression is well-established. The present study sought to examine whether life events hypothesised to be personally salient would be more strongly associated with depression than other life events. In a sample of women making the first transition to parenthood, we hypothesised that negative events related to the partner relationship would be particularly salient and thus more strongly predictive of depression than other events. Methods A community-based sample of 316 first-time mothers stratified by psychosocial risk completed interviews at 32 weeks gestation and 29 weeks postpartum to assess dated occurrence of life events and depression onsets from conception to 29 weeks postpartum. Complete data was available from 273 (86.4%). Cox proportional hazards regression was used to examine risk for onset of depression in the 6 months following a relationship event versus other events, after accounting for past history of depression and other potential confounders. Results 52 women (19.0%) experienced an onset of depression between conception and 6 months postpartum. Both relationship events (Hazard Ratio = 2.1, p = .001) and other life events (Hazard Ratio = 1.3, p = .020) were associated with increased risk for depression onset; however, relationship events showed a significantly greater risk for depression than did other life events (p = .044). Conclusions The results are consistent with the hypothesis that personally salient events are more predictive of depression onset than other events. Further, they indicate the clinical significance of events related to the partner relationship during pregnancy and the postpartum.

Low-level Laser Therapy Improves Skeletal Muscle Performance, Decreases Skeletal Muscle Damage and Modulates mRNA Expression of COX-1 and COX-2 in a Dose-dependent Manner

We tested if modulation in mRNA expression of cyclooxygenase isoforms (COX-1 and COX-2) can be related to protective effects of phototherapy in skeletal muscle. Thirty male Wistar rats were divided into five groups receiving either one of four laser doses (0.1, 0.3, 1.0 and 3.0 J) or a no-treatment control group. Laser irradiation (904 nm, 15 mW average power) was performed immediately before the first contraction for treated groups. Electrical stimulation was used to induce six tetanic tibial anterior muscle contractions. Immediately after sixth contraction, blood samples were collected to evaluate creatine kinase activity and muscles were dissected and frozen in liquid nitrogen to evaluate mRNA expression of COX-1 and COX-2. The 1.0 and 3.0 J groups showed significant enhancement (P < 0.01) in total work performed in six tetanic contractions compared with control group. All laser groups, except the 3.0 J group, presented significantly lower post-exercise CK activity than control group. Additionally, 1.0 J group showed increased COX-1 and decreased COX-2 mRNA expression compared with control group and 0.1, 0.3 and 3.0 J laser groups (P < 0.01). We conclude that pre-exercise infrared laser irradiation with dose of 1.0 J enhances skeletal muscle performance and decreases post-exercise skeletal muscle damage and inflammation.

Celecoxib prevents tumor growth in an animal model by a COX-2 independent mechanism

Nonsteroidal antiinflammatory drugs (NSAIDs) have been shown to reduce cell growth in several tumors. Among these possible antineoplastic drugs are cyclooxygenase-2 (COX-2)-selective drugs, such as celecoxib, in which antitumoral mechanisms were evaluated in rats bearing Walker-256 (W256) tumor. W256 carcinosarcoma cells were inoculated subcutaneously (10(7) cells/rat) in rats submitted to treatment with celecoxib (25 mg kg(-1)) or vehicle for 14 days. Tumor growth, body-weight gain, and survival data were evaluated. The mechanisms, such as COX-2 expression and activity, oxidative stress, by means of enzymes and lipoperoxidation levels, and apoptosis mediators were also investigated. A reduction in tumor growth and an increased weight gain were observed. Celecoxib provided a higher incidence of survival compared with the control group. Cellular effects are probably COX-2 independent, because neither enzyme expression nor its activity, measured by tumoral PGE(2), showed significant difference between groups. It is probable that this antitumor action is dependent on an apoptotic way, which has been evaluated by the expression of the antiapoptotic protein Bcl-xL, in addition to the cellular changes observed by electronic microscopy. Celecoxib has also a possible involvement with redox homeostasis, because its administration caused significant changes in the activity of oxidative enzymes, such as catalase and superoxide dismutase. These results confirm the antitumor effects of celecoxib in W256 cancer model, contributing to elucidating its antitumoral mechanism and corroborating scientific literature about its effect on other types of cancer.

Insulin suppresses LPS-induced iNOS and COX-2 expression and NF-kappa B activation in alveolar macrophages

The development of septic shock is a common and frequently lethal consequence of gram-negative infection. Mediators released by lung macrophages activated by bacterial products such as lipopolysaccharide (LPS) contribute to shock symptoms. We have shown that insulin downregulates LPS-induced TNF production by alveolar macrophages (AMs). In the present study, we investigated the effect of insulin on the LPS-induced production of nitric oxide (NO) and prostaglandin (PG)-E(2), on the expression of inducible nitric oxide synthase ( iNOS) and cyclooxygenase (COX)-2, and on nuclear factor kappa B (NF-kappa B) activation in AMs. Resident AMs from male Wistar rats were stimulated with LPS (100 ng/mL) for 30 minutes. Insulin (1 mU/mL) was added 10 min before LPS. Enzymes expression, NF-kappa B p65 activation and inhibitor of kappa B (I-kappa B) a phosphorylation were assessed by immunobloting; NO by Griess reaction and PGE(2) by enzyme immunoassay (EIA). LPS induced in AMs the expression of iNOS and COX-2 proteins and production of NO and PGE(2), and, in parallel, NF-kappa B p65 activation and cytoplasmic I-kappa B alpha phosphorylation. Administration of insulin before LPS suppressed the expression of iNOS and COX-2, of NO and PGE(2) production and Nuclear NF-kappa B p65 activation. Insulin also prevented cytoplasmic I-kappa Ba phosphorylation. These results show that in AMs stimulated by LPS, insulin prevents nuclear translocation of NF-kappa B, possibly by blocking I-kappa Ba degradation, and supresses the production of NO and PGE(2), two molecules that contribute to septic shock. Copyright (C) 2008 S. Karger AG, Basel.

Heteroscedastic Nonlinear Regression Models

In this article, we present a generalization of the Bayesian methodology introduced by Cepeda and Gamerman (2001) for modeling variance heterogeneity in normal regression models where we have orthogonality between mean and variance parameters to the general case considering both linear and highly nonlinear regression models. Under the Bayesian paradigm, we use MCMC methods to simulate samples for the joint posterior distribution. We illustrate this algorithm considering a simulated data set and also considering a real data set related to school attendance rate for children in Colombia. Finally, we present some extensions of the proposed MCMC algorithm.

Regression Models with Heteroscedasticity using Bayesian Approach

In this paper, we compare the performance of two statistical approaches for the analysis of data obtained from the social research area. In the first approach, we use normal models with joint regression modelling for the mean and for the variance heterogeneity. In the second approach, we use hierarchical models. In the first case, individual and social variables are included in the regression modelling for the mean and for the variance, as explanatory variables, while in the second case, the variance at level 1 of the hierarchical model depends on the individuals (age of the individuals), and in the level 2 of the hierarchical model, the variance is assumed to change according to socioeconomic stratum. Applying these methodologies, we analyze a Colombian tallness data set to find differences that can be explained by socioeconomic conditions. We also present some theoretical and empirical results concerning the two models. From this comparative study, we conclude that it is better to jointly modelling the mean and variance heterogeneity in all cases. We also observe that the convergence of the Gibbs sampling chain used in the Markov Chain Monte Carlo method for the jointly modeling the mean and variance heterogeneity is quickly achieved.

Methods of Estimation in Multiple Linear Regression: Application to Clinical Data

Nesse artigo, tem-se o interesse em avaliar diferentes estratégias de estimação de parâmetros para um modelo de regressão linear múltipla. Para a estimação dos parâmetros do modelo foram utilizados dados de um ensaio clínico em que o interesse foi verificar se o ensaio mecânico da propriedade de força máxima (EM-FM) está associada com a massa femoral, com o diâmetro femoral e com o grupo experimental de ratas ovariectomizadas da raça Rattus norvegicus albinus, variedade Wistar. Para a estimação dos parâmetros do modelo serão comparadas três metodologias: a metodologia clássica, baseada no método dos mínimos quadrados; a metodologia Bayesiana, baseada no teorema de Bayes; e o método Bootstrap, baseado em processos de reamostragem.

Bayesian nonlinear regression models with scale mixtures of skew-normal distributions: Estimation and case influence diagnostics

The purpose of this paper is to develop a Bayesian analysis for nonlinear regression models under scale mixtures of skew-normal distributions. This novel class of models provides a useful generalization of the symmetrical nonlinear regression models since the error distributions cover both skewness and heavy-tailed distributions such as the skew-t, skew-slash and the skew-contaminated normal distributions. The main advantage of these class of distributions is that they have a nice hierarchical representation that allows the implementation of Markov chain Monte Carlo (MCMC) methods to simulate samples from the joint posterior distribution. In order to examine the robust aspects of this flexible class, against outlying and influential observations, we present a Bayesian case deletion influence diagnostics based on the Kullback-Leibler divergence. Further, some discussions on the model selection criteria are given. The newly developed procedures are illustrated considering two simulations study, and a real data previously analyzed under normal and skew-normal nonlinear regression models. (C) 2010 Elsevier B.V. All rights reserved.

A bivariate regression model for matched paired survival data: local influence and residual analysis

The use of bivariate distributions plays a fundamental role in survival and reliability studies. In this paper, we consider a location scale model for bivariate survival times based on the proposal of a copula to model the dependence of bivariate survival data. For the proposed model, we consider inferential procedures based on maximum likelihood. Gains in efficiency from bivariate models are also examined in the censored data setting. For different parameter settings, sample sizes and censoring percentages, various simulation studies are performed and compared to the performance of the bivariate regression model for matched paired survival data. Sensitivity analysis methods such as local and total influence are presented and derived under three perturbation schemes. The martingale marginal and the deviance marginal residual measures are used to check the adequacy of the model. Furthermore, we propose a new measure which we call modified deviance component residual. The methodology in the paper is illustrated on a lifetime data set for kidney patients.

A nonlinear regression model with skew-normal errors

In this paper we have discussed inference aspects of the skew-normal nonlinear regression models following both, a classical and Bayesian approach, extending the usual normal nonlinear regression models. The univariate skew-normal distribution that will be used in this work was introduced by Sahu et al. (Can J Stat 29:129-150, 2003), which is attractive because estimation of the skewness parameter does not present the same degree of difficulty as in the case with Azzalini (Scand J Stat 12:171-178, 1985) one and, moreover, it allows easy implementation of the EM-algorithm. As illustration of the proposed methodology, we consider a data set previously analyzed in the literature under normality.