Two-stage sequential procedure for nonparametric regression estimation

In nonparametric statistics the functional form of the relationship between the response variable and its associated predictor variables is unspecified but it is assumed to be a smooth function. We develop a procedure for constructing a fixed width confidence interval for the predicted value at a specified point of the independent variable. The optimal sample size for constructing this interval is obtained using a two stage sequential procedure which relies on some asymptotic properties of the Nadaraya--Watson and local linear estimators. Finally, a large scale simulation study demonstrates the applicability of the developed procedure for small and moderate sample sizes. The procedure developed here should find wide applicability since many practical problems which arise in industry involve estimating an unknown function.

Application of sequential nonparametric confidence bands in finance

In a nonparametric setting, the functional form of the relationship between the response variable and the associated predictor variables is assumed to be unknown when data is fitted to the model. Non-parametric regression models can be used for the same types of applications such as estimation, prediction, calibration, and optimization that traditional regression models are used for. The main aim of nonparametric regression is to highlight an important structure in the data without any assumptions about the shape of an underlying regression function. Hence the nonparametric approach allows the data to speak for itself. Applications of sequential procedures to a nonparametric regression model at a given point are considered.

The primary goal of sequential analysis is to achieve a given accuracy by using the smallest possible sample sizes. These sequential procedures allow an experimenter to make decisions based on the smallest number of observations without compromising accuracy. In the nonparametric regression model with a random design based on independent and identically distributed pairs of observations (X ,Y ), where the regression function m(x) is given bym(x) = E(Y X = x), estimation of the Nadaraya-Watson kernel estimator (m (x)) NW and local linear kernel estimator (m (x)) LL for the curve m(x) is considered. In order to obtain asymptotic confidence intervals form(x), two stage sequential procedure is used under which some asymptotic properties of Nadaraya-Watson and local linear estimators have been obtained.

The proposed methodology is first tested with the help of simulated data from linear and nonlinear functions. Encouraged by the preliminary findings from simulation results, the proposed method is applied to estimate the nonparametric regression curve of CAPM.

Sequential fixed-width confidence bands for kernel regression estimation

We consider a random design model based on independent and identically distributed (iid) pairs of observations (Xi, Yi), where the regression function m(x) is given by m(x) = E(Yi|Xi = x) with one independent variable. In a nonparametric setting the aim is to produce a reasonable approximation to the unknown function m(x) when we have no precise information about the form of the true density, f(x) of X. We describe an estimation procedure of non-parametric regression model at a given point by some appropriately constructed fixed-width (2d) confidence interval with the confidence coefficient of at least 1−. Here, d(> 0) and 2 (0, 1) are two preassigned values. Fixed-width confidence intervals are developed using both Nadaraya-Watson and local linear kernel estimators of nonparametric regression with data-driven bandwidths.

The sample size was optimized using the purely and two-stage sequential procedure together with asymptotic properties of the Nadaraya-Watson and local linear estimators. A large scale simulation study was performed to compare their coverage accuracy. The numerical results indicate that the confidence bands based on the local linear estimator have the best performance than those constructed by using Nadaraya-Watson estimator. However both estimators are shown to have asymptotically correct coverage properties.

Sample size determination for kernel regression estimation using sequential fixed-width confidence bands

We consider a random design model based on independent and identically distributed pairs of observations (Xi, Yi), where the regression function m(x) is given by m(x) = E(Yi|Xi = x) with one independent variable. In a nonparametric setting the aim is to produce a reasonable approximation to the unknown function m(x) when we have no precise information about the form of the true density, f(x) of X. We describe an estimation procedure of non-parametric regression model at a given point by some appropriately constructed fixed-width (2d) confidence interval with the confidence coefficient of at least 1−. Here, d(> 0) and 2 (0, 1) are two preassigned values. Fixed-width confidence intervals are developed using both Nadaraya-Watson and local linear kernel estimators of nonparametric regression with data-driven bandwidths. The sample size was optimized using the purely and two-stage sequential procedures together with asymptotic properties of the Nadaraya-Watson and local linear estimators. A large scale simulation study was performed to compare their coverage accuracy. The numerical results indicate that the confi dence bands based on the local linear estimator have the better performance than those constructed by using Nadaraya-Watson estimator. However both estimators are shown to have asymptotically correct coverage properties.

Joint (X)over-bar and R charts with two-stage samplings

When joint (X) over bar and R charts are in use, samples of fixed size are regularly taken from the process, and their means and ranges are plotted on the (X) over bar and R charts, respectively. In this article, joint (X) over bar and R charts have been used for monitoring continuous production processes. The sampling is performed, in two stages. During the first stage, one item of the sample is inspected and, depending on the result, the sampling is interrupted if the process is found to be in control; otherwise, it goes on to the second stage, where the remaining sample items are inspected. The two-stage sampling procedure speeds up the detection of process disturbances. The proposed joint (X) over bar and R charts are easier to administer and are more efficient than the joint (X) over bar and R charts with variable sample size where the quality characteristic of interest can be evaluated either by attribute or variable. Copyright (C) 2004 John Wiley Sons, Ltd.

A sequential procedure for comparing two experimental treatments with a control

A procedure is described in which patients are randomized between two experimental treatments and a control. At a series of interim analyses, each experimental treatment is compared with control. One of the experimental treatments might then be found sufficiently superior to the control for it to be declared the best treatment, and the trial stopped. Alternatively, experimental treatments might be eliminated from further consideration at any stage. It is shown how the procedure can be conducted while controlling overall error probabilities. Data concerning evaluation of different doses of riluzole in the treatment of motor neurone disease are used for illustration.

Estimating process capability index Cpm using a bootstrap sequential sampling procedure

Construction of a confidence interval for process capability index CPM is often based on a normal approximation with fixed sample size. In this article, we describe a different approach in constructing a fixed-width confidence interval for process capability index CPM with a preassigned accuracy by using a combination of bootstrap and sequential sampling schemes. The optimal sample size required to achieve a preassigned confidence level is obtained using both two-stage and modified two-stage sequential procedures. The procedure developed is also validated using an extensive simulation study.

A two-stage algorithm for identification of nonlinear dynamic systems

This paper investigates the two-stage stepwise identification for a class of nonlinear dynamic systems that can be described by linear-in-the-parameters models, and the model has to be built from a very large pool of basis functions or model terms. The main objective is to improve the compactness of the model that is obtained by the forward stepwise methods, while retaining the computational efficiency. The proposed algorithm first generates an initial model using a forward stepwise procedure. The significance of each selected term is then reviewed at the second stage and all insignificant ones are replaced, resulting in an optimised compact model with significantly improved performance. The main contribution of this paper is that these two stages are performed within a well-defined regression context, leading to significantly reduced computational complexity. The efficiency of the algorithm is confirmed by the computational complexity analysis, and its effectiveness is demonstrated by the simulation results.

Two-stage mixed discrete-continuous identification of radial basis function (RBF) neural models for nonlinear systems

The identification of nonlinear dynamic systems using radial basis function (RBF) neural models is studied in this paper. Given a model selection criterion, the main objective is to effectively and efficiently build a parsimonious compact neural model that generalizes well over unseen data. This is achieved by simultaneous model structure selection and optimization of the parameters over the continuous parameter space. It is a mixed-integer hard problem, and a unified analytic framework is proposed to enable an effective and efficient two-stage mixed discrete-continuous; identification procedure. This novel framework combines the advantages of an iterative discrete two-stage subset selection technique for model structure determination and the calculus-based continuous optimization of the model parameters. Computational complexity analysis and simulation studies confirm the efficacy of the proposed algorithm.

Locally regularised two-stage learning algorithm for RBF network centre selection

Nonlinear models constructed from radial basis function (RBF) networks can easily be over-fitted due to the noise on the data. While information criteria, such as the final prediction error (FPE), can provide a trade-off between training error and network complexity, the tunable parameters that penalise a large size of network model are hard to determine and are usually network dependent. This article introduces a new locally regularised, two-stage stepwise construction algorithm for RBF networks. The main objective is to produce a parsomous network that generalises well over unseen data. This is achieved by utilising Bayesian learning within a two-stage stepwise construction procedure to penalise centres that are mainly interpreted by the noise.

Fast automatic two-stage nonlinear model identification based on the extreme learning machine

It is convenient and effective to solve nonlinear problems with a model that has a linear-in-the-parameters (LITP) structure. However, the nonlinear parameters (e.g. the width of Gaussian function) of each model term needs to be pre-determined either from expert experience or through exhaustive search. An alternative approach is to optimize them by a gradient-based technique (e.g. Newton’s method). Unfortunately, all of these methods still need a lot of computations. Recently, the extreme learning machine (ELM) has shown its advantages in terms of fast learning from data, but the sparsity of the constructed model cannot be guaranteed. This paper proposes a novel algorithm for automatic construction of a nonlinear system model based on the extreme learning machine. This is achieved by effectively integrating the ELM and leave-one-out (LOO) cross validation with our two-stage stepwise construction procedure [1]. The main objective is to improve the compactness and generalization capability of the model constructed by the ELM method. Numerical analysis shows that the proposed algorithm only involves about half of the computation of orthogonal least squares (OLS) based method. Simulation examples are included to confirm the efficacy and superiority of the proposed technique.

On the Convergence of Two-Stage Iterative Processes for Solving Linear Equations

This paper considers two-stage iterative processes for solving the linear system $Af = b$. The outer iteration is defined by $Mf^{k + 1} = Nf^k + b$, where $M$ is a nonsingular matrix such that $M - N = A$. At each stage $f^{k + 1} $ is computed approximately using an inner iteration process to solve $Mv = Nf^k + b$ for $v$. At the $k$th outer iteration, $p_k $ inner iterations are performed. It is shown that this procedure converges if $p_k \geqq P$ for some $P$ provided that the inner iteration is convergent and that the outer process would converge if $f^{k + 1} $ were determined exactly at every step. Convergence is also proved under more specialized conditions, and for the procedure where $p_k = p$ for all $k$, an estimate for $p$ is obtained which optimizes the convergence rate. Examples are given for systems arising from the numerical solution of elliptic partial differential equations and numerical results are presented.

The impact of ownership structure on corporate debt policy : two stage least square simultaneous model approach for post crisis period : evidence from Kuala Lumpur Stock Exchange

Ownership concentration as a governance mechanism has received considerable attention among academician, practitioners as well as policy makers because large-block shareholders are increasingly active in their demands that corporations adopt effective governance mechanisms to control managerial decisions, which include corporate debt policy. Earlier study on the agency model of the firm widely recognizes that the managerial ownership and external debt play an important role in mitigating agency conflicts and enhancing firm value. They also found that increase in the external monitors, for example the institutional investors, can actually play a useful role in limiting agency problems in the firm. This paper, using 100 Composite Index companies from Brusa Malaysia between 1998 to 2002 explores the impact of institutional holdings on managerial ownership and debt policy in an integrated framework by using a simultaneous equations estimation procedure (2SLS). The findings show that there is a significant impact of institutional ownership which serves effective control mechanism on managerial ownership and corporate debt policy as hypothesized. Findings of such evidence suggest that institutional holding thus have played an important role in managers' strategic management decision and reduce agency conflict. In addition, corporate debt policy too is governed by managerial ownership and exhibited a negative relation.

Monitoring the process mean and variance using a synthetic control chart with two-stage testing

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)