A constrained recursive least squares algorithm for adaptive combination of multiple models

In this paper, we develop a novel constrained recursive least squares algorithm for adaptively combining a set of given multiple models. With data available in an online fashion, the linear combination coefficients of submodels are adapted via the proposed algorithm.We propose to minimize the mean square error with a forgetting factor, and apply the sum to one constraint to the combination parameters. Moreover an l1-norm constraint to the combination parameters is also applied with the aim to achieve sparsity of multiple models so that only a subset of models may be selected into the final model. Then a weighted l2-norm is applied as an approximation to the l1-norm term. As such at each time step, a closed solution of the model combination parameters is available. The contribution of this paper is to derive the proposed constrained recursive least squares algorithm that is computational efficient by exploiting matrix theory. The effectiveness of the approach has been demonstrated using both simulated and real time series examples.

Language processing with dynamic fields

We construct a mapping from complex recursive linguistic data structures to spherical wave functions using Smolensky's filler/role bindings and tensor product representations. Syntactic language processing is then described by the transient evolution of these spherical patterns whose amplitudes are governed by nonlinear order parameter equations. Implications of the model in terms of brain wave dynamics are indicated.

Bayesian estimation of willingness-to-pay where respondents mis-report their preferences

We introduce a modified conditional logit model that takes account of uncertainty associated with mis-reporting in revealed preference experiments estimating willingness-to-pay (WTP). Like Hausman et al. [Journal of Econometrics (1988) Vol. 87, pp. 239-269], our model captures the extent and direction of uncertainty by respondents. Using a Bayesian methodology, we apply our model to a choice modelling (CM) data set examining UK consumer preferences for non-pesticide food. We compare the results of our model with the Hausman model. WTP estimates are produced for different groups of consumers and we find that modified estimates of WTP, that take account of mis-reporting, are substantially revised downwards. We find a significant proportion of respondents mis-reporting in favour of the non-pesticide option. Finally, with this data set, Bayes factors suggest that our model is preferred to the Hausman model.

Modifying willingness to pay estimates where respondents mis-report their preferences

The likelihood for the Logit model is modified, so as to take account of uncertainty associated with mis-reporting in stated preference experiments estimating willingness to pay (WTP). Monte Carlo results demonstrate the bias imparted to estimates where there is mis-reporting. The approach is applied to a data set examining consumer preferences for food produced employing a nonpesticide technology. Our modified approach leads to WTP that are substantially downwardly revised.

Consumer willingness to pay for in-flight service and comfort levels: A choice experiment

In the competitive aviation market as a result of the emergence of low cost carriers, charter airlines have had to reconsider their approach to service provision. Specifically, the reduction in service and comfort levels offered by the low cost airlines provides charter carriers with an opportunity to differentiate their product based on the quality of the offering. To consider this strategic option we employ an on-line choice experiment to examine consumer choices with respect to the bundle of services on offer when deciding to purchase a flight, With these data we use the Bayesian methods to estimate a mixed logit specification. Our results reveal that in principle passengers are willing to pay a relatively large amount for enhanced service quality. (C) 2008 Elsevier Ltd. All rights reserved.

An investigation into factors affecting the occupational choices of nominated farm heirs in Ireland

The resilience of family farming is an important feature of the structure of the farming industry in many countries, due largely to the 'smooth' succession of farms from one generation to the next. The stability of this structure is now threatened by the widening gap between the income expected from farming when compared with non-farming occupations in an economy like Ireland, operating at almost full employment. Nominated farm heirs are increasingly unlikely to choose full-time farming as their preferred occupation. To identify the factors that affect this occupational choice, a multinomial logit model is developed and applied to Irish data to examine the farm, economic and personal characteristics that influence a nominated heir's decision to enter farming as opposed to some non-farming occupation. The results show a significant negative relationship between higher education and the choice of full-time farming as an occupation. The interdependence between education and occupational choices is further explored using a bivariate probit model. The main findings are: the occupational choice and the decision to continue with higher education are made jointly; the nominated heirs on more profitable farms are less likely to pursue tertiary education and therefore more likely to enter full-time farming. The model developed is sufficiently general for studying the phenomenon of succession on farms.

Identifying important endemic areas using ecoregions: birds and mammals in the Indo-Pacific

Concentrations of large numbers of endemic species have been singled out in prioritization exercises as significant areas for global biodiversity conservation. This paper describes bird and mammal endemicity in Indo-Pacific ecoregions. An ecoregion is a relatively large unit of land or water that contains a distinct assemblage of natural communities. We prioritize 133 ecoregions according to their levels of endemicity, and explain how variables such as biome type, whether the ecoregion is on an island or continental mass, montane or non-montane, correlate with the proportion of the total species assemblage that are endemic. Following an exploratory principal components analysis we classify all ecoregions according to the relationship between numbers of endemics and overall species richness. Endemicity is negatively correlated with species richness. We show that plotting the logit transformation of the endemicity of birds and mammals against log of species richness is a more effective and useful way of identifying important ecoregions than simply ordering ecoregions by the proportion of endemic species, or any other single measure. The plot, divided into 16 regions corresponding to the quartiles of the two variables, was used to identify ecoregions of high conservation value. These are the ecoregions with the highest endemicity and lowest species richness. Further analysis shows that island and montane ecoregions, regardless of their biome type, are by far the most important for endemic species.

A practical approach to a multi-level analysis with a sparse binary outcome within a large surgical trial

Objectives: To assess the potential source of variation that surgeon may add to patient outcome in a clinical trial of surgical procedures. Methods: Two large (n = 1380) parallel multicentre randomized surgical trials were undertaken to compare laparoscopically assisted hysterectomy with conventional methods of abdominal and vaginal hysterectomy; involving 43 surgeons. The primary end point of the trial was the occurrence of at least one major complication. Patients were nested within surgeons giving the data set a hierarchical structure. A total of 10% of patients had at least one major complication, that is, a sparse binary outcome variable. A linear mixed logistic regression model (with logit link function) was used to model the probability of a major complication, with surgeon fitted as a random effect. Models were fitted using the method of maximum likelihood in SAS((R)). Results: There were many convergence problems. These were resolved using a variety of approaches including; treating all effects as fixed for the initial model building; modelling the variance of a parameter on a logarithmic scale and centring of continuous covariates. The initial model building process indicated no significant 'type of operation' across surgeon interaction effect in either trial, the 'type of operation' term was highly significant in the abdominal trial, and the 'surgeon' term was not significant in either trial. Conclusions: The analysis did not find a surgeon effect but it is difficult to conclude that there was not a difference between surgeons. The statistical test may have lacked sufficient power, the variance estimates were small with large standard errors, indicating that the precision of the variance estimates may be questionable.

An on-line algorithm of uncertain time delay estimation in a continuous system

In this paper, we present an on-line estimation algorithm for an uncertain time delay in a continuous system based on the observational input-output data, subject to observational noise. The first order Pade approximation is used to approximate the time delay. At each time step, the algorithm combines the well known Kalman filter algorithm and the recursive instrumental variable least squares (RIVLS) algorithm in cascade form. The instrumental variable least squares algorithm is used in order to achieve the consistency of the delay parameter estimate, since an error-in-the-variable model is involved. An illustrative example is utilized to demonstrate the efficacy of the proposed approach.

Full interference cancellation for two-path cooperative communications

This paper proposes a full interference cancellation (FIC) approach for two-path cooperative communications. Unlike the single relay schemes, the two-path cooperative scheme involves two relay nodes, so that the source can continuously transmit data to the two relays alternatively and the full bandwidth efficiency with respect to the direct transmission can be retained. The two-path relay scheme may however suffer from inter-relay interference which is caused by the simultaneous transmission of the source and one of the relays at any time. In this paper, first the inter-relay interference is expressed as a single recursive term in the received signal, and then the FIC approach is proposed to fully remove the inter-relay interference. The FIC has not only better performance but also less complexity than existing approaches. Numerical examples are also given to verify the proposed approach.

A kernel-based two-class classifier for imbalanced data sets

Many kernel classifier construction algorithms adopt classification accuracy as performance metrics in model evaluation. Moreover, equal weighting is often applied to each data sample in parameter estimation. These modeling practices often become problematic if the data sets are imbalanced. We present a kernel classifier construction algorithm using orthogonal forward selection (OFS) in order to optimize the model generalization for imbalanced two-class data sets. This kernel classifier identification algorithm is based on a new regularized orthogonal weighted least squares (ROWLS) estimator and the model selection criterion of maximal leave-one-out area under curve (LOO-AUC) of the receiver operating characteristics (ROCs). It is shown that, owing to the orthogonalization procedure, the LOO-AUC can be calculated via an analytic formula based on the new regularized orthogonal weighted least squares parameter estimator, without actually splitting the estimation data set. The proposed algorithm can achieve minimal computational expense via a set of forward recursive updating formula in searching model terms with maximal incremental LOO-AUC value. Numerical examples are used to demonstrate the efficacy of the algorithm.

A fast linear-in-the-parameters classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate

We propose a simple and computationally efficient construction algorithm for two class linear-in-the-parameters classifiers. In order to optimize model generalization, a forward orthogonal selection (OFS) procedure is used for minimizing the leave-one-out (LOO) misclassification rate directly. An analytic formula and a set of forward recursive updating formula of the LOO misclassification rate are developed and applied in the proposed algorithm. Numerical examples are used to demonstrate that the proposed algorithm is an excellent alternative approach to construct sparse two class classifiers in terms of performance and computational efficiency.

Backward elimination model construction for regression and classification using leave-one-out criteria

A fundamental principle in practical nonlinear data modeling is the parsimonious principle of constructing the minimal model that explains the training data well. Leave-one-out (LOO) cross validation is often used to estimate generalization errors by choosing amongst different network architectures (M. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974). Based upon the minimization of LOO criteria of either the mean squares of LOO errors or the LOO misclassification rate respectively, we present two backward elimination algorithms as model post-processing procedures for regression and classification problems. The proposed backward elimination procedures exploit an orthogonalization procedure to enable the orthogonality between the subspace as spanned by the pruned model and the deleted regressor. Subsequently, it is shown that the LOO criteria used in both algorithms can be calculated via some analytic recursive formula, as derived in this contribution, without actually splitting the estimation data set so as to reduce computational expense. Compared to most other model construction methods, the proposed algorithms are advantageous in several aspects; (i) There are no tuning parameters to be optimized through an extra validation data set; (ii) The procedure is fully automatic without an additional stopping criteria; and (iii) The model structure selection is directly based on model generalization performance. The illustrative examples on regression and classification are used to demonstrate that the proposed algorithms are viable post-processing methods to prune a model to gain extra sparsity and improved generalization.

Automatic nonlinear predictive model-construction algorithm using forward regression and the PRESS statistic

An automatic nonlinear predictive model-construction algorithm is introduced based on forward regression and the predicted-residual-sums-of-squares (PRESS) statistic. The proposed algorithm is based on the fundamental concept of evaluating a model's generalisation capability through crossvalidation. This is achieved by using the PRESS statistic as a cost function to optimise model structure. In particular, the proposed algorithm is developed with the aim of achieving computational efficiency, such that the computational effort, which would usually be extensive in the computation of the PRESS statistic, is reduced or minimised. The computation of PRESS is simplified by avoiding a matrix inversion through the use of the orthogonalisation procedure inherent in forward regression, and is further reduced significantly by the introduction of a forward-recursive formula. Based on the properties of the PRESS statistic, the proposed algorithm can achieve a fully automated procedure without resort to any other validation data set for iterative model evaluation. Numerical examples are used to demonstrate the efficacy of the algorithm.

Generalized matching networks and their properties

In this paper, we introduce two kinds of graphs: the generalized matching networks (GMNs) and the recursive generalized matching networks (RGMNs). The former generalize the hypercube-like networks (HLNs), while the latter include the generalized cubes and the star graphs. We prove that a GMN on a family of k-connected building graphs is -connected. We then prove that a GMN on a family of Hamiltonian-connected building graphs having at least three vertices each is Hamiltonian-connected. Our conclusions generalize some previously known results.