Noise robust voice activity detection using features extracted from the time-domain autocorrelation function

This paper presents a method of voice activity detection (VAD) for high noise scenarios, using a noise robust voiced speech detection feature. The developed method is based on the fusion of two systems. The first system utilises the maximum peak of the normalised time-domain autocorrelation function (MaxPeak). The second zone system uses a novel combination of cross-correlation and zero-crossing rate of the normalised autocorrelation to approximate a measure of signal pitch and periodicity (CrossCorr) that is hypothesised to be noise robust. The score outputs by the two systems are then merged using weighted sum fusion to create the proposed autocorrelation zero-crossing rate (AZR) VAD. Accuracy of AZR was compared to state of the art and standardised VAD methods and was shown to outperform the best performing system with an average relative improvement of 24.8% in half-total error rate (HTER) on the QUT-NOISE-TIMIT database created using real recordings from high-noise environments.

Computed success rates of various carrier phase integer estimation solutions and their comparison with statistical success rates

The success rate of carrier phase ambiguity resolution (AR) is the probability that the ambiguities are successfully fixed to their correct integer values. In existing works, an exact success rate formula for integer bootstrapping estimator has been used as a sharp lower bound for the integer least squares (ILS) success rate. Rigorous computation of success rate for the more general ILS solutions has been considered difficult, because of complexity of the ILS ambiguity pull-in region and computational load of the integration of the multivariate probability density function. Contributions of this work are twofold. First, the pull-in region mathematically expressed as the vertices of a polyhedron is represented by a multi-dimensional grid, at which the cumulative probability can be integrated with the multivariate normal cumulative density function (mvncdf) available in Matlab. The bivariate case is studied where the pull-region is usually defined as a hexagon and the probability is easily obtained using mvncdf at all the grid points within the convex polygon. Second, the paper compares the computed integer rounding and integer bootstrapping success rates, lower and upper bounds of the ILS success rates to the actual ILS AR success rates obtained from a 24 h GPS data set for a 21 km baseline. The results demonstrate that the upper bound probability of the ILS AR probability given in the existing literatures agrees with the actual ILS success rate well, although the success rate computed with integer bootstrapping method is a quite sharp approximation to the actual ILS success rate. The results also show that variations or uncertainty of the unit–weight variance estimates from epoch to epoch will affect the computed success rates from different methods significantly, thus deserving more attentions in order to obtain useful success probability predictions.

Obesity screening for young Japanese males and females using skin fold measurements : the classification revisited

Anthropometric assessment is a simple, safe, and cost-efficient method to examine the health status of individu-als. The Japanese obesity classification based on the sum of two skin folds (Σ2SF) was proposed nearly 40 years ago therefore its applicability to Japanese living today is unknown. The current study aimed to determine Σ2SF cut-off values that correspond to percent body fat (%BF) and BMI values using two datasets from young Japa-nese adults (233 males and 139 females). Using regression analysis, Σ2SF and height-corrected Σ2SF (HtΣ2SF) values that correspond to %BF of 20, 25, and 30% for males and 30, 35, and 40% for females were determined. In addition, cut-off values of both Σ2SF and HtΣ2SF that correspond to BMI values of 23 kg/m2, 25 kg/m2 and 30 kg/m2 were determined. In comparison with the original Σ2SF values, the proposed values are smaller by about 10 mm at maximum. The proposed values show an improvement in sensitivity from about 25% to above 90% to identify individuals with ≥20% body fat in males and ≥30% body fat in females with high specificity of about 95% in both genders. The results indicate that the original Σ2SF cut-off values to screen obese individuals cannot be applied to young Japanese adults living today and modification is required. Application of the pro-posed values may assist screening in the clinical setting.

Human-body shadowing effects on indoor MIMO-OFDM channels at 5.2 GHz

Pedestrian movement is known to cause significant effects on indoor MIMO channels. In this paper, a statistical characterization of the indoor MIMO-OFDM channel subject ot pedestrian movement is reported. The experiment used 4 sending and 4 receiving antennas and 114 sub-carriers at 5.2 GHz. Measurement scenarios varied from zero to ten pedestrians walking randomly between transmitter (tx) and receiver (Rx) arrays. The empirical cumulative distribution function (CDF) of the received fading envelope fits the Ricean distribution with K factors ranging from 7dB to 15 dB, for the 10 pedestrians and vacant scenarios respectively. In general, as the number of pedestrians increase, the CDF slope tends to decrease proportionally. Furthermore, as the number of pedestrians increase, increasing multipath contribution, the dynamic range of channel capacity increases proportionally. These results are consistent with measurement results obtained in controlled scenarios for a fixed narrowband Single-Input Single-Output (SISO) link at 5.2 GHz in previous work. The described empirical characterization provides an insight into the prediction of human-body shadowing effects for indoor MIMO-OFDM channels at 5.2 GHz.

Expected values for pedometer-determined physical activity in older populations

The purpose of this review is to update expected values for pedometer-determined physical activity in free-living healthy older populations. A search of the literature published since 2001 began with a keyword (pedometer, "step counter," "step activity monitor" or "accelerometer AND steps/day") search of PubMed, Cumulative Index to Nursing & Allied Health Literature (CINAHL), SportDiscus, and PsychInfo. An iterative process was then undertaken to abstract and verify studies of pedometer-determined physical activity (captured in terms of steps taken; distance only was not accepted) in free-living adult populations described as ≥ 50 years of age (studies that included samples which spanned this threshold were not included unless they provided at least some appropriately age-stratified data) and not specifically recruited based on any chronic disease or disability. We identified 28 studies representing at least 1,343 males and 3,098 females ranging in age from 50–94 years. Eighteen (or 64%) of the studies clearly identified using a Yamax pedometer model. Monitoring frames ranged from 3 days to 1 year; the modal length of time was 7 days (17 studies, or 61%). Mean pedometer-determined physical activity ranged from 2,015 steps/day to 8,938 steps/day. In those studies reporting such data, consistent patterns emerged: males generally took more steps/day than similarly aged females, steps/day decreased across study-specific age groupings, and BMI-defined normal weight individuals took more steps/day than overweight/obese older adults. The range of 2,000–9,000 steps/day likely reflects the true variability of physical activity behaviors in older populations. More explicit patterns, for example sex- and age-specific relationships, remain to be informed by future research endeavors.

Xavier Herbert

As a biographical documentary concept develops, its intention and its form are impacted and may be transformed by market demands. The documentary idea about the life of Xavier Herbert has been in development through a number of iterations within the shifting landscape of the Australian documentary industry from the mid- 1990s to 2009. This study is, on the one hand, an endeavour to find a workable way to express and practise the multi-layered complexity of creative work, a long-form documentary script on Herbert, an Australian literary icon. On the other hand, this thesis represents a cumulative research exercise, whereby my own experiences in the documentary industry in Queensland, Australia and overseas are analysed in an effort to enlighten the broader documentary community about such a complex, even labyrinthine, process.

Microscopic modelling of freeway operations

Freeways are divided roadways designed to facilitate the uninterrupted movement of motor vehicles. However, many freeways now experience demand flows in excess of capacity, leading to recurrent congestion. The Highway Capacity Manual (TRB, 1994) uses empirical macroscopic relationships between speed, flow and density to quantify freeway operations and performance. Capacity may be predicted as the maximum uncongested flow achievable. Although they are effective tools for design and analysis, macroscopic models lack an understanding of the nature of processes taking place in the system. Szwed and Smith (1972, 1974) and Makigami and Matsuo (1990) have shown that microscopic modelling is also applicable to freeway operations. Such models facilitate an understanding of the processes whilst providing for the assessment of performance, through measures of capacity and delay. However, these models are limited to only a few circumstances. The aim of this study was to produce more comprehensive and practical microscopic models. These models were required to accurately portray the mechanisms of freeway operations at the specific locations under consideration. The models needed to be able to be calibrated using data acquired at these locations. The output of the models needed to be able to be validated with data acquired at these sites. Therefore, the outputs should be truly descriptive of the performance of the facility. A theoretical basis needed to underlie the form of these models, rather than empiricism, which is the case for the macroscopic models currently used. And the models needed to be adaptable to variable operating conditions, so that they may be applied, where possible, to other similar systems and facilities. It was not possible to produce a stand-alone model which is applicable to all facilities and locations, in this single study, however the scene has been set for the application of the models to a much broader range of operating conditions. Opportunities for further development of the models were identified, and procedures provided for the calibration and validation of the models to a wide range of conditions. The models developed, do however, have limitations in their applicability. Only uncongested operations were studied and represented. Driver behaviour in Brisbane was applied to the models. Different mechanisms are likely in other locations due to variability in road rules and driving cultures. Not all manoeuvres evident were modelled. Some unusual manoeuvres were considered unwarranted to model. However the models developed contain the principal processes of freeway operations, merging and lane changing. Gap acceptance theory was applied to these critical operations to assess freeway performance. Gap acceptance theory was found to be applicable to merging, however the major stream, the kerb lane traffic, exercises only a limited priority over the minor stream, the on-ramp traffic. Theory was established to account for this activity. Kerb lane drivers were also found to change to the median lane where possible, to assist coincident mergers. The net limited priority model accounts for this by predicting a reduced major stream flow rate, which excludes lane changers. Cowan's M3 model as calibrated for both streams. On-ramp and total upstream flow are required as input. Relationships between proportion of headways greater than 1 s and flow differed for on-ramps where traffic leaves signalised intersections and unsignalised intersections. Constant departure onramp metering was also modelled. Minimum follow-on times of 1 to 1.2 s were calibrated. Critical gaps were shown to lie between the minimum follow-on time, and the sum of the minimum follow-on time and the 1 s minimum headway. Limited priority capacity and other boundary relationships were established by Troutbeck (1995). The minimum average minor stream delay and corresponding proportion of drivers delayed were quantified theoretically in this study. A simulation model was constructed to predict intermediate minor and major stream delays across all minor and major stream flows. Pseudo-empirical relationships were established to predict average delays. Major stream average delays are limited to 0.5 s, insignificant compared with minor stream delay, which reach infinity at capacity. Minor stream delays were shown to be less when unsignalised intersections are located upstream of on-ramps than signalised intersections, and less still when ramp metering is installed. Smaller delays correspond to improved merge area performance. A more tangible performance measure, the distribution of distances required to merge, was established by including design speeds. This distribution can be measured to validate the model. Merging probabilities can be predicted for given taper lengths, a most useful performance measure. This model was also shown to be applicable to lane changing. Tolerable limits to merging probabilities require calibration. From these, practical capacities can be estimated. Further calibration is required of traffic inputs, critical gap and minimum follow-on time, for both merging and lane changing. A general relationship to predict proportion of drivers delayed requires development. These models can then be used to complement existing macroscopic models to assess performance, and provide further insight into the nature of operations.

Machine learning techniques to improve software quality

A significant proportion of the cost of software development is due to software testing and maintenance. This is in part the result of the inevitable imperfections due to human error, lack of quality during the design and coding of software, and the increasing need to reduce faults to improve customer satisfaction in a competitive marketplace. Given the cost and importance of removing errors improvements in fault detection and removal can be of significant benefit. The earlier in the development process faults can be found, the less it costs to correct them and the less likely other faults are to develop. This research aims to make the testing process more efficient and effective by identifying those software modules most likely to contain faults, allowing testing efforts to be carefully targeted. This is done with the use of machine learning algorithms which use examples of fault prone and not fault prone modules to develop predictive models of quality. In order to learn the numerical mapping between module and classification, a module is represented in terms of software metrics. A difficulty in this sort of problem is sourcing software engineering data of adequate quality. In this work, data is obtained from two sources, the NASA Metrics Data Program, and the open source Eclipse project. Feature selection before learning is applied, and in this area a number of different feature selection methods are applied to find which work best. Two machine learning algorithms are applied to the data - Naive Bayes and the Support Vector Machine - and predictive results are compared to those of previous efforts and found to be superior on selected data sets and comparable on others. In addition, a new classification method is proposed, Rank Sum, in which a ranking abstraction is laid over bin densities for each class, and a classification is determined based on the sum of ranks over features. A novel extension of this method is also described based on an observed polarising of points by class when rank sum is applied to training data to convert it into 2D rank sum space. SVM is applied to this transformed data to produce models the parameters of which can be set according to trade-off curves to obtain a particular performance trade-off.

Conceptualising use for information systems (IS) success

This thesis conceptualises Use for IS (Information Systems) success. While Use in this study describes the extent to which an IS is incorporated into the user’s processes or tasks, success of an IS is the measure of the degree to which the person using the system is better off. For IS success, the conceptualisation of Use offers new perspectives on describing and measuring Use. We test the philosophies of the conceptualisation using empirical evidence in an Enterprise Systems (ES) context. Results from the empirical analysis contribute insights to the existing body of knowledge on the role of Use and demonstrate Use as an important factor and measure of IS success. System Use is a central theme in IS research. For instance, Use is regarded as an important dimension of IS success. Despite its recognition, the Use dimension of IS success reportedly suffers from an all too simplistic definition, misconception, poor specification of its complex nature, and an inadequacy of measurement approaches (Bokhari 2005; DeLone and McLean 2003; Zigurs 1993). Given the above, Burton-Jones and Straub (2006) urge scholars to revisit the concept of system Use, consider a stronger theoretical treatment, and submit the construct to further validation in its intended nomological net. On those considerations, this study re-conceptualises Use for IS success. The new conceptualisation adopts a work-process system-centric lens and draws upon the characteristics of modern system types, key user groups and their information needs, and the incorporation of IS in work processes. With these characteristics, the definition of Use and how it may be measured is systematically established. Use is conceptualised as a second-order measurement construct determined by three sub-dimensions: attitude of its users, depth, and amount of Use. The construct is positioned in a modified IS success research model, in an attempt to demonstrate its central role in determining IS success in an ES setting. A two-stage mixed-methods research design—incorporating a sequential explanatory strategy—was adopted to collect empirical data and to test the research model. The first empirical investigation involved an experiment and a survey of ES end users at a leading tertiary education institute in Australia. The second, a qualitative investigation, involved a series of interviews with real-world operational managers in large Indian private-sector companies to canvass their day-to-day experiences with ES. The research strategy adopted has a stronger quantitative leaning. The survey analysis results demonstrate the aptness of Use as an antecedent and a consequence of IS success, and furthermore, as a mediator between the quality of IS and the impacts of IS on individuals. Qualitative data analysis on the other hand, is used to derive a framework for classifying the diversity of ES Use behaviour. The qualitative results establish that workers Use IS in their context to orientate, negotiate, or innovate. The implications are twofold. For research, this study contributes to cumulative IS success knowledge an approach for defining, contextualising, measuring, and validating Use. For practice, research findings not only provide insights for educators when incorporating ES for higher education, but also demonstrate how operational managers incorporate ES into their work practices. Research findings leave the way open for future, larger-scale research into how industry practitioners interact with an ES to complete their work in varied organisational environments.

A Stochastic View of Optimal Regret through Minimax Duality

We study the regret of optimal strategies for online convex optimization games. Using von Neumann's minimax theorem, we show that the optimal regret in this adversarial setting is closely related to the behavior of the empirical minimization algorithm in a stochastic process setting: it is equal to the maximum, over joint distributions of the adversary's action sequence, of the difference between a sum of minimal expected losses and the minimal empirical loss. We show that the optimal regret has a natural geometric interpretation, since it can be viewed as the gap in Jensen's inequality for a concave functional--the minimizer over the player's actions of expected loss--defined on a set of probability distributions. We use this expression to obtain upper and lower bounds on the regret of an optimal strategy for a variety of online learning problems. Our method provides upper bounds without the need to construct a learning algorithm; the lower bounds provide explicit optimal strategies for the adversary. Peter L. Bartlett, Alexander Rakhlin

Networks as a novel tool for studying team ball sports as complex social systems

This paper describes and evaluates the novel utility of network methods for understanding human interpersonal interactions within social neurobiological systems such as sports teams. We show how collective system networks are supported by the sum of interpersonal interactions that emerge from the activity of system agents (such as players in a sports team). To test this idea we trialled the methodology in analyses of intra-team collective behaviours in the team sport of water polo. We observed that the number of interactions between team members resulted in varied intra-team coordination patterns of play, differentiating between successful and unsuccessful performance outcomes. Future research on small-world networks methodologies needs to formalize measures of node connections in analyses of collective behaviours in sports teams, to verify whether a high frequency of interactions is needed between players in order to achieve competitive performance outcomes.

Cardiac health diagnosis using higher order spectra and support vector machine

The Electrocardiogram (ECG) is an important bio-signal representing the sum total of millions of cardiac cell depolarization potentials. It contains important insight into the state of health and nature of the disease afflicting the heart. Heart rate variability (HRV) refers to the regulation of the sinoatrial node, the natural pacemaker of the heart by the sympathetic and parasympathetic branches of the autonomic nervous system. The HRV signal can be used as a base signal to observe the heart's functioning. These signals are non-linear and non-stationary in nature. So, higher order spectral (HOS) analysis, which is more suitable for non-linear systems and is robust to noise, was used. An automated intelligent system for the identification of cardiac health is very useful in healthcare technology. In this work, we have extracted seven features from the heart rate signals using HOS and fed them to a support vector machine (SVM) for classification. Our performance evaluation protocol uses 330 subjects consisting of five different kinds of cardiac disease conditions. We demonstrate a sensitivity of 90% for the classifier with a specificity of 87.93%. Our system is ready to run on larger data sets.

Influence of culture conditions on the molecular signature of mesenchymal stem cells

Cell based therapies require cells capable of self renewal and differentiation, and a prerequisite is the ability to prepare an effective dose of ex vivo expanded cells for autologous transplants. The in vivo identification of a source of physiologically relevant cell types suitable for cell therapies is therefore an integral part of tissue engineering. Bone marrow is the most easily accessible source of mesenchymal stem cells (MSCs), and harbours two distinct populations of adult stem cells; namely hematopoietic stem cells (HSCs) and bone mesenchymal stem cells (BMSCs). Unlike HSCs, there are yet no rigorous criteria for characterizing BMSCs. Changing understanding about the pluripotency of BMSCs in recent studies has expanded their potential application; however, the underlying molecular pathways which impart the features distinctive to BMSCs remain elusive. Furthermore, the sparse in vivo distribution of these cells imposes a clear limitation to their in vitro study. Also, when BMSCs are cultured in vitro there is a loss of the in vivo microenvironment which results in a progressive decline in proliferation potential and multipotentiality. This is further exacerbated with increased passage number, characterized by the onset of senescence related changes. Accordingly, establishing protocols for generating large numbers of BMSCs without affecting their differentiation potential is necessary. The principal aims of this thesis were to identify potential molecular factors for characterizing BMSCs from osteoarthritic patients, and also to attempt to establish culture protocols favourable for generating large number of BMSCs, while at the same time retaining their proliferation and differentiation potential. Previously published studies concerning clonal cells have demonstrated that BMSCs are heterogeneous populations of cells at various stages of growth. Some cells are higher in the hierarchy and represent the progenitors, while other cells occupy a lower position in the hierarchy and are therefore more committed to a particular lineage. This feature of BMSCs was made evident by the work of Mareddy et al., which involved generating clonal populations of BMSCs from bone marrow of osteoarthritic patients, by a single cell clonal culture method. Proliferation potential and differentiation capabilities were used to group cells into fast growing and slow growing clones. The study presented here is a continuation of the work of Mareddy et al. and employed immunological and array based techniques to identify the primary molecular factors involved in regulating phenotypic characteristics exhibited by contrasting clonal populations. The subtractive immunization (SI) was used to generate novel antibodies against favourably expressed proteins in the fast growing clonal cell population. The difference between the clonal populations at the transcriptional level was determined using a Stem Cell RT2 Profiler TM PCR Array which focuses on stem cell pathway gene expression. Monoclonal antibodies (mAb) generated by SI were able to effectively highlight differentially expressed antigenic determinants, as was evident by Western blot analysis and confocal microscopy. Co-immunoprecipitation, followed by mass spectroscopy analysis, identified a favourably expressed protein as the cytoskeletal protein vimentin. The stem cell gene array highlighted genes that were highly upregulated in the fast growing clonal cell population. Based on their functions these genes were grouped into growth factors, cell fate determination and maintenance of embryonic and neural stem cell renewal. Furthermore, on a closer analysis it was established that the cytoskeletal protein vimentin and nine out of ten genes identified by gene array were associated with chondrogenesis or cartilage repair, consistent with the potential role played by BMSCs in defect repair and maintaining tissue homeostasis, by modulating the gene expression pattern to compensate for degenerated cartilage in osteoarthritic tissues. The gene array also presented transcripts for embryonic lineage markers such as FOXA2 and Sox2, both of which were significantly over expressed in fast growing clonal populations. A recent groundbreaking study by Yamanaka et al imparted embryonic stem cell (ESCs) -like characteristic to somatic cells in a process termed nuclear reprogramming, by the ectopic expression of the genes Sox2, cMyc and Oct4. The expression of embryonic lineage markers in adult stem cells may be a mechanism by which the favourable behaviour of fast growing clonal cells is determined and suggests a possible active phenomenon of spontaneous reprogramming in fast growing clonal cells. The expression pattern of these critical molecular markers could be indicative of the competence of BMSCs. For this reason, the expression pattern of Sox2, Oct4 and cMyc, at various passages in heterogeneous BMSCs population and tissue derived cells (osteoblasts and chondrocytes), was investigated by a real-time PCR and immunoflourescence staining. A strong nuclear staining was observed for Sox2, Oct4 and cMyc, which gradually weakened accompanied with cytoplasmic translocation after several passage. The mRNA and protein expression of Sox2, Oct4 and cMyc peaked at the third passage for osteoblasts, chondrocytes and third passage for BMSCs, and declined with each subsequent passage, indicating towards a possible mechanism of spontaneous reprogramming. This study proposes that the progressive decline in proliferation potential and multipotentiality associated with increased passaging of BMSCs in vitro might be a consequence of loss of these propluripotency factors. We therefore hypothesise that the expression of these master genes is not an intrinsic cell function, but rather an outcome of interaction of the cells with their microenvironment; this was evident by the fact that when removed from their in vivo microenvironment, BMSCs undergo a rapid loss of stemness after only a few passages. One of the most interesting aspects of this study was the integration of factors in the culture conditions, which to some extent, mimicked the in vivo microenvironmental niche of the BMSCs. A number of studies have successfully established that the cellular niche is not an inert tissue component but is of prime importance. The total sum of stimuli from the microenvironment underpins the complex interplay of regulatory mechanisms which control multiple functions in stem cells most importantly stem cell renewal. Therefore, well characterised factors which affect BMSCs characteristics, such as fibronectin (FN) coating, and morphogens such as FGF2 and BMP4, were incorporated into the cell culture conditions. The experimental set up was designed to provide insight into the expression pattern of the stem cell related transcription factors Sox2, cMyc and Oct4, in BMSCs with respect to passaging and changes in culture conditions. Induction of these pluripotency markers in somatic cells by retroviral transfection has been shown to confer pluripotency and an ESCs like state. Our study demonstrated that all treatments could transiently induce the expression of Sox2, cMyc and Oct4, and favourably affect the proliferation potential of BMSCs. The combined effect of these treatments was able to induce and retain the endogenous nuclear expression of stem cell transcription factors in BMSCs over an extended number of in vitro passages. Our results therefore suggest that the transient induction and manipulation of endogenous expression of transcription factors critical for stemness can be achieved by modulating the culture conditions; the benefit of which is to circumvent the need for genetic manipulations. In summary, this study has explored the role of BMSCs in the diseased state of osteoarthritis, by employing transcriptional profiling along with SI. In particular this study pioneered the use of primary cells for generating novel antibodies by SI. We established that somatic cells and BMSCs have a basal level of expression of pluripotency markers. Furthermore, our study indicates that intrinsic signalling mechanisms of BMSCs are intimately linked with extrinsic cues from the microenvironment and that these signals appear to be critical for retaining the expression of genes to maintain cell stemness in long term in vitro culture. This project provides a basis for developing an “artificial niche” required for reversion of commitment and maintenance of BMSC in their uncommitted homeostatic state.

The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Fast rates for estimation error and oracle inequalities for model section

We consider complexity penalization methods for model selection. These methods aim to choose a model to optimally trade off estimation and approximation errors by minimizing the sum of an empirical risk term and a complexity penalty. It is well known that if we use a bound on the maximal deviation between empirical and true risks as a complexity penalty, then the risk of our choice is no more than the approximation error plus twice the complexity penalty. There are many cases, however, where complexity penalties like this give loose upper bounds on the estimation error. In particular, if we choose a function from a suitably simple convex function class with a strictly convex loss function, then the estimation error (the difference between the risk of the empirical risk minimizer and the minimal risk in the class) approaches zero at a faster rate than the maximal deviation between empirical and true risks. In this paper, we address the question of whether it is possible to design a complexity penalized model selection method for these situations. We show that, provided the sequence of models is ordered by inclusion, in these cases we can use tight upper bounds on estimation error as a complexity penalty. Surprisingly, this is the case even in situations when the difference between the empirical risk and true risk (and indeed the error of any estimate of the approximation error) decreases much more slowly than the complexity penalty. We give an oracle inequality showing that the resulting model selection method chooses a function with risk no more than the approximation error plus a constant times the complexity penalty.