The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Neural network learning:theoretical foundations

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics

Overcoming the complexity of diagnostic problems due to sensor network architecture

In fault detection and diagnostics, limitations coming from the sensor network architecture are one of the main challenges in evaluating a system’s health status. Usually the design of the sensor network architecture is not solely based on diagnostic purposes, other factors like controls, financial constraints, and practical limitations are also involved. As a result, it quite common to have one sensor (or one set of sensors) monitoring the behaviour of two or more components. This can significantly extend the complexity of diagnostic problems. In this paper a systematic approach is presented to deal with such complexities. It is shown how the problem can be formulated as a Bayesian network based diagnostic mechanism with latent variables. The developed approach is also applied to the problem of fault diagnosis in HVAC systems, an application area with considerable modeling and measurement constraints.

Group maintenance scheduling : a case study for a pipeline network

This paper presents a group maintenance scheduling case study for a water distributed network. This water pipeline network presents the challenge of maintaining aging pipelines with the associated increases in annual maintenance costs. The case study focuses on developing an effective maintenance plan for the water utility. Current replacement planning is difficult as it needs to balance the replacement needs under limited budgets. A Maintenance Grouping Optimization (MGO) model based on a modified genetic algorithm was utilized to develop an optimum group maintenance schedule over a 20-year cycle. The adjacent geographical distribution of pipelines was used as a grouping criterion to control the searching space of the MGO model through a Judgment Matrix. Based on the optimum group maintenance schedule, the total cost was effectively reduced compared with the schedules without grouping maintenance jobs. This optimum result can be used as a guidance to optimize the current maintenance plan for the water utility.

Bricolage as a path to innovation for resource constrained new firms

Evidence suggests that both start-up and young firms (henceforth: new firms) – despite typically being resource-constrained – are sometimes able to innovate (Katila & Shane 2005). Such firms are seldom able to invest in expensive innovation processes, which suggests that they may rely on other pathways to innovation. In this paper, we test arguments that “bricolage,” defined as making do by applying combinations of the resources at hand to new problems and opportunities, provides a pathway to innovation for new firms. Our results suggest that variations in bricolage behaviors can provide an explanation of innovation under resource constraints by new firms.

Managerial cognition and reputation : does communication about firms’ intangibles affect performance?

Corporate reputation is viewed as fundamental to firm performance, growth and survival and the maintenance and enhancement of that reputation is a key responsibility of senior executives. However, relatively little is known about the main dimensions of corporate reputation and the amount of attention given to them by senior executives. Based on the corporate reputation and intangible resources literatures, thirteen reputational elements were identified and the amount of attention given to those elements in a large, longitudinal sample of annual reports from Australian firms was measured using computer aided text analysis. This identified five, main reputational dimensions that were both stable over time and related to firms’ future financial performance.

Why adopt now? Multiple case studies and survey studies comparing small, medium and large firms

While a number of factors have been highlighted in the innovation adoption literature, little is known about whether different factors are related to innovation adoption in differently-sized firms. We used preliminary case studies of small, medium and large firms to ground our hypotheses, which were then tested using a survey of 94 firms. We found that external stakeholder pressure and non-financial readiness were related to innovation adoption in SMEs; but that for large firms, adoption was related to the opportunity to innovate. It may be that the difficulties of adopting innovations, including both the financial cost and the effort involved, are too great for SMEs to overcome unless there is either a compelling need (external pressure) or enough in-house capability (non-financial readiness). This suggests that SMEs are more likely to have innovation “pushed” onto them while large firms are more likely to “pull” innovations when they have the opportunity.

Data preprocessing for anomaly based network intrusion detection : a review

Data preprocessing is widely recognized as an important stage in anomaly detection. This paper reviews the data preprocessing techniques used by anomaly-based network intrusion detection systems (NIDS), concentrating on which aspects of the network traffic are analyzed, and what feature construction and selection methods have been used. Motivation for the paper comes from the large impact data preprocessing has on the accuracy and capability of anomaly-based NIDS. The review finds that many NIDS limit their view of network traffic to the TCP/IP packet headers. Time-based statistics can be derived from these headers to detect network scans, network worm behavior, and denial of service attacks. A number of other NIDS perform deeper inspection of request packets to detect attacks against network services and network applications. More recent approaches analyze full service responses to detect attacks targeting clients. The review covers a wide range of NIDS, highlighting which classes of attack are detectable by each of these approaches. Data preprocessing is found to predominantly rely on expert domain knowledge for identifying the most relevant parts of network traffic and for constructing the initial candidate set of traffic features. On the other hand, automated methods have been widely used for feature extraction to reduce data dimensionality, and feature selection to find the most relevant subset of features from this candidate set. The review shows a trend toward deeper packet inspection to construct more relevant features through targeted content parsing. These context sensitive features are required to detect current attacks.

Using complex network metrics to predict the persistence of metapopulations with asymmetric connectivity patterns

Almost all metapopulation modelling assumes that connectivity between patches is only a function of distance, and is therefore symmetric. However, connectivity will not depend only on the distance between the patches, as some paths are easy to traverse, while others are difficult. When colonising organisms interact with the heterogeneous landscape between patches, connectivity patterns will invariably be asymmetric. There have been few attempts to theoretically assess the effects of asymmetric connectivity patterns on the dynamics of metapopulations. In this paper, we use the framework of complex networks to investigate whether metapopulation dynamics can be determined by directly analysing the asymmetric connectivity patterns that link the patches. Our analyses focus on “patch occupancy” metapopulation models, which only consider whether a patch is occupied or not. We propose three easily calculated network metrics: the “asymmetry” and “average path strength” of the connectivity pattern, and the “centrality” of each patch. Together, these metrics can be used to predict the length of time a metapopulation is expected to persist, and the relative contribution of each patch to a metapopulation’s viability. Our results clearly demonstrate the negative effect that asymmetry has on metapopulation persistence. Complex network analyses represent a useful new tool for understanding the dynamics of species existing in fragmented landscapes, particularly those existing in large metapopulations.

HR practices and challenges in Chinese firms : comparison with Western firms

This research compares Chinese HRM with Western HRM, particularly in the areas of development of HR information systems (HRIS) and HR measurement systems and their relation to HR’s involvement as a strategic partner in firms. The research uses a 3-stage model of HRIS (workforce profiling, business insight, and strategic driver) based on studies of Irmer and Ellerby (2005) and Boudreau and Ramstad (2003) to compare the relative stages of development of Chinese and Western HRM. The quantitative aspect of the study comprises a survey of senior HR practitioners from 171 Chinese firms whose data is compared with data from Irmer and Ellerby’s study of Australian and U.S. HRM (2005) and Lawler et al’s series of studies of U.S firms (1995, 1998, 2001, 2004). The main results of the comparison are that Chinese HRM generally lags behind Western HRM. In particular, Chinese HR professionals allocate less time to strategic activities and their roles are less strategic than those of Western HR professionals. The HR measurement systems of Chinese firms are more limited in function, and the HR information systems of Chinese companies are less automated and integrated. However there is also evidence of a “two speed” HR system in China with a small proportion of firms having highly sophisticated HR systems but with a much larger proportion of Chinese firms than in the West having only the most basic HR information systems. This ‘two speed” system is in part attributable to a split between the relatively advanced HR systems of large State Owned Enterprises and the basic systems that predominate in smaller, growing Local Private firms. The survey study is complemented by a series of interviews with a number of senior Chinese HR practitioners who provide richer insights into their experiences and the challenges they face in contemporary Chinese firms.