The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Public sector strategy classification system for shared service identification

The purpose of this conceptual paper is to address the lack of consistent means through which strategies are identified and discussed across theoretical perspectives in the field of business strategy. A standardised referencing system is offered to codify the means by which strategies can be identified, from which new business services and information systems may be derived. This taxonomy was developed using qualitative content analysis study of government agencies’ strategic plans. This taxonomy is useful for identifying strategy formation and determining gaps and opportunities. Managers will benefit from a more transparent strategic design process that reduces ambiguity, aids in identifying and correcting gaps in strategy formulation, and fosters enhanced strategic analysis. Key benefits to academics are the improved dialogue in strategic management field and suggest that progress in the field requires that fundamentals of strategy formulation and classification be considered more carefully. Finally, the formalization of strategy can lead to the clear identification of new business services, which inform ICT investment decisions and shared service prioritisation.

Interaction of contractile activity and training history on mRNA abundance in skeletal muscle from trained athletes

Skeletal muscle displays enormous plasticity to respond to contractile activity with muscle from strength- (ST) and endurance-trained (ET) athletes representing diverse states of the adaptation continuum. Training adaptation can be viewed as the accumulation of specific proteins. Hence, the altered gene expression that allows for changes in protein concentration is of major importance for any training adaptation. Accordingly, the aim of the present study was to quantify acute subcellular responses in muscle to habitual and unfamiliar exercise. After 24-h diet/exercise control, 13 male subjects (7 ST and 6 ET) performed a random order of either resistance (8 × 5 maximal leg extensions) or endurance exercise (1 h of cycling at 70% peak O2 uptake). Muscle biopsies were taken from vastus lateralis at rest and 3 h after exercise. Gene expression was analyzed using real-time PCR with changes normalized relative to preexercise values. After cycling exercise, peroxisome proliferator-activated receptor-γ coactivator-1α (ET ∼8.5-fold, ST ∼10-fold, P < 0.001), pyruvate dehydrogenase kinase-4 (PDK-4; ET ∼26-fold, ST ∼39-fold), vascular endothelial growth factor (VEGF; ET ∼4.5-fold, ST ∼4-fold), and muscle atrophy F-box protein (MAFbx) (ET ∼2-fold, ST ∼0.4-fold) mRNA increased in both groups, whereas MyoD (∼3-fold), myogenin (∼0.9-fold), and myostatin (∼2-fold) mRNA increased in ET but not in ST (P < 0.05). After resistance exercise PDK-4 (∼7-fold, P < 0.01) and MyoD (∼0.7-fold) increased, whereas MAFbx (∼0.7-fold) and myostatin (∼0.6-fold) decreased in ET but not in ST. We conclude that prior training history can modify the acute gene responses in skeletal muscle to subsequent exercise.

The design and development of an undergraduate signal processing laboratory

The School of Electrical and Electronic Systems Engineering of Queensland University of Technology (like many other universities around the world) has recognised the importance of complementing the teaching of signal processing with computer based experiments. A laboratory has been developed to provide a "hands-on" approach to the teaching of signal processing techniques. The motivation for the development of this laboratory was the cliche "What I hear I remember but what I do I understand." The laboratory has been named as the "Signal Computing and Real-time DSP Laboratory" and provides practical training to approximately 150 final year undergraduate students each year. The paper describes the novel features of the laboratory, techniques used in the laboratory based teaching, interesting aspects of the experiments that have been developed and student evaluation of the teaching techniques