The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

A new control-volume finite-element scheme for heterogeneous porous media : application to the drying of softwood

An improved mesoscopic model is presented for simulating the drying of porous media. The aim of this model is to account for two scales simultaneously: the scale of the whole product and the scale of the heterogeneities of the porous medium. The innovation of this method is the utilization of a new mass-conservative scheme based on the Control-Volume Finite-Element (CV-FE) method that partitions the moisture content field over the individual sub-control volumes surrounding each node within the mesh. Although the new formulation has potential for application across a wide range of transport processes in heterogeneous porous media, the focus here is on applying the model to the drying of small sections of softwood consisting of several growth rings. The results conclude that, when compared to a previously published scheme, only the new mass-conservative formulation correctly captures the true moisture content evolution in the earlywood and latewood components of the growth rings during drying.

Experimental and finite element analysis of a double strap joint between steel plates and normal modulus CFRP

Strengthening of steel structures using externally-bonded carbon fibre reinforced polymers ‘CFRP’ is a rapidly developing technique. This paper describes the behaviour of axially loaded flat steel plates strengthened using carbon fibre reinforced polymer sheets. Two steel plates were joined together with adhesive and followed by the application of carbon fibre sheet double strap joint with different bond lengths. The behaviour of the specimens was further investigated by using nonlinear finite element analysis to predict the failure modes and load capacity. In this study, bond failure is the dominant failure mode for normal modulus (240 GPa) CFRP bonding which closely matched the results of finite elements. The predicted ultimate loads from the FE analysis are found to be in good agreement with experimental values.