The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Neural network learning:theoretical foundations

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics

Evaluating effects of eco-driving at traffic intersections based on traffic micro-simulation

Eco-driving is an initiative driving behavior which aims to reduce fuel consumption and emissions from automobiles. Recently, it has attracted increasing interests and has been adopted by many drivers in Australia. Although many of the studies have revealed considerable benefits in terms of fuel consumption and emissions after utilising eco-driving, most of the literature investigated eco-driving effects on individual driver but not traffic flow. The driving behavior of eco-drivers will potentially affect other drivers and thereby affects the entire traffic flow. To comprehensively assess and understand how effectively eco-driving can perform, therefore, measurement on traffic flow is necessary. In this paper, we proposed and demonstrated an evaluation method based on a microscopic traffic simulator (Aimsun). We focus on one particular eco-driving style which involves moderate and smooth acceleration. We evaluated both traffic performance (travel time) and environmental performance (fuel consumption and CO2 emission) at traffic intersection level in a simple simulation model. The before-and-after comparisons indicated potentially negative impacts when using eco-driving, which highlighted the necessity to carefully evaluate and improve eco-driving before wide promotion and implementation.

Aerial image analysis using spiking neural networks with application to power line corridor monitoring

Trees, shrubs and other vegetation are of continued importance to the environment and our daily life. They provide shade around our roads and houses, offer a habitat for birds and wildlife, and absorb air pollutants. However, vegetation touching power lines is a risk to public safety and the environment, and one of the main causes of power supply problems. Vegetation management, which includes tree trimming and vegetation control, is a significant cost component of the maintenance of electrical infrastructure. For example, Ergon Energy, the Australia’s largest geographic footprint energy distributor, currently spends over $80 million a year inspecting and managing vegetation that encroach on power line assets. Currently, most vegetation management programs for distribution systems are calendar-based ground patrol. However, calendar-based inspection by linesman is labour-intensive, time consuming and expensive. It also results in some zones being trimmed more frequently than needed and others not cut often enough. Moreover, it’s seldom practicable to measure all the plants around power line corridors by field methods. Remote sensing data captured from airborne sensors has great potential in assisting vegetation management in power line corridors. This thesis presented a comprehensive study on using spiking neural networks in a specific image analysis application: power line corridor monitoring. Theoretically, the thesis focuses on a biologically inspired spiking cortical model: pulse coupled neural network (PCNN). The original PCNN model was simplified in order to better analyze the pulse dynamics and control the performance. Some new and effective algorithms were developed based on the proposed spiking cortical model for object detection, image segmentation and invariant feature extraction. The developed algorithms were evaluated in a number of experiments using real image data collected from our flight trails. The experimental results demonstrated the effectiveness and advantages of spiking neural networks in image processing tasks. Operationally, the knowledge gained from this research project offers a good reference to our industry partner (i.e. Ergon Energy) and other energy utilities who wants to improve their vegetation management activities. The novel approaches described in this thesis showed the potential of using the cutting edge sensor technologies and intelligent computing techniques in improve power line corridor monitoring. The lessons learnt from this project are also expected to increase the confidence of energy companies to move from traditional vegetation management strategy to a more automated, accurate and cost-effective solution using aerial remote sensing techniques.

Frequency response function based structural damage detection using artificial neural networks

Damage detection in structures has become increasingly important in recent years. While a number of damage detection and localization methods have been proposed, few attempts have been made to explore the structure damage with frequency response functions (FRFs). This paper illustrates the damage identification and condition assessment of a beam structure using a new frequency response functions (FRFs) based damage index and Artificial Neural Networks (ANNs). In practice, usage of all available FRF data as an input to artificial neural networks makes the training and convergence impossible. Therefore one of the data reduction techniques Principal Component Analysis (PCA) is introduced in the algorithm. In the proposed procedure, a large set of FRFs are divided into sub-sets in order to find the damage indices for different frequency points of different damage scenarios. The basic idea of this method is to establish features of damaged structure using FRFs from different measurement points of different sub-sets of intact structure. Then using these features, damage indices of different damage cases of the structure are identified after reconstructing of available FRF data using PCA. The obtained damage indices corresponding to different damage locations and severities are introduced as input variable to developed artificial neural networks. Finally, the effectiveness of the proposed method is illustrated and validated by using the finite element modal of a beam structure. The illustrated results show that the PCA based damage index is suitable and effective for structural damage detection and condition assessment of building structures.

Deformation behaviours of coarse-grained and nanocrystalline Mg-5wt% Al alloys

Magnesium alloys have been of growing interest to various engineering applications, such as the automobile, aerospace, communication and computer industries due to their low density, high specific strength, good machineability and availability as compared with other structural materials. However, most Mg alloys suffer from poor plasticity due to their Hexagonal Close Packed structure. Grain refinement has been proved to be an effective method to enhance the strength and alter the ductility of the materials. Several methods have been proposed to produce materials with nanocrystalline grain structures. So far, most of the research work on nanocrystalline materials has been carried out on Face-Centered Cubic and Body-Centered Cubic metals. However, there has been little investigation of nanocrystalline Mg alloys. In this study, bulk coarse-grained and nanocrystalline Mg alloys were fabricated by a mechanical alloying method. The mixed powder of Mg chips and Al powder was mechanically milled under argon atmosphere for different durations of 0 hours (MA0), 10 hours (MA10), 20 hours (MA20), 30 hours (MA30) and 40 hours (MA40), followed by compaction and sintering. Then the sintered billets were hot-extruded into metallic rods with a 7 mm diameter. The obtained Mg alloys have a nominal composition of Mg–5wt% Al, with grain sizes ranging from 13 μm down to 50 nm, depending on the milling durations. The microstructure characterization and evolution after deformation were carried out by means of Optical microscopy, X-Ray Diffraction, Scanning Electron Microscopy, Transmission Electron Microscopy, Scanning Probe Microscopy and Neutron Diffraction techniques. Nanoindentaion, compression and micro-compression tests on micro-pillars were used to study the size effects on the mechanical behaviour of the Mg alloys. Two kinds of size effects on the mechanical behaviours and deformation mechanisms were investigated: grain size effect and sample size effect. The nanoindentation tests were composed of constant strain rate, constant loading rate and indentation creep tests. The normally reported indentation size effect in single crystal and coarse-grained crystals was observed in both the coarse-grained and nanocrystalline Mg alloys. Since the indentation size effect is correlated to the Geometrically Necessary Dislocations under the indenter to accommodate the plastic deformation, the good agreement between the experimental results and the Indentation Size Effect model indicated that, in the current nanocrystalline MA20 and MA30, the dislocation plasticity was still the dominant deformation mechanism. Significant hardness enhancement with decreasing grain size, down to 58 nm, was found in the nanocrystalline Mg alloys. Further reduction of grain size would lead to a drop in the hardness values. The failure of grain refinement strengthening with the relatively high strain rate sensitivity of nanocrystalline Mg alloys suggested a change in the deformation mechanism. Indentation creep tests showed that the stress exponent was dependent on the loading rate during the loading section of the indentation, which was related to the dislocation structures before the creep starts. The influence of grain size on the mechanical behaviour and strength of extruded coarse-grained and nanocrystalline Mg alloys were investigated using uniaxial compression tests. The macroscopic response of the Mg alloys transited from strain hardening to strain softening behaviour, with grain size reduced from 13 ìm to 50 nm. The strain hardening was related to the twinning induced hardening and dislocation hardening effect, while the strain softening was attributed to the localized deformation in the nanocrystalline grains. The tension–compression yield asymmetry was noticed in the nanocrystalline region, demonstrating the twinning effect in the ultra-fine-grained and nanocrystalline region. The relationship k tensions < k compression failed in the nanocrystalline Mg alloys; this was attributed to the twofold effect of grain size on twinning. The nanocrystalline Mg alloys were found to exhibit increased strain rate sensitivity with decreasing grain size, with strain rate ranging from 0.0001/s to 0.01/s. Strain rate sensitivity of coarse-grained MA0 was increased by more than 10 times in MA40. The Hall-Petch relationship broke down at a critical grain size in the nanocrystalline region. The breakdown of the Hall-Petch relationship and the increased strain rate sensitivity were due to the localized dislocation activities (generalization and annihilation at grain boundaries) and the more significant contribution from grain boundary mediated mechanisms. In the micro-compression tests, the sample size effects on the mechanical behaviours were studied on MA0, MA20 and MA40 micro-pillars. In contrast to the bulk samples under compression, the stress-strain curves of MA0 and MA20 micro-pillars were characterized with a number of discrete strain burst events separated by nearly elastic strain segments. Unlike MA0 and MA20, the stress-strain curves of MA40 micro-pillars were smooth, without obvious strain bursts. The deformation mechanisms of the MA0 and MA20 micro-pillars under micro-compression tests were considered to be initially dominated by deformation twinning, followed by dislocation mechanisms. For MA40 pillars, the deformation mechanisms were believed to be localized dislocation activities and grain boundary related mechanisms. The strain hardening behaviours of the micro-pillars suggested that the grain boundaries in the nanocrystalline micro-pillars would reduce the source (nucleation sources for twins/dislocations) starvation hardening effect. The power law relationship of the yield strength on pillar dimensions in MA0, MA20 supported the fact that the twinning mechanism was correlated to the pre-existing defects, which can promote the nucleation of the twins. Then, we provided a latitudinal comparison of the results and conclusions derived from the different techniques used for testing the coarse-grained and nanocrystalline Mg alloy; this helps to better understand the deformation mechanisms of the Mg alloys as a whole. At the end, we summarized the thesis and highlighted the conclusions, contributions, innovations and outcomes of the research. Finally, it outlined recommendations for future work.

Neural interface systems : a new legal frontier?

In an age where digital innovation knows no boundaries, research in the area of brain-computer interface and other neural interface devices go where none have gone before. The possibilities are endless and as dreams become reality, the implications of these amazing developments should be considered. Some of these new devices have been created to correct or minimise the effects of disease or injury so the paper discusses some of the current research and development in the area, including neuroprosthetics. To assist researchers and academics in identifying some of the legal and ethical issues that might arise as a result of research and development of neural interface devices, using both non-invasive techniques and invasive procedures, the paper discusses a number of recent observations of authors in the field. The issue of enhancing human attributes by incorporating these new devices is also considered. Such enhancement may be regarded as freeing the mind from the constraints of the body, but there are legal and moral issues that researchers and academics would be well advised to contemplate as these new devices are developed and used. While different fact situation surround each of these new devices, and those that are yet to come, consideration of the legal and ethical landscape may assist researchers and academics in dealing effectively with matters that arise in these times of transition. Lawyers could seek to facilitate the resolution of the legal disputes that arise in this area of research and development within the existing judicial and legislative frameworks. Whether these frameworks will suffice, or will need to change in order to enable effective resolution, is a broader question to be explored.