Neural network learning:theoretical foundations

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics

Aerial image analysis using spiking neural networks with application to power line corridor monitoring

Trees, shrubs and other vegetation are of continued importance to the environment and our daily life. They provide shade around our roads and houses, offer a habitat for birds and wildlife, and absorb air pollutants. However, vegetation touching power lines is a risk to public safety and the environment, and one of the main causes of power supply problems. Vegetation management, which includes tree trimming and vegetation control, is a significant cost component of the maintenance of electrical infrastructure. For example, Ergon Energy, the Australia’s largest geographic footprint energy distributor, currently spends over $80 million a year inspecting and managing vegetation that encroach on power line assets. Currently, most vegetation management programs for distribution systems are calendar-based ground patrol. However, calendar-based inspection by linesman is labour-intensive, time consuming and expensive. It also results in some zones being trimmed more frequently than needed and others not cut often enough. Moreover, it’s seldom practicable to measure all the plants around power line corridors by field methods. Remote sensing data captured from airborne sensors has great potential in assisting vegetation management in power line corridors. This thesis presented a comprehensive study on using spiking neural networks in a specific image analysis application: power line corridor monitoring. Theoretically, the thesis focuses on a biologically inspired spiking cortical model: pulse coupled neural network (PCNN). The original PCNN model was simplified in order to better analyze the pulse dynamics and control the performance. Some new and effective algorithms were developed based on the proposed spiking cortical model for object detection, image segmentation and invariant feature extraction. The developed algorithms were evaluated in a number of experiments using real image data collected from our flight trails. The experimental results demonstrated the effectiveness and advantages of spiking neural networks in image processing tasks. Operationally, the knowledge gained from this research project offers a good reference to our industry partner (i.e. Ergon Energy) and other energy utilities who wants to improve their vegetation management activities. The novel approaches described in this thesis showed the potential of using the cutting edge sensor technologies and intelligent computing techniques in improve power line corridor monitoring. The lessons learnt from this project are also expected to increase the confidence of energy companies to move from traditional vegetation management strategy to a more automated, accurate and cost-effective solution using aerial remote sensing techniques.

Assessment of retinal structure and visual function in association with diabetic peripheral neuropathy

Diabetes is an increasingly prevalent disease worldwide. Providing early management of the complications can prevent morbidity and mortality in this population. Peripheral neuropathy, a significant complication of diabetes, is the major cause of foot ulceration and amputation in diabetes. Delay in attending to complication of the disease contributes to significant medical expenses for diabetic patients and the community. Early structural changes to the neural components of the retina have been demonstrated to occur prior to the clinically visible retinal vasculature complication of diabetic retinopathy. Additionally visual functionloss has been shown to exist before the ophthalmoscopic manifestations of vasculature damage. The purpose of this thesis was to evaluate the relationship between diabetic peripheral neuropathy and both retinal structure and visual function. The key question was whether diabetic peripheral neuropathy is the potential underlying factor responsible for retinal anatomical change and visual functional loss in people with diabetes. This study was conducted on a cohort with type 2 diabetes. Retinal nerve fibre layer thickness was assessed by means of Optical Coherence Tomography (OCT). Visual function was assessed using two different methods; Standard Automated Perimetry (SAP) and flicker perimetry were performed within the central 30 degrees of fixation. The level of diabetic peripheral neuropathy (DPN) was assessed using two techniques - Quantitative Sensory Testing and Neuropathy Disability Score (NDS). These techniques are known to be capable of detecting DPN at very early stages. NDS has also been shown as a gold standard for detecting 'risk of foot ulceration'. Findings reported in this thesis showed that RNFL thickness, particularly in the inferior quadrant, has a significant association with severity of DPN when the condition has been assessed using NDS. More specifically it was observed that inferior RNFL thickness has the ability to differentiate individuals who are at higher risk of foot ulceration from those who are at lower risk, indicating that RNFL thickness can predict late-staged DPN. Investigating the association between RNFL and QST did not show any meaningful interaction, which indicates that RNFL thickness for this cohort was not as predictive of neuropathy status as NDS. In both of these studies, control participants did not have different results from the type 2 cohort who did not DPN suggesting that RNFL thickness is not a marker for diagnosing DPN at early stages. The latter finding also indicated that diabetes per se, is unlikely to affect the RNFL thickness. Visual function as measured by SAP and flicker perimetry was found to be associated with severity of peripheral neuropathy as measured by NDS. These findings were also capable of differentiating individuals at higher risk of foot ulceration; however, visual function also proved not to be a maker for early diagnosis of DPN. It was found that neither SAP, nor flicker sensitivity have meaningful associations with DPN when neuropathy status was measured using QST. Importantly diabetic retinopathy did not explain any of the findings in these experiments. The work described here is valuable as no other research to date has investigated the association between diabetic peripheral neuropathy and either retinal structure or visual function.

Frequency response function based structural damage detection using artificial neural networks

Damage detection in structures has become increasingly important in recent years. While a number of damage detection and localization methods have been proposed, few attempts have been made to explore the structure damage with frequency response functions (FRFs). This paper illustrates the damage identification and condition assessment of a beam structure using a new frequency response functions (FRFs) based damage index and Artificial Neural Networks (ANNs). In practice, usage of all available FRF data as an input to artificial neural networks makes the training and convergence impossible. Therefore one of the data reduction techniques Principal Component Analysis (PCA) is introduced in the algorithm. In the proposed procedure, a large set of FRFs are divided into sub-sets in order to find the damage indices for different frequency points of different damage scenarios. The basic idea of this method is to establish features of damaged structure using FRFs from different measurement points of different sub-sets of intact structure. Then using these features, damage indices of different damage cases of the structure are identified after reconstructing of available FRF data using PCA. The obtained damage indices corresponding to different damage locations and severities are introduced as input variable to developed artificial neural networks. Finally, the effectiveness of the proposed method is illustrated and validated by using the finite element modal of a beam structure. The illustrated results show that the PCA based damage index is suitable and effective for structural damage detection and condition assessment of building structures.

Neural interface systems : a new legal frontier?

In an age where digital innovation knows no boundaries, research in the area of brain-computer interface and other neural interface devices go where none have gone before. The possibilities are endless and as dreams become reality, the implications of these amazing developments should be considered. Some of these new devices have been created to correct or minimise the effects of disease or injury so the paper discusses some of the current research and development in the area, including neuroprosthetics. To assist researchers and academics in identifying some of the legal and ethical issues that might arise as a result of research and development of neural interface devices, using both non-invasive techniques and invasive procedures, the paper discusses a number of recent observations of authors in the field. The issue of enhancing human attributes by incorporating these new devices is also considered. Such enhancement may be regarded as freeing the mind from the constraints of the body, but there are legal and moral issues that researchers and academics would be well advised to contemplate as these new devices are developed and used. While different fact situation surround each of these new devices, and those that are yet to come, consideration of the legal and ethical landscape may assist researchers and academics in dealing effectively with matters that arise in these times of transition. Lawyers could seek to facilitate the resolution of the legal disputes that arise in this area of research and development within the existing judicial and legislative frameworks. Whether these frameworks will suffice, or will need to change in order to enable effective resolution, is a broader question to be explored.

Neural differentiation potential of human bone marrow-derived mesenchymal stromal cells: misleading marker gene expression

Background In contrast to pluripotent embryonic stem cells, adult stem cells have been considered to be multipotent, being somewhat more restricted in their differentiation capacity and only giving rise to cell types related to their tissue of origin. Several studies, however, have reported that bone marrow-derived mesenchymal stromal cells (MSCs) are capable of transdifferentiating to neural cell types, effectively crossing normal lineage restriction boundaries. Such reports have been based on the detection of neural-related proteins by the differentiated MSCs. In order to assess the potential of human adult MSCs to undergo true differentiation to a neural lineage and to determine the degree of homogeneity between donor samples, we have used RT-PCR and immunocytochemistry to investigate the basal expression of a range of neural related mRNAs and proteins in populations of non-differentiated MSCs obtained from 4 donors. Results The expression analysis revealed that several of the commonly used marker genes from other studies like nestin, Enolase2 and microtubule associated protein 1b (MAP1b) are already expressed by undifferentiated human MSCs. Furthermore, mRNA for some of the neural-related transcription factors, e.g. Engrailed-1 and Nurr1 were also strongly expressed. However, several other neural-related mRNAs (e.g. DRD2, enolase2, NFL and MBP) could be identified, but not in all donor samples. Similarly, synaptic vesicle-related mRNA, STX1A could only be detected in 2 of the 4 undifferentiated donor hMSC samples. More significantly, each donor sample revealed a unique expression pattern, demonstrating a significant variation of marker expression. Conclusion The present study highlights the existence of an inter-donor variability of expression of neural-related markers in human MSC samples that has not previously been described. This donor-related heterogeneity might influence the reproducibility of transdifferentiation protocols as well as contributing to the ongoing controversy about differentiation capacities of MSCs. Therefore, further studies need to consider the differences between donor samples prior to any treatment as well as the possibility of harvesting donor cells that may be inappropriate for transplantation strategies.

Human neural cell interactions with orientated electrospun nanofibers in vitro

Aim: Electrospun nanofibers represent potent guidance substrates for nervous tissue repair. Development of nanofiber-based scaffolds for CNS repair requires, as a first step, an understanding of appropriate neural cell type-substrate interactions. Materials & methods: Astrocyte–nanofiber interactions (e.g., adhesion, proliferation, process extension and migration) were studied by comparing human neural progenitor-derived astrocytes (hNP-ACs) and a human astrocytoma cell line (U373) with aligned polycaprolactone (PCL) nanofibers or blended (25% type I collagen/75% PCL) nanofibers. Neuron–nanofiber interactions were assessed using a differentiated human neuroblastoma cell line (SH-SY5Y). Results & discussion: U373 cells and hNP-AC showed similar process alignment and length when associated with PCL or Type I collagen/PCL nanofibers. Cell adhesion and migration by hNP-AC were clearly improved by functionalization of nanofiber surfaces with type I collagen. Functionalized nanofibers had no such effect on U373 cells. Another clear difference between the U373 cells and hNP-AC interactions with the nanofiber substrate was proliferation; the cell line demonstrating strong proliferation, whereas the hNP-AC line showed no proliferation on either type of nanofiber. Long axonal growth (up to 600 µm in length) of SH-SY5Y neurons followed the orientation of both types of nanofibers even though adhesion of the processes to the fibers was poor. Conclusion: The use of cell lines is of only limited predictive value when studying cell–substrate interactions but both morphology and alignment of human astrocytes were affected profoundly by nanofibers. Nanofiber surface functionalization with collagen significantly improved hNP-AC adhesion and migration. Alternative forms of functionalization may be required for optimal axon–nanofiber interactions.

Using strategic movement to calibrate a neural compass : a spiking network for tracking head direction in rats and robots

The head direction (HD) system in mammals contains neurons that fire to represent the direction the animal is facing in its environment. The ability of these cells to reliably track head direction even after the removal of external sensory cues implies that the HD system is calibrated to function effectively using just internal (proprioceptive and vestibular) inputs. Rat pups and other infant mammals display stereotypical warm-up movements prior to locomotion in novel environments, and similar warm-up movements are seen in adult mammals with certain brain lesion-induced motor impairments. In this study we propose that synaptic learning mechanisms, in conjunction with appropriate movement strategies based on warm-up movements, can calibrate the HD system so that it functions effectively even in darkness. To examine the link between physical embodiment and neural control, and to determine that the system is robust to real-world phenomena, we implemented the synaptic mechanisms in a spiking neural network and tested it on a mobile robot platform. Results show that the combination of the synaptic learning mechanisms and warm-up movements are able to reliably calibrate the HD system so that it accurately tracks real-world head direction, and that calibration breaks down in systematic ways if certain movements are omitted. This work confirms that targeted, embodied behaviour can be used to calibrate neural systems, demonstrates that ‘grounding’ of modeled biological processes in the real world can reveal underlying functional principles (supporting the importance of robotics to biology), and proposes a functional role for stereotypical behaviours seen in infant mammals and those animals with certain motor deficits. We conjecture that these calibration principles may extend to the calibration of other neural systems involved in motion tracking and the representation of space, such as grid cells in entorhinal cortex.

Quo Vadis? Reliable and Practical Rule Extraction from Neural Networks

Rule extraction from neural network algorithms have been investigated for two decades and there have been significant applications. Despite this level of success, rule extraction from neural network methods are generally not part of data mining tools, and a significant commercial breakthrough may still be some time away. This paper briefly reviews the state-of-the-art and points to some of the obstacles, namely a lack of evaluation techniques in experiments and larger benchmark data sets. A significant new development is the view that rule extraction from neural networks is an interactive process which actively involves the user. This leads to the application of assessment and evaluation techniques from information retrieval which may lead to a range of new methods.

Structural damage detection using frequency response functions and neural networks

This paper illustrates the damage identification and condition assessment of a three story bookshelf structure using a new frequency response functions (FRFs) based damage index and Artificial Neural Networks (ANNs). A major obstacle of using measured frequency response function data is a large size input variables to ANNs. This problem is overcome by applying a data reduction technique called principal component analysis (PCA). In the proposed procedure, ANNs with their powerful pattern recognition and classification ability were used to extract damage information such as damage locations and severities from measured FRFs. Therefore, simple neural network models are developed, trained by Back Propagation (BP), to associate the FRFs with the damage or undamaged locations and severity of the damage of the structure. Finally, the effectiveness of the proposed method is illustrated and validated by using the real data provided by the Los Alamos National Laboratory, USA. The illustrated results show that the PCA based artificial Neural Network method is suitable and effective for damage identification and condition assessment of building structures. In addition, it is clearly demonstrated that the accuracy of proposed damage detection method can also be improved by increasing number of baseline datasets and number of principal components of the baseline dataset.

Early local functional changes in the human diabetic retina : a global flash multifocal electroretinogram study

Purpose: To investigate early functional changes of local retinal defects in type II diabetic patients using the global flash multifocal electroretinogram (MOFO mfERG). Methods: Thirty-eight diabetic patients and 14 age-matched controls were recruited. Nine of the diabetics were free from diabetic retinopathy (DR), while the remainder had mild to moderate non-proliferative diabetic retinopathy. The MOFO mfERG was performed at high (98%) and low (46%) contrast levels. MfERG responses were grouped into 35 regions for comparison with DR classification at those locations. Z-scores of the regional mfERG responses were compared across different types of DR defects. Results: The mfERG waveform consisted of the direct component (DC) and the induced component (IC). Local reduction in DC and IC amplitudes were found in diabetic patients with and without DR. With increasing severity of retinopathy, there was a further deterioration in amplitude of both components. Under MOFO mfERG paradigm, amplitude was a useful screening parameter. Conclusion: The MOFO mfERG can help in detecting early functional anomalies before the appearance of visible signs, and may assist in monitoring further functional deterioration in diabetic patients.