Alleviating ‘overfitting’ via genetically-regularised neural network

A hybrid genetic algorithm/scaled conjugate gradient regularisation method is designed to alleviate ANN `over-fitting'. In application to day-ahead load forecasting, the proposed algorithm performs better than early-stopping and Bayesian regularisation, showing promising initial results.

Bayesian methodology for genetics of complex diseases

Genetic research of complex diseases is a challenging, but exciting, area of research. The early development of the research was limited, however, until the completion of the Human Genome and HapMap projects, along with the reduction in the cost of genotyping, which paves the way for understanding the genetic composition of complex diseases. In this thesis, we focus on the statistical methods for two aspects of genetic research: phenotype definition for diseases with complex etiology and methods for identifying potentially associated Single Nucleotide Polymorphisms (SNPs) and SNP-SNP interactions. With regard to phenotype definition for diseases with complex etiology, we firstly investigated the effects of different statistical phenotyping approaches on the subsequent analysis. In light of the findings, and the difficulties in validating the estimated phenotype, we proposed two different methods for reconciling phenotypes of different models using Bayesian model averaging as a coherent mechanism for accounting for model uncertainty. In the second part of the thesis, the focus is turned to the methods for identifying associated SNPs and SNP interactions. We review the use of Bayesian logistic regression with variable selection for SNP identification and extended the model for detecting the interaction effects for population based case-control studies. In this part of study, we also develop a machine learning algorithm to cope with the large scale data analysis, namely modified Logic Regression with Genetic Program (MLR-GEP), which is then compared with the Bayesian model, Random Forests and other variants of logic regression.

The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network

Sample complexity results from computational learning theory, when applied to neural network learning for pattern classification problems, suggest that for good generalization performance the number of training examples should grow at least linearly with the number of adjustable parameters in the network. Results in this paper show that if a large neural network is used for a pattern classification problem and the learning algorithm finds a network with small weights that has small squared error on the training patterns, then the generalization performance depends on the size of the weights rather than the number of weights. For example, consider a two-layer feedforward network of sigmoid units, in which the sum of the magnitudes of the weights associated with each unit is bounded by A and the input dimension is n. We show that the misclassification probability is no more than a certain error estimate (that is related to squared error on the training set) plus A3 √((log n)/m) (ignoring log A and log m factors), where m is the number of training patterns. This may explain the generalization performance of neural networks, particularly when the number of training examples is considerably smaller than the number of weights. It also supports heuristics (such as weight decay and early stopping) that attempt to keep the weights small during training. The proof techniques appear to be useful for the analysis of other pattern classifiers: when the input domain is a totally bounded metric space, we use the same approach to give upper bounds on misclassification probability for classifiers with decision boundaries that are far from the training examples.

Neural network learning:theoretical foundations

This important work describes recent theoretical advances in the study of artificial neural networks. It explores probabilistic models of supervised learning problems, and addresses the key statistical and computational questions. Chapters survey research on pattern classification with binary-output networks, including a discussion of the relevance of the Vapnik Chervonenkis dimension, and of estimates of the dimension for several neural network models. In addition, Anthony and Bartlett develop a model of classification by real-output networks, and demonstrate the usefulness of classification with a "large margin." The authors explain the role of scale-sensitive versions of the Vapnik Chervonenkis dimension in large margin classification, and in real prediction. Key chapters also discuss the computational complexity of neural network learning, describing a variety of hardness results, and outlining two efficient, constructive learning algorithms. The book is self-contained and accessible to researchers and graduate students in computer science, engineering, and mathematics

Exploring health behaviour determinants of ageing Australians with chronic diseases

Background: Chronic disease presents overwhelming challenges to elderly patients, their families, health care providers and the health care system. The aim of this study was to explore a theoretical model for effective management of chronic diseases, especially type 2 diabetes mellitus and/or cardiovascular disease. The assumed theoretical model considered the connections between physical function, mental health, social support and health behaviours. The study effort was to improve the quality of life for people with chronic diseases, especially type 2 diabetes and/or cardiovascular disease and to reduce health costs. Methods: A cross-sectional post questionnaire survey was conducted in early 2009 from a randomised sample of Australians aged 50 to 80 years. A total of 732 subjects were eligible for analysis. Firstly, factors influencing respondents‘ quality of life were investigated through bivariate and multivariate regression analysis. Secondly, the Theory of Planned Behaviour (TPB) model for regular physical activity, healthy eating and medication adherence behaviours was tested for all relevant respondents using regression analysis. Thirdly, TPB variable differences between respondents who have diabetes and/or cardiovascular disease and those without these diseases were compared. Finally, the TPB model for three behaviours including regular physical activity, healthy eating and medication adherence were tested in respondents with diabetes and/or cardiovascular diseases using Structure Equation Modelling (SEM). Results: This was the first study combining the three behaviours using a TPB model, while testing the influence of extra variables on the TPB model in one study. The results of this study provided evidence that the ageing process was a cumulative effect of biological change, socio-economic environment and lifelong behaviours. Health behaviours, especially physical activity and healthy eating were important modifiable factors influencing respondents‘ quality of life. Since over 80% of the respondents had at least one chronic disease, it was important to consider supporting older people‘s chronic disease self-management skills such as healthy diet, regular physical activity and medication adherence to improve their quality of life. Direct measurement of the TPB model was helpful in understanding respondents‘ intention and behaviour toward physical activity, healthy eating and medication adherence. In respondents with diabetes and/or cardiovascular disease, the TPB model predicted different proportions of intention toward three different health behaviours with 39% intending to engage in physical activity, 49% intending to engage in healthy eating and 47% intending to comply with medication adherence. Perceived behavioural control, which was proven to be the same as self-efficacy in measurement in this study, played an important role in predicting intention towards the three health behaviours. Also social norms played a slightly more important role than attitude for physical activity and medication adherence, while attitude and social norms had similar effects on healthy eating in respondents with diabetes and/or cardiovascular disease. Both perceived behavioural control and intention directly predicted recent actual behaviours. Physical activity was more a volitional control behaviour than healthy eating and medication adherence. Step by step goal setting and motivation was more important for physical activity, while accessibility, resources and other social environmental factors were necessary for improving healthy eating and medication adherence. The extra variables of age, waist circumference, health related quality of life and depression indirectly influenced intention towards the three behaviours mainly mediated through attitude and perceived behavioural control. Depression was a serious health problem that reduced the three health behaviours‘ motivation, mediated through decreased self-efficacy and negative attitude. This research provided evidence that self-efficacy is similar to perceived behavioural control in the TPB model and intention is a proximal goal toward a particular behaviour. Combining four sources of information in the self-efficacy model with the TPB model would improve chronic disease patients‘ self management behaviour and reach an improved long-term treatment outcome. Conclusion: Health intervention programs that target chronic disease management should focus on patients‘ self-efficacy. A holistic approach which is patient-centred and involves a multidisciplinary collaboration strategy would be effective. Supporting the socio-economic environment and the mental/ emotional environment for older people needs to be considered within an integrated health care system.

Aerial image analysis using spiking neural networks with application to power line corridor monitoring

Trees, shrubs and other vegetation are of continued importance to the environment and our daily life. They provide shade around our roads and houses, offer a habitat for birds and wildlife, and absorb air pollutants. However, vegetation touching power lines is a risk to public safety and the environment, and one of the main causes of power supply problems. Vegetation management, which includes tree trimming and vegetation control, is a significant cost component of the maintenance of electrical infrastructure. For example, Ergon Energy, the Australia’s largest geographic footprint energy distributor, currently spends over $80 million a year inspecting and managing vegetation that encroach on power line assets. Currently, most vegetation management programs for distribution systems are calendar-based ground patrol. However, calendar-based inspection by linesman is labour-intensive, time consuming and expensive. It also results in some zones being trimmed more frequently than needed and others not cut often enough. Moreover, it’s seldom practicable to measure all the plants around power line corridors by field methods. Remote sensing data captured from airborne sensors has great potential in assisting vegetation management in power line corridors. This thesis presented a comprehensive study on using spiking neural networks in a specific image analysis application: power line corridor monitoring. Theoretically, the thesis focuses on a biologically inspired spiking cortical model: pulse coupled neural network (PCNN). The original PCNN model was simplified in order to better analyze the pulse dynamics and control the performance. Some new and effective algorithms were developed based on the proposed spiking cortical model for object detection, image segmentation and invariant feature extraction. The developed algorithms were evaluated in a number of experiments using real image data collected from our flight trails. The experimental results demonstrated the effectiveness and advantages of spiking neural networks in image processing tasks. Operationally, the knowledge gained from this research project offers a good reference to our industry partner (i.e. Ergon Energy) and other energy utilities who wants to improve their vegetation management activities. The novel approaches described in this thesis showed the potential of using the cutting edge sensor technologies and intelligent computing techniques in improve power line corridor monitoring. The lessons learnt from this project are also expected to increase the confidence of energy companies to move from traditional vegetation management strategy to a more automated, accurate and cost-effective solution using aerial remote sensing techniques.

Frequency response function based structural damage detection using artificial neural networks

Damage detection in structures has become increasingly important in recent years. While a number of damage detection and localization methods have been proposed, few attempts have been made to explore the structure damage with frequency response functions (FRFs). This paper illustrates the damage identification and condition assessment of a beam structure using a new frequency response functions (FRFs) based damage index and Artificial Neural Networks (ANNs). In practice, usage of all available FRF data as an input to artificial neural networks makes the training and convergence impossible. Therefore one of the data reduction techniques Principal Component Analysis (PCA) is introduced in the algorithm. In the proposed procedure, a large set of FRFs are divided into sub-sets in order to find the damage indices for different frequency points of different damage scenarios. The basic idea of this method is to establish features of damaged structure using FRFs from different measurement points of different sub-sets of intact structure. Then using these features, damage indices of different damage cases of the structure are identified after reconstructing of available FRF data using PCA. The obtained damage indices corresponding to different damage locations and severities are introduced as input variable to developed artificial neural networks. Finally, the effectiveness of the proposed method is illustrated and validated by using the finite element modal of a beam structure. The illustrated results show that the PCA based damage index is suitable and effective for structural damage detection and condition assessment of building structures.

Neural interface systems : a new legal frontier?

In an age where digital innovation knows no boundaries, research in the area of brain-computer interface and other neural interface devices go where none have gone before. The possibilities are endless and as dreams become reality, the implications of these amazing developments should be considered. Some of these new devices have been created to correct or minimise the effects of disease or injury so the paper discusses some of the current research and development in the area, including neuroprosthetics. To assist researchers and academics in identifying some of the legal and ethical issues that might arise as a result of research and development of neural interface devices, using both non-invasive techniques and invasive procedures, the paper discusses a number of recent observations of authors in the field. The issue of enhancing human attributes by incorporating these new devices is also considered. Such enhancement may be regarded as freeing the mind from the constraints of the body, but there are legal and moral issues that researchers and academics would be well advised to contemplate as these new devices are developed and used. While different fact situation surround each of these new devices, and those that are yet to come, consideration of the legal and ethical landscape may assist researchers and academics in dealing effectively with matters that arise in these times of transition. Lawyers could seek to facilitate the resolution of the legal disputes that arise in this area of research and development within the existing judicial and legislative frameworks. Whether these frameworks will suffice, or will need to change in order to enable effective resolution, is a broader question to be explored.

Neural differentiation potential of human bone marrow-derived mesenchymal stromal cells: misleading marker gene expression

Background In contrast to pluripotent embryonic stem cells, adult stem cells have been considered to be multipotent, being somewhat more restricted in their differentiation capacity and only giving rise to cell types related to their tissue of origin. Several studies, however, have reported that bone marrow-derived mesenchymal stromal cells (MSCs) are capable of transdifferentiating to neural cell types, effectively crossing normal lineage restriction boundaries. Such reports have been based on the detection of neural-related proteins by the differentiated MSCs. In order to assess the potential of human adult MSCs to undergo true differentiation to a neural lineage and to determine the degree of homogeneity between donor samples, we have used RT-PCR and immunocytochemistry to investigate the basal expression of a range of neural related mRNAs and proteins in populations of non-differentiated MSCs obtained from 4 donors. Results The expression analysis revealed that several of the commonly used marker genes from other studies like nestin, Enolase2 and microtubule associated protein 1b (MAP1b) are already expressed by undifferentiated human MSCs. Furthermore, mRNA for some of the neural-related transcription factors, e.g. Engrailed-1 and Nurr1 were also strongly expressed. However, several other neural-related mRNAs (e.g. DRD2, enolase2, NFL and MBP) could be identified, but not in all donor samples. Similarly, synaptic vesicle-related mRNA, STX1A could only be detected in 2 of the 4 undifferentiated donor hMSC samples. More significantly, each donor sample revealed a unique expression pattern, demonstrating a significant variation of marker expression. Conclusion The present study highlights the existence of an inter-donor variability of expression of neural-related markers in human MSC samples that has not previously been described. This donor-related heterogeneity might influence the reproducibility of transdifferentiation protocols as well as contributing to the ongoing controversy about differentiation capacities of MSCs. Therefore, further studies need to consider the differences between donor samples prior to any treatment as well as the possibility of harvesting donor cells that may be inappropriate for transplantation strategies.

Human neural cell interactions with orientated electrospun nanofibers in vitro

Aim: Electrospun nanofibers represent potent guidance substrates for nervous tissue repair. Development of nanofiber-based scaffolds for CNS repair requires, as a first step, an understanding of appropriate neural cell type-substrate interactions. Materials & methods: Astrocyte–nanofiber interactions (e.g., adhesion, proliferation, process extension and migration) were studied by comparing human neural progenitor-derived astrocytes (hNP-ACs) and a human astrocytoma cell line (U373) with aligned polycaprolactone (PCL) nanofibers or blended (25% type I collagen/75% PCL) nanofibers. Neuron–nanofiber interactions were assessed using a differentiated human neuroblastoma cell line (SH-SY5Y). Results & discussion: U373 cells and hNP-AC showed similar process alignment and length when associated with PCL or Type I collagen/PCL nanofibers. Cell adhesion and migration by hNP-AC were clearly improved by functionalization of nanofiber surfaces with type I collagen. Functionalized nanofibers had no such effect on U373 cells. Another clear difference between the U373 cells and hNP-AC interactions with the nanofiber substrate was proliferation; the cell line demonstrating strong proliferation, whereas the hNP-AC line showed no proliferation on either type of nanofiber. Long axonal growth (up to 600 µm in length) of SH-SY5Y neurons followed the orientation of both types of nanofibers even though adhesion of the processes to the fibers was poor. Conclusion: The use of cell lines is of only limited predictive value when studying cell–substrate interactions but both morphology and alignment of human astrocytes were affected profoundly by nanofibers. Nanofiber surface functionalization with collagen significantly improved hNP-AC adhesion and migration. Alternative forms of functionalization may be required for optimal axon–nanofiber interactions.

Using strategic movement to calibrate a neural compass : a spiking network for tracking head direction in rats and robots

The head direction (HD) system in mammals contains neurons that fire to represent the direction the animal is facing in its environment. The ability of these cells to reliably track head direction even after the removal of external sensory cues implies that the HD system is calibrated to function effectively using just internal (proprioceptive and vestibular) inputs. Rat pups and other infant mammals display stereotypical warm-up movements prior to locomotion in novel environments, and similar warm-up movements are seen in adult mammals with certain brain lesion-induced motor impairments. In this study we propose that synaptic learning mechanisms, in conjunction with appropriate movement strategies based on warm-up movements, can calibrate the HD system so that it functions effectively even in darkness. To examine the link between physical embodiment and neural control, and to determine that the system is robust to real-world phenomena, we implemented the synaptic mechanisms in a spiking neural network and tested it on a mobile robot platform. Results show that the combination of the synaptic learning mechanisms and warm-up movements are able to reliably calibrate the HD system so that it accurately tracks real-world head direction, and that calibration breaks down in systematic ways if certain movements are omitted. This work confirms that targeted, embodied behaviour can be used to calibrate neural systems, demonstrates that ‘grounding’ of modeled biological processes in the real world can reveal underlying functional principles (supporting the importance of robotics to biology), and proposes a functional role for stereotypical behaviours seen in infant mammals and those animals with certain motor deficits. We conjecture that these calibration principles may extend to the calibration of other neural systems involved in motion tracking and the representation of space, such as grid cells in entorhinal cortex.