Quo Vadis? Reliable and Practical Rule Extraction from Neural Networks

Rule extraction from neural network algorithms have been investigated for two decades and there have been significant applications. Despite this level of success, rule extraction from neural network methods are generally not part of data mining tools, and a significant commercial breakthrough may still be some time away. This paper briefly reviews the state-of-the-art and points to some of the obstacles, namely a lack of evaluation techniques in experiments and larger benchmark data sets. A significant new development is the view that rule extraction from neural networks is an interactive process which actively involves the user. This leads to the application of assessment and evaluation techniques from information retrieval which may lead to a range of new methods.

Structural damage detection using frequency response functions and neural networks

This paper illustrates the damage identification and condition assessment of a three story bookshelf structure using a new frequency response functions (FRFs) based damage index and Artificial Neural Networks (ANNs). A major obstacle of using measured frequency response function data is a large size input variables to ANNs. This problem is overcome by applying a data reduction technique called principal component analysis (PCA). In the proposed procedure, ANNs with their powerful pattern recognition and classification ability were used to extract damage information such as damage locations and severities from measured FRFs. Therefore, simple neural network models are developed, trained by Back Propagation (BP), to associate the FRFs with the damage or undamaged locations and severity of the damage of the structure. Finally, the effectiveness of the proposed method is illustrated and validated by using the real data provided by the Los Alamos National Laboratory, USA. The illustrated results show that the PCA based artificial Neural Network method is suitable and effective for damage identification and condition assessment of building structures. In addition, it is clearly demonstrated that the accuracy of proposed damage detection method can also be improved by increasing number of baseline datasets and number of principal components of the baseline dataset.

Damage identification and condition assessment of building structures using frequency response functions and neural networks

This thesis investigated the viability of using Frequency Response Functions in combination with Artificial Neural Network technique in damage assessment of building structures. The proposed approach can help overcome some of limitations associated with previously developed vibration based methods and assist in delivering more accurate and robust damage identification results. Excellent results are obtained for damage identification of the case studies proving that the proposed approach has been developed successfully.

The use of artificial neural networks for identifying sustainable biodiesel feedstocks

Over the past few decades, biodiesel produced from oilseed crops and animal fat is receiving much attention as a renewable and sustainable alternative for automobile engine fuels, and particularly petroleum diesel. However, current biodiesel production is heavily dependent on edible oil feedstocks which are unlikely to be sustainable in the longer term due to the rising food prices and the concerns about automobile engine durability. Therefore, there is an urgent need for researchers to identify and develop sustainable biodiesel feedstocks which overcome the disadvantages of current ones. On the other hand, artificial neural network (ANN) modeling has been successfully used in recent years to gain new knowledge in various disciplines. The main goal of this article is to review recent literatures and assess the state of the art on the use of ANN as a modeling tool for future generation biodiesel feedstocks. Biodiesel feedstocks, production processes, chemical compositions, standards, physio-chemical properties and in-use performance are discussed. Limitations of current biodiesel feedstocks over future generation biodiesel feedstock have been identified. The application of ANN in modeling key biodiesel quality parameters and combustion performance in automobile engines is also discussed. This review has determined that ANN modeling has a high potential to contribute to the development of renewable energy systems by accelerating biodiesel research.

THE quest for alpha : can artificial neural networks help?

The application of artificial neural networks (ANN) in finance is relatively new area of research. We employed ANNs that used both fundamental and technical inputs to predict future prices of widely held Australian stocks and used these predicted prices for stock portfolio selection over a 10-year period (2001-2011). We found that the ANNs generally do well in predicting the direction of stock price movements. The stock portfolios selected by the ANNs with median accuracy are able to generate positive alpha over the 10-year period. More importantly, we found that a portfolio based on randomly selected network configuration had zero chance of resulting in a significantly negative alpha but a 27% chance of yielding a significantly positive alpha. This is in stark contrast to the findings of the research on mutual fund performance where active fund managers with negative alphas outnumber those with positive alphas.

Out of their minds: legal theory in neural networks

This paper examines the use of connectionism (neural networks) in modelling legal reasoning. I discuss how the implementations of neural networks have failed to account for legal theoretical perspectives on adjudication. I criticise the use of neural networks in law, not because connectionism is inherently unsuitable in law, but rather because it has been done so poorly to date. The paper reviews a number of legal theories which provide a grounding for the use of neural networks in law. It then examines some implementations undertaken in law and criticises their legal theoretical naïvete. It then presents a lessons from the implementations which researchers must bear in mind if they wish to build neural networks which are justified by legal theories.

Artificial neural networks to predict activity type and energy expenditure in youth

Previous studies have demonstrated that pattern recognition approaches to accelerometer data reduction are feasible and moderately accurate in classifying activity type in children. Whether pattern recognition techniques can be used to provide valid estimates of physical activity (PA) energy expenditure in youth remains unexplored in the research literature. Purpose: The objective of this study is to develop and test artificial neural networks (ANNs) to predict PA type and energy expenditure (PAEE) from processed accelerometer data collected in children and adolescents. Methods: One hundred participants between the ages of 5 and 15 yr completed 12 activity trials that were categorized into five PA types: sedentary, walking, running, light-intensity household activities or games, and moderate-to-vigorous intensity games or sports. During each trial, participants wore an ActiGraph GTIM on the right hip, and (V) Over dotO(2) was measured using the Oxycon Mobile (Viasys Healthcare, Yorba Linda, CA) portable metabolic system. ANNs to predict PA type and PAEE (METs) were developed using the following features: 10th, 25th, 50th, 75th, and 90th percentiles and the lag one autocorrelation. To determine the highest time resolution achievable, we extracted features from 10-, 15-, 20-, 30-, and 60-s windows. Accuracy was assessed by calculating the percentage of windows correctly classified and root mean square en-or (RMSE). Results: As window size increased from 10 to 60 s, accuracy for the PA-type ANN increased from 81.3% to 88.4%. RMSE for the MET prediction ANN decreased from 1.1 METs to 0.9 METs. At any given window size, RMSE values for the MET prediction ANN were 30-40% lower than the conventional regression-based approaches. Conclusions: ANNs can be used to predict both PA type and PAEE in children and adolescents using count data from a single waist mounted accelerometer.

Classification of echolocation calls from 14 species of bat by Support Vector Machines and Ensembles of Neural Networks

Calls from 14 species of bat were classified to genus and species using discriminant function analysis (DFA), support vector machines (SVM) and ensembles of neural networks (ENN). Both SVMs and ENNs outperformed DFA for every species while ENNs (mean identification rate – 97%) consistently outperformed SVMs (mean identification rate – 87%). Correct classification rates produced by the ENNs varied from 91% to 100%; calls from six species were correctly identified with 100% accuracy. Calls from the five species of Myotis, a genus whose species are considered difficult to distinguish acoustically, had correct identification rates that varied from 91 – 100%. Five parameters were most important for classifying calls correctly while seven others contributed little to classification performance.

Human vs. machine : identification of bat species from their echolocation calls by humans and by artificial neural networks

Automated remote ultrasound detectors allow large amounts of data on bat presence and activity to be collected. Processing of such data involves identifying bat species from their echolocation calls. Automated species identification has the potential to provide more consistent, predictable, and potentially higher levels of accuracy than identification by humans. In contrast, identification by humans permits flexibility and intelligence in identification, as well as the incorporation of features and patterns that may be difficult to quantify. We compared humans with artificial neural networks (ANNs) in their ability to classify short recordings of bat echolocation calls of variable signal to noise ratios; these sequences are typical of those obtained from remote automated recording systems that are often used in large-scale ecological studies. We presented 45 recordings (1–4 calls) produced by known species of bats to ANNs and to 26 human participants with 1 month to 23 years of experience in acoustic identification of bats. Humans correctly classified 86% of recordings to genus and 56% to species; ANNs correctly identified 92% and 62%, respectively. There was no significant difference between the performance of ANNs and that of humans, but ANNs performed better than about 75% of humans. There was little relationship between the experience of the human participants and their classification rate. However, humans with <1 year of experience performed worse than others. Currently, identification of bat echolocation calls by humans is suitable for ecological research, after careful consideration of biases. However, improvements to ANNs and the data that they are trained on may in future increase their performance to beyond those demonstrated by humans.

Identification of New Zealand bats in flight from analysis of echolocation calls by artificial neural networks.

Time-expanded and heterodyned echolocation calls of the New Zealand long-tailed Chalinolobus tuberculatus and lesser short-tailed bat Mystacina tuberculata were recorded and digitally analysed. Temporal and spectral parameters were measured from time-expanded calls and power spectra generated for both time-expanded and heterodyned calls. Artificial neural networks were trained to classify the calls of both species using temporal and spectral parameters and power spectra as input data. Networks were then tested using data not previously seen. Calls could be unambiguously identified using parameters and power spectra from time-expanded calls. A neural network, trained and tested using power spectra of calls from both species recorded using a heterodyne detector set to 40 kHz (the frequency with the most energy of the fundamental of C. tuberculatus call), could identify 99% and 84% of calls of C. tuberculatus and M. tuberculata, respectively. A second network, trained and tested using power spectra of calls from both species recorded using a heterodyne detector set to 27 kHz (the frequency with the most energy of the fundamental of M. tuberculata call), could identify 34% and 100% of calls of C. tuberculatus and M. tuberculata, respectively. This study represents the first use of neural networks for the identification of bats from their echolocation calls. It is also the first study to use power spectra of time-expanded and heterodyned calls for identification of chiropteran species. The ability of neural networks to identify bats from their echolocation calls is discussed, as is the ecology of both species in relation to the design of their echolocation calls.

Acoustic identification of 12 species of echolocating bat by discriminant function analysis and artificial neural networks

We recorded echolocation calls from 14 sympatric species of bat in Britain. Once digitised, one temporal and four spectral features were measured from each call. The frequency-time course of each call was approximated by fitting eight mathematical functions, and the goodness of fit, represented by the mean-squared error, was calculated. Measurements were taken using an automated process that extracted a single call from background noise and measured all variables without intervention. Two species of Rhinolophus were easily identified from call duration and spectral measurements. For the remaining 12 species, discriminant function analysis and multilayer back-propagation perceptrons were used to classify calls to species level. Analyses were carried out with and without the inclusion of curve-fitting data to evaluate its usefulness in distinguishing among species. Discriminant function analysis achieved an overall correct classification rate of 79% with curve-fitting data included, while an artificial neural network achieved 87%. The removal of curve-fitting data improved the performance of the discriminant function analysis by 2 %, while the performance of a perceptron decreased by 2 %. However, an increase in correct identification rates when curve-fitting information was included was not found for all species. The use of a hierarchical classification system, whereby calls were first classified to genus level and then to species level, had little effect on correct classification rates by discriminant function analysis but did improve rates achieved by perceptrons. This is the first published study to use artificial neural networks to classify the echolocation calls of bats to species level. Our findings are discussed in terms of recent advances in recording and analysis technologies, and are related to factors causing convergence and divergence of echolocation call design in bats.

Primary and secondary neural networks of auditory prepulse inhibition: a functional magnetic resonance imaging study of sensorimotor gating of the human acoustic startle response

Feedforward inhibition deficits have been consistently demonstrated in a range of neuropsychiatric conditions using prepulse inhibition (PPI) of the acoustic startle eye-blink reflex when assessing sensorimotor gating. While PPI can be recorded in acutely decerebrated rats, behavioural, pharmacological and psychophysiological studies suggest the involvement of a complex neural network extending from brainstem nuclei to higher order cortical areas. The current functional magnetic resonance imaging study investigated the neural network underlying PPI and its association with electromyographically (EMG) recorded PPI of the acoustic startle eye-blink reflex in 16 healthy volunteers. A sparse imaging design was employed to model signal changes in blood oxygenation level-dependent (BOLD) responses to acoustic startle probes that were preceded by a prepulse at 120 ms or 480 ms stimulus onset asynchrony or without prepulse. Sensorimotor gating was EMG confirmed for the 120-ms prepulse condition, while startle responses in the 480-ms prepulse condition did not differ from startle alone. Multiple regression analysis of BOLD contrasts identified activation in pons, thalamus, caudate nuclei, left angular gyrus and bilaterally in anterior cingulate, associated with EMGrecorded sensorimotor gating. Planned contrasts confirmed increased pons activation for startle alone vs 120-ms prepulse condition, while increased anterior superior frontal gyrus activation was confirmed for the reverse contrast. Our findings are consistent with a primary pontine circuitry of sensorimotor gating that interconnects with inferior parietal, superior temporal, frontal and prefrontal cortices via thalamus and striatum. PPI processes in the prefrontal, frontal and superior temporal cortex were functionally distinct from sensorimotor gating.