Quo Vadis? Reliable and Practical Rule Extraction from Neural Networks

Rule extraction from neural network algorithms have been investigated for two decades and there have been significant applications. Despite this level of success, rule extraction from neural network methods are generally not part of data mining tools, and a significant commercial breakthrough may still be some time away. This paper briefly reviews the state-of-the-art and points to some of the obstacles, namely a lack of evaluation techniques in experiments and larger benchmark data sets. A significant new development is the view that rule extraction from neural networks is an interactive process which actively involves the user. This leads to the application of assessment and evaluation techniques from information retrieval which may lead to a range of new methods.

Utilising Reliability and Condition Monitoring Data for Asset Health Prognosis

The ability to forecast machinery health is vital to reducing maintenance costs, operation downtime and safety hazards. Recent advances in condition monitoring technologies have given rise to a number of prognostic models which attempt to forecast machinery health based on condition data such as vibration measurements. This paper demonstrates how the population characteristics and condition monitoring data (both complete and suspended) of historical items can be integrated for training an intelligent agent to predict asset health multiple steps ahead. The model consists of a feed-forward neural network whose training targets are asset survival probabilities estimated using a variation of the Kaplan–Meier estimator and a degradation-based failure probability density function estimator. The trained network is capable of estimating the future survival probabilities when a series of asset condition readings are inputted. The output survival probabilities collectively form an estimated survival curve. Pump data from a pulp and paper mill were used for model validation and comparison. The results indicate that the proposed model can predict more accurately as well as further ahead than similar models which neglect population characteristics and suspended data. This work presents a compelling concept for longer-range fault prognosis utilising available information more fully and accurately.

On robust face recognition via sparse encoding : the good, the bad, and the ugly

In the field of face recognition, Sparse Representation (SR) has received considerable attention during the past few years. Most of the relevant literature focuses on holistic descriptors in closed-set identification applications. The underlying assumption in SR-based methods is that each class in the gallery has sufficient samples and the query lies on the subspace spanned by the gallery of the same class. Unfortunately, such assumption is easily violated in the more challenging face verification scenario, where an algorithm is required to determine if two faces (where one or both have not been seen before) belong to the same person. In this paper, we first discuss why previous attempts with SR might not be applicable to verification problems. We then propose an alternative approach to face verification via SR. Specifically, we propose to use explicit SR encoding on local image patches rather than the entire face. The obtained sparse signals are pooled via averaging to form multiple region descriptors, which are then concatenated to form an overall face descriptor. Due to the deliberate loss spatial relations within each region (caused by averaging), the resulting descriptor is robust to misalignment & various image deformations. Within the proposed framework, we evaluate several SR encoding techniques: l1-minimisation, Sparse Autoencoder Neural Network (SANN), and an implicit probabilistic technique based on Gaussian Mixture Models. Thorough experiments on AR, FERET, exYaleB, BANCA and ChokePoint datasets show that the proposed local SR approach obtains considerably better and more robust performance than several previous state-of-the-art holistic SR methods, in both verification and closed-set identification problems. The experiments also show that l1-minimisation based encoding has a considerably higher computational than the other techniques, but leads to higher recognition rates.

Damage identification and condition assessment of building structures using frequency response functions and neural networks

This thesis investigated the viability of using Frequency Response Functions in combination with Artificial Neural Network technique in damage assessment of building structures. The proposed approach can help overcome some of limitations associated with previously developed vibration based methods and assist in delivering more accurate and robust damage identification results. Excellent results are obtained for damage identification of the case studies proving that the proposed approach has been developed successfully.

The use of artificial neural networks for identifying sustainable biodiesel feedstocks

Over the past few decades, biodiesel produced from oilseed crops and animal fat is receiving much attention as a renewable and sustainable alternative for automobile engine fuels, and particularly petroleum diesel. However, current biodiesel production is heavily dependent on edible oil feedstocks which are unlikely to be sustainable in the longer term due to the rising food prices and the concerns about automobile engine durability. Therefore, there is an urgent need for researchers to identify and develop sustainable biodiesel feedstocks which overcome the disadvantages of current ones. On the other hand, artificial neural network (ANN) modeling has been successfully used in recent years to gain new knowledge in various disciplines. The main goal of this article is to review recent literatures and assess the state of the art on the use of ANN as a modeling tool for future generation biodiesel feedstocks. Biodiesel feedstocks, production processes, chemical compositions, standards, physio-chemical properties and in-use performance are discussed. Limitations of current biodiesel feedstocks over future generation biodiesel feedstock have been identified. The application of ANN in modeling key biodiesel quality parameters and combustion performance in automobile engines is also discussed. This review has determined that ANN modeling has a high potential to contribute to the development of renewable energy systems by accelerating biodiesel research.

Intelligent reliability analysis with incomplete covariates

The reliability analysis is crucial to reducing unexpected down time, severe failures and ever tightened maintenance budget of engineering assets. Hazard based reliability methods are of particular interest as hazard reflects the current health status of engineering assets and their imminent failure risks. Most existing hazard models were constructed using the statistical methods. However, these methods were established largely based on two assumptions: one is the assumption of baseline failure distributions being accurate to the population concerned and the other is the assumption of effects of covariates on hazards. These two assumptions may be difficult to achieve and therefore compromise the effectiveness of hazard models in the application. To address this issue, a non-linear hazard modelling approach is developed in this research using neural networks (NNs), resulting in neural network hazard models (NNHMs), to deal with limitations due to the two assumptions for statistical models. With the success of failure prevention effort, less failure history becomes available for reliability analysis. Involving condition data or covariates is a natural solution to this challenge. A critical issue for involving covariates in reliability analysis is that complete and consistent covariate data are often unavailable in reality due to inconsistent measuring frequencies of multiple covariates, sensor failure, and sparse intrusive measurements. This problem has not been studied adequately in current reliability applications. This research thus investigates such incomplete covariates problem in reliability analysis. Typical approaches to handling incomplete covariates have been studied to investigate their performance and effects on the reliability analysis results. Since these existing approaches could underestimate the variance in regressions and introduce extra uncertainties to reliability analysis, the developed NNHMs are extended to include handling incomplete covariates as an integral part. The extended versions of NNHMs have been validated using simulated bearing data and real data from a liquefied natural gas pump. The results demonstrate the new approach outperforms the typical incomplete covariates handling approaches. Another problem in reliability analysis is that future covariates of engineering assets are generally unavailable. In existing practices for multi-step reliability analysis, historical covariates were used to estimate the future covariates. Covariates of engineering assets, however, are often subject to substantial fluctuation due to the influence of both engineering degradation and changes in environmental settings. The commonly used covariate extrapolation methods thus would not be suitable because of the error accumulation and uncertainty propagation. To overcome this difficulty, instead of directly extrapolating covariate values, projection of covariate states is conducted in this research. The estimated covariate states and unknown covariate values in future running steps of assets constitute an incomplete covariate set which is then analysed by the extended NNHMs. A new assessment function is also proposed to evaluate risks of underestimated and overestimated reliability analysis results. A case study using field data from a paper and pulp mill has been conducted and it demonstrates that this new multi-step reliability analysis procedure is able to generate more accurate analysis results.

Traversability estimation for a planetary rover via experimental kernel learning in a Gaussian process framework

A critical requirement for safe autonomous navigation of a planetary rover is the ability to accurately estimate the traversability of the terrain. This work considers the problem of predicting the attitude and configuration angles of the platform from terrain representations that are often incomplete due to occlusions and sensor limitations. Using Gaussian Processes (GP) and exteroceptive data as training input, we can provide a continuous and complete representation of terrain traversability, with uncertainty in the output estimates. In this paper, we propose a novel method that focuses on exploiting the explicit correlation in vehicle attitude and configuration during operation by learning a kernel function from vehicle experience to perform GP regression. We provide an extensive experimental validation of the proposed method on a planetary rover. We show significant improvement in the accuracy of our estimation compared with results obtained using standard kernels (Squared Exponential and Neural Network), and compared to traversability estimation made over terrain models built using state-of-the-art GP techniques.

An Artificial Neutral Network (ANN) model for predicting biodiesel kinetic viscosity as a function of temperature and chemical compositions

An Artificial Neural Network (ANN) is a computational modeling tool which has found extensive acceptance in many disciplines for modeling complex real world problems. An ANN can model problems through learning by example, rather than by fully understanding the detailed characteristics and physics of the system. In the present study, the accuracy and predictive power of an ANN was evaluated in predicting kinetic viscosity of biodiesels over a wide range of temperatures typically encountered in diesel engine operation. In this model, temperature and chemical composition of biodiesel were used as input variables. In order to obtain the necessary data for model development, the chemical composition and temperature dependent fuel properties of ten different types of biodiesels were measured experimentally using laboratory standard testing equipments following internationally recognized testing procedures. The Neural Networks Toolbox of MatLab R2012a software was used to train, validate and simulate the ANN model on a personal computer. The network architecture was optimised following a trial and error method to obtain the best prediction of the kinematic viscosity. The predictive performance of the model was determined by calculating the absolute fraction of variance (R2), root mean squared (RMS) and maximum average error percentage (MAEP) between predicted and experimental results. This study found that ANN is highly accurate in predicting the viscosity of biodiesel and demonstrates the ability of the ANN model to find a meaningful relationship between biodiesel chemical composition and fuel properties at different temperature levels. Therefore the model developed in this study can be a useful tool in accurately predict biodiesel fuel properties instead of undertaking costly and time consuming experimental tests.

Understanding the abnormal brain activity in epilepsy as a potential predictor of the onset of an epileptic seizure

The Brain Research Institute (BRI) uses various types of indirect measurements, including EEG and fMRI, to understand and assess brain activity and function. As well as the recovery of generic information about brain function, research also focuses on the utilisation of such data and understanding to study the initiation, dynamics, spread and suppression of epileptic seizures. To assist with the future focussing of this aspect of their research, the BRI asked the MISG 2010 participants to examine how the available EEG and fMRI data and current knowledge about epilepsy should be analysed and interpreted to yield an enhanced understanding about brain activity occurring before, at commencement of, during, and after a seizure. Though the deliberations of the study group were wide ranging in terms of the related matters considered and discussed, considerable progress was made with the following three aspects. (1) The science behind brain activity investigations depends crucially on the quality of the analysis and interpretation of, as well as the recovery of information from, EEG and fMRI measurements. A number of specific methodologies were discussed and formalised, including independent component analysis, principal component analysis, profile monitoring and change point analysis (hidden Markov modelling, time series analysis, discontinuity identification). (2) Even though EEG measurements accurately and very sensitively record the onset of an epileptic event or seizure, they are, from the perspective of understanding the internal initiation and localisation, of limited utility. They only record neuronal activity in the cortical (surface layer) neurons of the brain, which is a direct reflection of the type of electrical activity they have been designed to record. Because fMRI records, through the monitoring of blood flow activity, the location of localised brain activity within the brain, the possibility of combining fMRI measurements with EEG, as a joint inversion activity, was discussed and examined in detail. (3) A major goal for the BRI is to improve understanding about ``when'' (at what time) an epileptic seizure actually commenced before it is identified on an eeg recording, ``where'' the source of this initiation is located in the brain, and ``what'' is the initiator. Because of the general agreement in the literature that, in one way or another, epileptic events and seizures represent abnormal synchronisations of localised and/or global brain activity the modelling of synchronisations was examined in some detail. References C. M. Michel, G. Thut, S. Morand, A. Khateb, A. J. Pegna, R. Grave de Peralta, S. Gonzalez, M. Seeck and T. Landis, Electric source imaging of human brain functions, Brain Res. Rev. , 36 (2--3), 2001, 108--118. doi:10.1016/S0165-0173(01)00086-8 S. Ogawa, R. S. Menon, S. G. Kim and K. Ugurbil, On the characteristics of functional magnetic resonance imaging of the brain, Annu. Rev. Bioph. Biom. , 27 , 1998, 447--474. doi:10.1146/annurev.biophys.27.1.447 C. D. Binnie and H. Stefan, Modern electroencephalography: its role in epilepsy management, Clin. Neurophysiol. , 110 (10), 1999, 1671--1697. doi:10.1016/S1388-2457(99)00125-X J. X. Tao, A. Ray, S. Hawes-Ebersole and J. S. Ebersole, Intracranial eeg substrates of scalp eeg interictal spikes, Epilepsia , 46 (5), 2005, 669--76. doi:10.1111/j.1528-1167.2005.11404.x S. Ogawa, D. W. Tank, R. Menon, J. M. Ellermann, S. G. Kim, H. Merkle and K. Ugurbil, Intrinsic signal changes accompanying sensory stimulation: Functional brain mapping with magnetic resonance imaging, P. Natl. Acad. Sci. USA , 89 (13), 1992, 5951--5955. doi:10.1073/pnas.89.13.5951 J. Engel Jr., Report of the ilae classification core group, Epilepsia , 47 (9), 2006, 1558--1568. doi:10.1111/j.1528-1167.2006.00215.x L. Lemieux, A. Salek-Haddadi, O. Josephs, P. Allen, N. Toms, C. Scott, K. Krakow, R. Turner and D. R. Fish, Event-related fmri with simultaneous and continuous eeg: description of the method and initial case r port, NeuroImage , 14 (3), 2001, 780--7. doi:10.1006/nimg.2001.0853 P. Federico, D. F. Abbott, R. S. Briellmann, A. S. Harvey and G. D. Jackson, Functional mri of the pre-ictal state, Brain , 128 (8), 2005, 1811-7. doi:10.1093/brain/awh533 C. S. Hawco, A. P. Bagshaw, Y. Lu, F. Dubeau and J. Gotman, bold changes occur prior to epileptic spikes seen on scalp eeg, NeuroImage , 35 (4), 2007, 1450--1458. doi:10.1016/j.neuroimage.2006.12.042 F. Moeller, H. R. Siebner, S. Wolff, H. Muhle, R. Boor, O. Granert, O. Jansen, U. Stephani and M. Siniatchkin, Changes in activity of striato-thalamo-cortical network precede generalized spike wave discharges, NeuroImage , 39 (4), 2008, 1839--1849. doi:10.1016/j.neuroimage.2007.10.058 V. Osharina, E. Ponchel, A. Aarabi, R. Grebe and F. Wallois, Local haemodynamic changes preceding interictal spikes: A simultaneous electrocorticography (ecog) and near-infrared spectroscopy (nirs) analysis in rats, NeuroImage , 50 (2), 2010, 600--607. doi:10.1016/j.neuroimage.2010.01.009 R. S. Fisher, W. Boas, W. Blume, C. Elger, P. Genton, P. Lee and J. Engel, Epileptic seizures and epilepsy: Definitions proposed by the international league against epilepsy (ilae) and the international bureau for epilepsy (ibe), Epilepsia , 46 (4), 2005, 470--472. doi:10.1111/j.0013-9580.2005.66104.x H. Berger, Electroencephalogram in humans, Arch. Psychiat. Nerven. , 87 , 1929, 527--570. C. M. Michel, M. M. Murray, G. Lantz, S. Gonzalez, L. Spinelli and R. G. de Peralta, eeg source imaging, Clin. Neurophysiol. , 115 (10), 2004, 2195--2222. doi:10.1016/j.clinph.2004.06.001 P. L. Nunez and R. B. Silberstein, On the relationship of synaptic activity to macroscopic measurements: Does co-registration of eeg with fmri make sense?, Brain Topogr. , 13 (2), 2000, 79--96. doi:10.1023/A:1026683200895 S. Ogawa, T. M. Lee, A. R. Kay and D. W. Tank, Brain magnetic resonance imaging with contrast dependent on blood oxygenation, P. Natl. Acad. Sci. USA , 87 (24), 1990, 9868--9872. doi:10.1073/pnas.87.24.9868 J. S. Gati, R. S. Menon, K. Ugurbil and B. K. Rutt, Experimental determination of the bold field strength dependence in vessels and tissue, Magn. Reson. Med. , 38 (2), 1997, 296--302. doi:10.1002/mrm.1910380220 P. A. Bandettini, E. C. Wong, R. S. Hinks, R. S. Tikofsky and J. S. Hyde, Time course EPI of human brain function during task activation, Magn. Reson. Med. , 25 (2), 1992, 390--397. K. K. Kwong, J. W. Belliveau, D. A. Chesler, I. E. Goldberg, R. M. Weisskoff, B. P. Poncelet, D. N. Kennedy, B. E. Hoppelm, M. S. Cohen and R. Turner, Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation, P. Natl. Acad. Sci. USA , 89 (12), 1992, 5675--5679. doi:10.1073/pnas.89.12.5675 J. Frahm, K. D. Merboldt and W. Hnicke, Functional mri of human brain activation at high spatial resolution, Magn. Reson. Med. , 29 (1), 1993, 139--144. P. A. Bandettini, A. Jesmanowicz, E. C. Wong and J. S. Hyde, Processing strategies for time-course data sets in functional MRI of the human brain, Magn. Reson. Med. , 30 (2), 1993, 161--173. K. J. Friston, P. Jezzard and R. Turner, Analysis of functional MRI time-series, Hum. Brain Mapp. , 1 (2), 1994, 153--171. B. Biswal, F. Z. Yetkin, V. M. Haughton and J. S. Hyde, Functional connectivity in the motor cortex of resting human brain using echo-planar mri, Mag. Reson. Med. , 34 (4), 1995, 537--541. doi:10.1002/mrm.1910340409 K. J. Friston, J. Ashburner, C. D. Frith, J. Poline, J. D. Heather and R. S. J. Frackowiak, Spatial registration and normalization of images, Hum. Brain Mapp. , 3 (3), 1995, 165--189. K. J. Friston, S. Williams, R. Howard, R. S. Frackowiak and R. Turner, Movement-related effects in fmri time-series, Magn. Reson. Med. , 35 (3), 1996, 346--355. G. H. Glover, T. Q. Li and D. Ress, Image-based method for retrospective correction of physiological motion effects in fmri: Retroicor, Magn. Reson. Med. , 44 (1), 2000, 162--167. doi:10.1002/1522-2594(200007)44:13.0.CO;2-E K. J. Friston, O. Josephs, G. Rees and R. Turner, Nonlinear event-related responses in fmri, Magn. Reson. Med. , 39 (1), 1998, 41--52. doi:10.1002/mrm.1910390109 K. Ugurbil, L. Toth and D. Kim, How accurate is magnetic resonance imaging of brain function?, Trends Neurosci. , 26 (2), 2003, 108--114. doi:10.1016/S0166-2236(02)00039-5 D. S. Kim, I. Ronen, C. Olman, S. G. Kim, K. Ugurbil and L. J. Toth, Spatial relationship between neuronal activity and bold functional mri, NeuroImage , 21 (3), 2004, 876--885. doi:10.1016/j.neuroimage.2003.10.018 A. Connelly, G. D. Jackson, R. S. Frackowiak, J. W. Belliveau, F. Vargha-Khadem and D. G. Gadian, Functional mapping of activated human primary cortex with a clinical mr imaging system, Radiology , 188 (1), 1993, 125--130. L. Allison, Hidden Markov Models, Technical Report , School of Computer and Software Engineering, Monash University, 2000. R. J. Elliott, L. Aggoun and J.B. Moore, Hidden Markov Models: Estimation and Control, Appl. Math.-Czech. , 2004. B. Bhavnagri, Discontinuities of plane functions projected from a surface with methods for finding these , Technical Report, 2009. B. Bhavnagri, Computer Vision using Shape Spaces , Technical Report,1996, University of Adelaide. B. Bhavnagri, A method for representing shape based on an equivalence relation on polygons, Pattern Recogn. , 27 (2), 1994, 247--260. doi:10.1016/0031-3203(94)90057-4 D. F. Abbott, A. B. Waites, A. S. Harvey and G. D. Jackson, Exploring epileptic seizure onset with fmri, NeuroImage , 36(S1) (344TH-PM), 2007. M. C. Mackey and L. Glass, Oscillation and chaos in physiological control systems, Science , 197 , 1977, 287--289. S. H. Strogatz, SYNC - The Emerging Science of Spontaneous Order , Theia, New York, 2003. J. W. Kim, J. A. Roberts and P. A. Robinson, Dynamics of epileptic seizures: Evolution, spreading, and suppression, J. Theor. Biol. , 257 (4), 2009, 527--532. doi:10.1016/j.jtbi.2008.12.009 Y. Kuramoto, T. Aoyagi, I. Nishikawa, T. Chawanya T and K. Okuda, Neural network model carrying phase information with application to collective dynamics, J. Theor. Phys. , 87 (5), 1992, 1119--1126. V. B. Mountcastle, The columnar organization of the neocortex, Brain , 120 (4), 1997, 701. doi:10.1093/brain/120.4.701 F. L. Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Epilepsies as dynamical diseases of brain systems: Basic models of the transition between normal and epileptic activity, Epilepsia , 44 (12), 2003, 72--83. F. H. Lopes da Silva, W. Blanes, S. N. Kalitzin, J. Parra, P. Suffczynski and D. N. Velis, Dynamical diseases of brain systems: different routes to epileptic seizures, ieee T. Bio-Med. Eng. , 50 (5), 2003, 540. L.D. Iasemidis, Epileptic seizure prediction and control, ieee T. Bio-Med. Eng. , 50 (5), 2003, 549--558. L. D. Iasemidis, D. S. Shiau, W. Chaovalitwongse, J. C. Sackellares, P. M. Pardalos, J. C. Principe, P. R. Carney, A. Prasad, B. Veeramani, and K. Tsakalis, Adaptive epileptic seizure prediction system, ieee T. Bio-Med. Eng. , 50 (5), 2003, 616--627. K. Lehnertz, F. Mormann, T. Kreuz, R.G. Andrzejak, C. Rieke, P. David and C. E. Elger, Seizure prediction by nonlinear eeg analysis, ieee Eng. Med. Biol. , 22 (1), 2003, 57--63. doi:10.1109/MEMB.2003.1191451 K. Lehnertz, R. G. Andrzejak, J. Arnhold, T. Kreuz, F. Mormann, C. Rieke, G. Widman and C. E. Elger, Nonlinear eeg analysis in epilepsy: Its possible use for interictal focus localization, seizure anticipation, and prevention, J. Clin. Neurophysiol. , 18 (3), 2001, 209. B. Litt and K. Lehnertz, Seizure prediction and the preseizure period, Curr. Opin. Neurol. , 15 (2), 2002, 173. doi:10.1097/00019052-200204000-00008 B. Litt and J. Echauz, Prediction of epileptic seizures, Lancet Neurol. , 1 (1), 2002, 22--30. doi:10.1016/S1474-4422(02)00003-0 M. M{a}kiranta, J. Ruohonen, K Suominen, J. Niinim{a}ki, E. Sonkaj{a}rvi, V. Kiviniemi, T. Sepp{a}nen, S. Alahuhta, V. J{a}ntti and O. Tervonen, {bold} signal increase preceeds eeg spike activity--a dynamic penicillin induced focal epilepsy in deep anesthesia, NeuroImage , 27 (4), 2005, 715--724. doi:10.1016/j.neuroimage.2005.05.025 K. Lehnertz, F. Mormann, H. Osterhage, A. M{u}ller, J. Prusseit, A. Chernihovskyi, M. Staniek, D. Krug, S. Bialonski and C. E. Elger, State-of-the-art of seizure prediction, J. Clin. Neurophysiol. , 24 (2), 2007, 147. doi:10.1097/WNP.0b013e3180336f16 F. Mormann, T. Kreuz, C. Rieke, R. G. Andrzejak, A. Kraskov, P. David, C. E. Elger and K. Lehnertz, On the predictability of epileptic seizures, Clin. Neurophysiol. , 116 (3), 2005, 569--587. doi:10.1016/j.clinph.2004.08.025 F. Mormann, R. G. Andrzejak, C. E. Elger and K. Lehnertz, Seizure prediction: the long and winding road, Brain , 130 (2), 2007, 314--333. doi:10.1093/brain/awl241 Z. Rogowski, I. Gath and E. Bental, On the prediction of epileptic seizures, Biol. Cybern. , 42 (1), 1981, 9--15. Y. Salant, I. Gath, O. Henriksen, Prediction of epileptic seizures from two-channel eeg, Med. Biol. Eng. Comput. , 36 (5), 1998, 549--556. doi:10.1007/BF02524422 J. Gotman and D.J. Koffler, Interictal spiking increases after seizures but does not after decrease in medication, Evoked Potential , 72 (1), 1989, 7--15. J. Gotman and M. G. Marciani, Electroencephalographic spiking activity, drug levels, and seizure occurence in epileptic patients, Ann. Neurol. , 17 (6), 1985, 59--603. A. Katz, D. A. Marks, G. McCarthy and S. S. Spencer, Does interictal spiking change prior to seizures?, Electroen. Clin. Neuro. , 79 (2), 1991, 153--156. A. Granada, R. M. Hennig, B. Ronacher, A. Kramer and H. Herzel, Phase Response Curves: Elucidating the dynamics of couples oscillators, Method Enzymol. , 454 (A), 2009, 1--27. doi:10.1016/S0076-6879(08)03801-9 doi:10.1016/S0076-6879(08)03801-9 H. Kantz and T. Schreiber, Nonlinear time series analysis , 2004, Cambridge Univ Press. M. V. L. Bennett and R. S Zukin, Electrical coupling and neuronal synchronization in the mammalian brain, Neuron , 41 (4), 2004, 495 --511. doi:10.1016/S0896-6273(04)00043-1 L.D. Iasemidis, J. Chris Sackellares, H. P. Zaveri and W. J. Williams, Phase space topography and the Lyapunov exponent of electrocorticograms in partial seizures, Brain Topogr. , 2 (3), 1990, 187--201. doi:10.1007/BF01140588 M. Le Van Quyen, J. Martinerie, V. Navarro, M. Baulac and F. J. Varela, Characterizing neurodynamic changes before seizures, J. Clin. Neurophysiol. , 18 (3), 2001, 191. J. Martinerie, C. Adam, M. Le Van Quyen, M. Baulac, S. Clemenceau, B. Renault and F. J. Varela, Epileptic seizures can be anticipated by non-linear analysis, Nat. Med. , 4 (10), 1998, 1173--1176. doi:10.1038/2667 A. Pikovsky, M. Rosenblum, J. Kurths and R. C. Hilborn, Synchronization: A universal concept in nonlinear science, Amer. J. Phys. , 70 , 2002, 655. H. R. Wilson and J. D. Cowan, Excitatory and inhibitory interactions in localized populations of model neurons, Biophys. J. , 12 (1), 1972, 1--24. D. Cumin and C. P. Unsworth, Generalising the Kuramoto model for the study of neuronal synchronisation in the brain, Physica D , 226 (2), 2007, 181--196. doi:10.1016/j.physd.2006.12.004 F. K. Skinner, H. Bazzazi and S. A. Campbell, Two-cell to N-cell heterogeneous, inhibitory networks: Precise linking of multistable and coherent properties, J. Comput. Neurosci. , 18 (3), 2005, 343--352. doi:10.1007/s10827-005-0331-1 W. W. Lytton, Computer modelling of epilepsy, Nat. Rev. Neurosci. , 9 (8), 2008, 626--637. doi:10.1038/nrn2416 R. D. Traub, A. Bibbig, F. E. N. LeBeau, E. H. Buhl and M. A. Whittington, Cellular mechanisms of neuronal population oscillations in the hippocampus in vitro, Ann. Rev. , 2004. R. D. Traub, A. Draguhn, M. A. Whittington, T. Baldeweg, A. Bibbig, E. H. Buhl and D. Schmitz, Axonal gap junc ions between principal neurons: A novel source of network oscillations, and perhaps epileptogenesis., Rev. Neuroscience , 13 (1), 2002, 1. doi:10.1146/annurev.neuro.27.070203.144303 M. Scheffer, J. Bascompte, W. A. Brock, V. Brovkin, S. R. Carpenter, V. Dakos, H. Held, E. H. van Nes, M. Rietkerk and G. Sugihara, Early-warning signals for critical transitions, Nature , 461 (7260), 2009, 53--59. doi:10.1038/nature08227 K. Murphy, A Brief Introduction to Graphical Models and Bayesian Networks , 2008, http://www.cs.ubc.ca/murphyk/Bayes/bnintro.html . R. C. Bradley, An elementary

Features-pooling blind jpeg image steganalysis

In this research, we introduce a new blind steganalysis in detecting grayscale JPEG images. Features-pooling method is employed to extract the steganalytic features and the classification is done by using neural network. Three different steganographic models are tested and classification results are compared to the five state-of-the-art blind steganalysis.

Prediction of activity type in preschool children using machine learning techniques

Objectives Recent research has shown that machine learning techniques can accurately predict activity classes from accelerometer data in adolescents and adults. The purpose of this study is to develop and test machine learning models for predicting activity type in preschool-aged children. Design Participants completed 12 standardised activity trials (TV, reading, tablet game, quiet play, art, treasure hunt, cleaning up, active game, obstacle course, bicycle riding) over two laboratory visits. Methods Eleven children aged 3–6 years (mean age = 4.8 ± 0.87; 55% girls) completed the activity trials while wearing an ActiGraph GT3X+ accelerometer on the right hip. Activities were categorised into five activity classes: sedentary activities, light activities, moderate to vigorous activities, walking, and running. A standard feed-forward Artificial Neural Network and a Deep Learning Ensemble Network were trained on features in the accelerometer data used in previous investigations (10th, 25th, 50th, 75th and 90th percentiles and the lag-one autocorrelation). Results Overall recognition accuracy for the standard feed forward Artificial Neural Network was 69.7%. Recognition accuracy for sedentary activities, light activities and games, moderate-to-vigorous activities, walking, and running was 82%, 79%, 64%, 36% and 46%, respectively. In comparison, overall recognition accuracy for the Deep Learning Ensemble Network was 82.6%. For sedentary activities, light activities and games, moderate-to-vigorous activities, walking, and running recognition accuracy was 84%, 91%, 79%, 73% and 73%, respectively. Conclusions Ensemble machine learning approaches such as Deep Learning Ensemble Network can accurately predict activity type from accelerometer data in preschool children.

Identification of New Zealand bats in flight from analysis of echolocation calls by artificial neural networks.

Time-expanded and heterodyned echolocation calls of the New Zealand long-tailed Chalinolobus tuberculatus and lesser short-tailed bat Mystacina tuberculata were recorded and digitally analysed. Temporal and spectral parameters were measured from time-expanded calls and power spectra generated for both time-expanded and heterodyned calls. Artificial neural networks were trained to classify the calls of both species using temporal and spectral parameters and power spectra as input data. Networks were then tested using data not previously seen. Calls could be unambiguously identified using parameters and power spectra from time-expanded calls. A neural network, trained and tested using power spectra of calls from both species recorded using a heterodyne detector set to 40 kHz (the frequency with the most energy of the fundamental of C. tuberculatus call), could identify 99% and 84% of calls of C. tuberculatus and M. tuberculata, respectively. A second network, trained and tested using power spectra of calls from both species recorded using a heterodyne detector set to 27 kHz (the frequency with the most energy of the fundamental of M. tuberculata call), could identify 34% and 100% of calls of C. tuberculatus and M. tuberculata, respectively. This study represents the first use of neural networks for the identification of bats from their echolocation calls. It is also the first study to use power spectra of time-expanded and heterodyned calls for identification of chiropteran species. The ability of neural networks to identify bats from their echolocation calls is discussed, as is the ecology of both species in relation to the design of their echolocation calls.

Acoustic identification of 12 species of echolocating bat by discriminant function analysis and artificial neural networks

We recorded echolocation calls from 14 sympatric species of bat in Britain. Once digitised, one temporal and four spectral features were measured from each call. The frequency-time course of each call was approximated by fitting eight mathematical functions, and the goodness of fit, represented by the mean-squared error, was calculated. Measurements were taken using an automated process that extracted a single call from background noise and measured all variables without intervention. Two species of Rhinolophus were easily identified from call duration and spectral measurements. For the remaining 12 species, discriminant function analysis and multilayer back-propagation perceptrons were used to classify calls to species level. Analyses were carried out with and without the inclusion of curve-fitting data to evaluate its usefulness in distinguishing among species. Discriminant function analysis achieved an overall correct classification rate of 79% with curve-fitting data included, while an artificial neural network achieved 87%. The removal of curve-fitting data improved the performance of the discriminant function analysis by 2 %, while the performance of a perceptron decreased by 2 %. However, an increase in correct identification rates when curve-fitting information was included was not found for all species. The use of a hierarchical classification system, whereby calls were first classified to genus level and then to species level, had little effect on correct classification rates by discriminant function analysis but did improve rates achieved by perceptrons. This is the first published study to use artificial neural networks to classify the echolocation calls of bats to species level. Our findings are discussed in terms of recent advances in recording and analysis technologies, and are related to factors causing convergence and divergence of echolocation call design in bats.

Pollen image classification using the classifynder system: Algorithm comparison and a case study on New Zealand honey

We describe an investigation into how Massey University’s Pollen Classifynder can accelerate the understanding of pollen and its role in nature. The Classifynder is an imaging microscopy system that can locate, image and classify slide based pollen samples. Given the laboriousness of purely manual image acquisition and identification it is vital to exploit assistive technologies like the Classifynder to enable acquisition and analysis of pollen samples. It is also vital that we understand the strengths and limitations of automated systems so that they can be used (and improved) to compliment the strengths and weaknesses of human analysts to the greatest extent possible. This article reviews some of our experiences with the Classifynder system and our exploration of alternative classifier models to enhance both accuracy and interpretability. Our experiments in the pollen analysis problem domain have been based on samples from the Australian National University’s pollen reference collection (2,890 grains, 15 species) and images bundled with the Classifynder system (400 grains, 4 species). These samples have been represented using the Classifynder image feature set.We additionally work through a real world case study where we assess the ability of the system to determine the pollen make-up of samples of New Zealand honey. In addition to the Classifynder’s native neural network classifier, we have evaluated linear discriminant, support vector machine, decision tree and random forest classifiers on these data with encouraging results. Our hope is that our findings will help enhance the performance of future releases of the Classifynder and other systems for accelerating the acquisition and analysis of pollen samples.

Design, construction and evaluation of systems to predict risk in obstetrics

We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy. We present a systematic, practical approach to developing risk prediction systems, suitable for use with large databases of medical information. An important part of this approach is a novel feature selection algorithm which uses the area under the receiver operating characteristic (ROC) curve to measure the expected discriminative power of different sets of predictor variables. We describe this algorithm and use it to select variables to predict risk of a specific adverse pregnancy outcome: failure to progress in labour. Neural network, logistic regression and hierarchical Bayesian risk prediction models are constructed, all of which achieve close to the limit of performance attainable on this prediction task. We show that better prediction performance requires more discriminative clinical information rather than improved modelling techniques. It is also shown that better diagnostic criteria in clinical records would greatly assist the development of systems to predict risk in pregnancy.