Variance-covariance based weighing for neural network ensembles

Neural network (NN) is a popular artificial intelligence technique for solving complicated problems due to their inherent capabilities. However generalization in NN can be harmed by a number of factors including parameter's initialization, inappropriate network topology and setting parameters of the training process itself. Forecast combinations of NN models have the potential for improved generalization and lower training time. A weighted averaging based on Variance-Covariance method that assigns greater weight to the forecasts producing lower error, instead of equal weights is practiced in this paper. While implementing the method, combination of forecasts is done with all candidate models in one experiment and with the best selected models in another experiment. It is observed during the empirical analysis that forecasting accuracy is improved by combining the best individual NN models. Another finding of this study is that reducing the number of NN models increases the diversity and, hence, accuracy.

Bayesian model averaging of load demand forecasts from neural network models

Creating a set of a number of neural network (NN) models in an ensemble and accumulating them can achieve better overview capability as compared to single neural network. Neural network ensembles are designed to provide solutions to particular problems. Many researchers and academicians have adopted this NN ensemble technique, especially in machine learning, and has been applied in various fields of engineering, medicine and information technology. This paper present a robust aggregation methodology for load demand forecasting based on Bayesian Model Averaging of a set of neural network models in an ensemble. This paper estimate a vector of coefficient for individual NN models' forecasts using validation data-set. These coefficients, also known as weights, are equal to posterior probabilities of the models generating the forecasts. These BMA weights are then used in combining forecasts generated from NN models with test data-set. By comparing the Bayesian results with the Simple Averaging method, it was observed that benefits are obtained by utilizing an advanced method like BMA for forecast combinations.

Neural network ensemble : evaluation of aggregation algorithms for electricity demand forecasting

This paper examines and analyzes different aggregation algorithms to improve accuracy of forecasts obtained using neural network (NN) ensembles. These algorithms include equal-weights combination of Best NN models, combination of trimmed forecasts, and Bayesian Model Averaging (BMA). The predictive performance of these algorithms are evaluated using Australian electricity demand data. The output of the aggregation algorithms of NN ensembles are compared with a Naive approach. Mean absolute percentage error is applied as the performance index for assessing the quality of aggregated forecasts. Through comprehensive simulations, it is found that the aggregation algorithms can significantly improve the forecasting accuracies. The BMA algorithm also demonstrates the best performance amongst aggregation algorithms investigated in this study.

Examining performance of aggregation algorithms for neural network-based electricity demand forecasting

The aim of this research is to examine the efficiency of different aggregation algorithms to the forecasts obtained from individual neural network (NN) models in an ensemble. In this study an ensemble of 100 NN models are constructed with a heterogeneous architecture. The outputs from NN models are combined by three different aggregation algorithms. These aggregation algorithms comprise of a simple average, trimmed mean, and a Bayesian model averaging. These methods are utilized with certain modifications and are employed on the forecasts obtained from all individual NN models. The output of the aggregation algorithms is analyzed and compared with the individual NN models used in NN ensemble and with a Naive approach. Thirty-minutes interval electricity demand data from Australian Energy Market Operator (AEMO) and the New York Independent System Operator's web site (NYISO) are used in the empirical analysis. It is observed that the aggregation algorithm perform better than many of the individual NN models. In comparison with the Naive approach, the aggregation algorithms exhibit somewhat better forecasting performance.

Neural network activity in the neonatal rat barrel cortex in vivo

Coordinated patterns of electrical activity are important for the early development of sensory systems. The spatiotemporal dynamics of these early activity patterns and the role of the peripheral sensory input for their generation are essentially unknown. There are two projects in this thesis. In project1, we performed extracellular multielectrode recordings in the somatosensory cortex of postnatal day 0 to 7 rats in vivo and observed three distinct patterns of synchronized oscillatory activity. (1) Spontaneous and periphery-driven spindle bursts of 1–2 s in duration and ~10 Hz in frequency occurred approximately every 10 s. (2) Spontaneous and sensory-driven gamma oscillations of 150–300 ms duration and 30–40 Hz in frequency occurred every 10–30 s. (3) Long oscillations appeared only every ~20 min and revealed the largest amplitude (250–750 µV) and longest duration (>40 s). These three distinct patterns of early oscillatory activity differently synchronized the neonatal cortical network. Whereas spindle bursts and gamma oscillations did not propagate and synchronized a local neuronal network of 200–400 µm in diameter, long oscillations propagated with 25–30 µm/s and synchronized 600-800 µm large ensembles. All three activity patterns were triggered by sensory activation. Single electrical stimulation of the whisker pad or tactile whisker activation elicited neocortical spindle bursts and gamma activity. Long oscillations could be only evoked by repetitive sensory stimulation. The neonatal oscillatory patterns in vivo depended on NMDAreceptor-mediated synaptic transmission and gap junctional coupling. Whereas spindle bursts and gamma oscillations may represent an early functional columnar-like pattern, long oscillations may serve as a propagating activation signal consolidating these immature neuronal networks. In project2, Using voltage-sensitive dye imaging and simultaneous multi-channel extracellular recordings in the barrel cortex and somatosensory thalamus of newborn rats in vivo, we found that spontaneous and whisker stimulation induced activity patterns were restricted to functional cortical columns already at the day of birth. Spontaneous and stimulus evoked cortical activity consisted of gamma oscillations followed by spindle bursts. Spontaneous events were mainly generated in the thalamus or by spontaneous whisker movements. Our findings indicate that during early developmental stages cortical networks self-organize in ontogenetic columns via spontaneous gamma oscillations triggered by the thalamus or sensory periphery.