Data-Driven Text Generation using Neural Networks & Provenance is Complicated and Boring — Is there a solution?

Title: Data-Driven Text Generation using Neural Networks Speaker: Pavlos Vougiouklis, University of Southampton Abstract: Recent work on neural networks shows their great potential at tackling a wide variety of Natural Language Processing (NLP) tasks. This talk will focus on the Natural Language Generation (NLG) problem and, more specifically, on the extend to which neural network language models could be employed for context-sensitive and data-driven text generation. In addition, a neural network architecture for response generation in social media along with the training methods that enable it to capture contextual information and effectively participate in public conversations will be discussed. Speaker Bio: Pavlos Vougiouklis obtained his 5-year Diploma in Electrical and Computer Engineering from the Aristotle University of Thessaloniki in 2013. He was awarded an MSc degree in Software Engineering from the University of Southampton in 2014. In 2015, he joined the Web and Internet Science (WAIS) research group of the University of Southampton and he is currently working towards the acquisition of his PhD degree in the field of Neural Network Approaches for Natural Language Processing. Title: Provenance is Complicated and Boring — Is there a solution? Speaker: Darren Richardson, University of Southampton Abstract: Paper trails, auditing, and accountability — arguably not the sexiest terms in computer science. But then you discover that you've possibly been eating horse-meat, and the importance of provenance becomes almost palpable. Having accepted that we should be creating provenance-enabled systems, the challenge of then communicating that provenance to casual users is not trivial: users should not have to have a detailed working knowledge of your system, and they certainly shouldn't be expected to understand the data model. So how, then, do you give users an insight into the provenance, without having to build a bespoke system for each and every different provenance installation? Speaker Bio: Darren is a final year Computer Science PhD student. He completed his undergraduate degree in Electronic Engineering at Southampton in 2012.

Predicting metal-binding site residues in low-resolution structural models

The accurate prediction of the biochemical function of a protein is becoming increasingly important, given the unprecedented growth of both structural and sequence databanks. Consequently, computational methods are required to analyse such data in an automated manner to ensure genomes are annotated accurately. Protein structure prediction methods, for example, are capable of generating approximate structural models on a genome-wide scale. However, the detection of functionally important regions in such crude models, as well as structural genomics targets, remains an extremely important problem. The method described in the current study, MetSite, represents a fully automatic approach for the detection of metal-binding residue clusters applicable to protein models of moderate quality. The method involves using sequence profile information in combination with approximate structural data. Several neural network classifiers are shown to be able to distinguish metal sites from non-sites with a mean accuracy of 94.5%. The method was demonstrated to identify metal-binding sites correctly in LiveBench targets where no obvious metal-binding sequence motifs were detectable using InterPro. Accurate detection of metal sites was shown to be feasible for low-resolution predicted structures generated using mGenTHREADER where no side-chain information was available. High-scoring predictions were observed for a recently solved hypothetical protein from Haemophilus influenzae, indicating a putative metal-binding site.

Complex-valued B-spline neural networks for modeling and inverting Hammerstein systems

Many communication signal processing applications involve modelling and inverting complex-valued (CV) Hammerstein systems. We develops a new CV B-spline neural network approach for efficient identification of the CV Hammerstein system and effective inversion of the estimated CV Hammerstein model. Specifically, the CV nonlinear static function in the Hammerstein system is represented using the tensor product from two univariate B-spline neural networks. An efficient alternating least squares estimation method is adopted for identifying the CV linear dynamic model’s coefficients and the CV B-spline neural network’s weights, which yields the closed-form solutions for both the linear dynamic model’s coefficients and the B-spline neural network’s weights, and this estimation process is guaranteed to converge very fast to a unique minimum solution. Furthermore, an accurate inversion of the CV Hammerstein system can readily be obtained using the estimated model. In particular, the inversion of the CV nonlinear static function in the Hammerstein system can be calculated effectively using a Gaussian-Newton algorithm, which naturally incorporates the efficient De Boor algorithm with both the B-spline curve and first order derivative recursions. The effectiveness of our approach is demonstrated using the application to equalisation of Hammerstein channels.

Intelligent animal fiber classification with artificial neural networks

Artificial neural networks (ANN) are increasingly used to solve many problems related to pattern recognition and object classification. In this paper, we report on a study using artificial neural networks to classify two kinds of animal fibers: merino and mohair. We have developed two different models, one extracting nine scale parameters with image processing, and the other using an unsupervised artificial neural network to extract features automatically, which are determined in accordance with the complexity of the scale structure and the accuracy of the model. Although the first model can achieve higher accuracy, it requires more effort for image processing and more prior knowledge, since the accuracy of the ANN largely depends on the parameters selected. The second model is more robust than the first, since only raw images are used. Because only ordinary optical images taken with a microscope are employed, we can use the approach for many textile applications without expensive equipment such as scanning electron microscopy.

Solving regression problems using competitive ensemble models

The use of ensemble models in many problem domains has increased significantly in the last fewyears. The ensemble modeling, in particularly boosting, has shown a great promise in improving predictive performance of a model. Combining the ensemble members is normally done in a co-operative fashion where each of the ensemble members performs the same task and their predictions are aggregated to obtain the improved performance. However, it is also possible to combine the ensemble members in a competitive fashion where the best prediction of a relevant ensemble member is selected for a particular input. This option has been previously somewhat overlooked. The aim of this article is to investigate and compare the competitive and co-operative approaches to combining the models in the ensemble. A comparison is made between a competitive ensemble model and that of MARS with bagging, mixture of experts, hierarchical mixture of experts and a neural network ensemble over several public domain regression problems that have a high degree of nonlinearity and noise. The empirical results showa substantial advantage of competitive learning versus the co-operative learning for all the regression problems investigated. The requirements for creating the efficient ensembles and the available guidelines are also discussed.

Predicting the distribution of discrete spatial events using artificial neural networks

Although the development of geographic information system (GIS) technology and digital data manipulation techniques has enabled practitioners in the geographical and geophysical sciences to make more efficient use of resource information, many of the methods used in forming spatial prediction models are still inherently based on traditional techniques of map stacking in which layers of data are combined under the guidance of a theoretical domain model. This paper describes a data-driven approach by which Artificial Neural Networks (ANNs) can be trained to represent a function characterising the probability that an instance of a discrete event, such as the presence of a mineral deposit or the sighting of an endangered animal species, will occur over some grid element of the spatial area under consideration. A case study describes the application of the technique to the task of mineral prospectivity mapping in the Castlemaine region of Victoria using a range of geological, geophysical and geochemical input variables. Comparison of the maps produced using neural networks with maps produced using a density estimation-based technique demonstrates that the maps can reliably be interpreted as representing probabilities. However, while the neural network model and the density estimation-based model yield similar results under an appropriate choice of values for the respective parameters, the neural network approach has several advantages, especially in high dimensional input spaces.

Australian forex market anaylsis using connectionist models

The need for intelligent monitoring systems has become a necessity to keep track of the complex forex market. The forex market is difficult to understand by an average individual. However, once the market is broken down into simple terms, the average individual can begin to understand the foreign exchange market and use it as a financial instrument for future investing. This paper is an attempt to compare the performance of a Takagi-Sugeno type neuro-fuzzy system and a feed forward neural network trained using the scaled conjugate gradient algorithm to predict the average monthly forex rates. The exchange values of Australian dollar are considered with respect to US dollar, Singapore dollar, New Zealand dollar, Japanese yen and United Kingdom pound. The connectionist models were trained using 70% of the data and remaining was used for testing and validation purposes. It is observed that the proposed connectionist models were able to predict the average forex rates one month ahead accurately. Experiment results also reveal that neuro-fuzzy technique performed better than the neural network.

Design and degradation modelling through artificial neural networks

Automotive is one of the major manufacturing industries in Australia that requires extensive reliability test for the components used in vehicles. To achieve a shorter time-to-market and a highly reliable product while reducing the amount of physical prototyping, there is a growing need for better understanding on the effect that the design parameters have on the degradation of the product. This paper presents comprehensive descriptions of applying Artificial Neural Network (ANN) to capture the relationships between design and degradation. Consequently, two models of different practical significance are created as the result of the work. The vision of the models is to be used by the testers and designers as a guideline in design evaluation, so that time-consuming and expensive iterations of the product developmental cycle can be reduced substantially. The degradation of the folding force of a mechanical system is used to illustrate our approach.

Chaos theory based detection against network mimicking DDoS attacks

DDoS attack traffic is difficult to differentiate from legitimate network traffic during transit from the attacker, or zombies, to the victim. In this paper, we use the theory of network self-similarity to differentiate DDoS flooding attack traffic from legitimate self-similar traffic in the network. We observed that DDoS traffic causes a strange attractor to develop in the pattern of network traffic. From this observation, we developed a neural network detector trained by our DDoS prediction algorithm. Our preliminary experiments and analysis indicate that our proposed chaotic model can accurately and effectively detect DDoS attack traffic. Our approach has the potential to not only detect attack traffic during transit, but to also filter it.

Mixed transfer function neural networks for knowledge acquistition

Modeling helps to understand and predict the outcome of complex systems. Inductive modeling methodologies are beneficial for modeling the systems where the uncertainties involved in the system do not permit to obtain an accurate physical model. However inductive models, like artificial neural networks (ANNs), may suffer from a few drawbacks involving over-fitting and the difficulty to easily understand the model itself. This can result in user reluctance to accept the model or even complete rejection of the modeling results. Thus, it becomes highly desirable to make such inductive models more comprehensible and to automatically determine the model complexity to avoid over-fitting. In this paper, we propose a novel type of ANN, a mixed transfer function artificial neural network (MTFANN), which aims to improve the complexity fitting and comprehensibility of the most popular type of ANN (MLP - a Multilayer Perceptron).

Load forecasting and neural networks : a prediction interval-based perspective

Successfully determining competitive optimal schedules for electricity generation intimately hinges on the forecasts of loads. The nonstationarity and high volatility of loads make their accurate prediction somewhat problematic. Presence of uncertainty in data also significantly degrades accuracy of point predictions produced by deterministic load forecasting models. Therefore, operation planning utilizing these predictions will be unreliable. This paper aims at developing prediction intervals rather than producing exact point prediction. Prediction intervals are theatrically more reliable and practical than predicted values. The delta and Bayesian techniques for constructing prediction intervals for forecasted loads are implemented here. To objectively and comprehensively assess quality of constructed prediction intervals, a new index based on length and coverage probability of prediction intervals is developed. In experiments with real data, and through calculation of global statistics, it is shown that neural network point prediction performance is unreliable. In contrast, prediction intervals developed using the delta and Bayesian techniques are satisfactorily narrow, with a high coverage probability.

Modelling optimal risk allocation in PPP projects using artificial neural networks

This paper aims to establish, train, validate, and test artificial neural network (ANN) models for modelling risk allocation decision-making process in public-private partnership (PPP) projects, mainly drawing upon transaction cost economics. An industry-wide questionnaire survey was conducted to examine the risk allocation practice in PPP projects and collect the data for training the ANN models. The training and evaluation results, when compared with those of using traditional MLR modelling technique, show that the ANN models are satisfactory for modelling risk allocation decision-making process. The empirical evidence further verifies that it is appropriate to utilize transaction cost economics to interpret risk allocation decision-making process. It is recommended that, in addition to partners' risk management mechanism maturity level, decision-makers, both from public and private sectors, should also seriously consider influential factors including partner's risk management routines, partners' cooperation history, partners' risk management commitment, and risk management environmental uncertainty. All these factors influence the formation of optimal risk allocation strategies, either by their individual or interacting effects.

Hybrid neural-evolutionary model for electricity price forecasting

Evolving artificial neural networks has attracted much attention among researchers recently, especially in the fields where plenty of data exist but explanatory theories and models are lacking or based upon too many simplifying assumptions. Financial time series forecasting is one of them. A hybrid model is used to forecast the hourly electricity price from the California Power Exchange. A collaborative approach is adopted to combine ANN and evolutionary algorithm. The main contributions of this thesis include: Investigated the effect of changing values of several important parameters on the performance of the model, and selected the best combination of these parameters; good forecasting results have been obtained with the implemented hybrid model when the best combination of parameters is used. The lowest MAPE through a single run is 5. 28134%. And the lowest averaged MAPE over 10 runs is 6.088%, over 30 runs is 6.786%; through the investigation of the parameter period, it is found that by including future values of the homogenous moments of the instant being forecasted into the input vector, forecasting accuracy is greatly enhanced. A comparison of results with other works reported in the literature shows that the proposed model gives superior performance on the same data set.

Integration of supervised ART-based neural networks with a hybrid genetic algorithm

In this paper, two evolutionary artificial neural network (EANN) models that are based on integration of two supervised adaptive resonance theory (ART)-based artificial neural networks with a hybrid genetic algorithm (HGA) are proposed. The search process of the proposed EANN models is guided by a knowledge base established by ART with respect to the training data samples. The EANN models explore the search space for “coarse” solutions, and such solutions are then refined using the local search process of the HGA. The performances of the proposed EANN models are evaluated and compared with those from other classifiers using more than ten benchmark data sets. The applicability of the EANN models to a real medical classification task is also demonstrated. The results from the experimental studies demonstrate the effectiveness and usefulness of the proposed EANN models in undertaking pattern classification problems.

Development of a speaker recognition system using wavelets and artificial neural networks

This paper addresses the problem of speaker recognition from speech signals. The study focuses on the development of a speaker recognition system comprising two modules: a wavelet-based feature extractor, and a neural-network-based classifier. We have conducted a number of experiments to investigate the applicability of Discrete Wavelet Transform (D WT) in extracting discriminative features from the speech signals, and have examined various models from the Adaptive Resonance Theory (ART) family of neural networks in classijjing the extracted features. The results indicate that DWT could be a potential feature extraction tool for speaker recognition. In addition, the ART-based classijiers have yielded very promising recognition accuracy at more than 81%.