Real-time control of a Tokamak plasma using neural networks

This paper presents results from the first use of neural networks for the real-time feedback control of high temperature plasmas in a Tokamak fusion experiment. The Tokamak is currently the principal experimental device for research into the magnetic confinement approach to controlled fusion. In the Tokamak, hydrogen plasmas, at temperatures of up to 100 Million K, are confined by strong magnetic fields. Accurate control of the position and shape of the plasma boundary requires real-time feedback control of the magnetic field structure on a time-scale of a few tens of microseconds. Software simulations have demonstrated that a neural network approach can give significantly better performance than the linear technique currently used on most Tokamak experiments. The practical application of the neural network approach requires high-speed hardware, for which a fully parallel implementation of the multi-layer perceptron, using a hybrid of digital and analogue technology, has been developed.

Combining the wavelet transform and forecasting models to predict gas forward prices

This paper presents a forecasting technique for forward energy prices, one day ahead. This technique combines a wavelet transform and forecasting models such as multi- layer perceptron, linear regression or GARCH. These techniques are applied to real data from the UK gas markets to evaluate their performance. The results show that the forecasting accuracy is improved significantly by using the wavelet transform. The methodology can be also applied to forecasting market clearing prices and electricity/gas loads.

Estimating conditional volatility with neural networks

It is well known that one of the obstacles to effective forecasting of exchange rates is heteroscedasticity (non-stationary conditional variance). The autoregressive conditional heteroscedastic (ARCH) model and its variants have been used to estimate a time dependent variance for many financial time series. However, such models are essentially linear in form and we can ask whether a non-linear model for variance can improve results just as non-linear models (such as neural networks) for the mean have done. In this paper we consider two neural network models for variance estimation. Mixture Density Networks (Bishop 1994, Nix and Weigend 1994) combine a Multi-Layer Perceptron (MLP) and a mixture model to estimate the conditional data density. They are trained using a maximum likelihood approach. However, it is known that maximum likelihood estimates are biased and lead to a systematic under-estimate of variance. More recently, a Bayesian approach to parameter estimation has been developed (Bishop and Qazaz 1996) that shows promise in removing the maximum likelihood bias. However, up to now, this model has not been used for time series prediction. Here we compare these algorithms with two other models to provide benchmark results: a linear model (from the ARIMA family), and a conventional neural network trained with a sum-of-squares error function (which estimates the conditional mean of the time series with a constant variance noise model). This comparison is carried out on daily exchange rate data for five currencies.

Neural network-based wind vector retrieval from satellite scatterometer data

Obtaining wind vectors over the ocean is important for weather forecasting and ocean modelling. Several satellite systems used operationally by meteorological agencies utilise scatterometers to infer wind vectors over the oceans. In this paper we present the results of using novel neural network based techniques to estimate wind vectors from such data. The problem is partitioned into estimating wind speed and wind direction. Wind speed is modelled using a multi-layer perceptron (MLP) and a sum of squares error function. Wind direction is a periodic variable and a multi-valued function for a given set of inputs; a conventional MLP fails at this task, and so we model the full periodic probability density of direction conditioned on the satellite derived inputs using a Mixture Density Network (MDN) with periodic kernel functions. A committee of the resulting MDNs is shown to improve the results.

First year qualifying report: neural networks for extracting wind vectors from satellite scatterometer data

The ERS-1 Satellite was launched in July 1991 by the European Space Agency into a polar orbit at about km800, carrying a C-band scatterometer. A scatterometer measures the amount of radar back scatter generated by small ripples on the ocean surface induced by instantaneous local winds. Operational methods that extract wind vectors from satellite scatterometer data are based on the local inversion of a forward model, mapping scatterometer observations to wind vectors, by the minimisation of a cost function in the scatterometer measurement space.par This report uses mixture density networks, a principled method for modelling conditional probability density functions, to model the joint probability distribution of the wind vectors given the satellite scatterometer measurements in a single cell (the `inverse' problem). The complexity of the mapping and the structure of the conditional probability density function are investigated by varying the number of units in the hidden layer of the multi-layer perceptron and the number of kernels in the Gaussian mixture model of the mixture density network respectively. The optimal model for networks trained per trace has twenty hidden units and four kernels. Further investigation shows that models trained with incidence angle as an input have results comparable to those models trained by trace. A hybrid mixture density network that incorporates geophysical knowledge of the problem confirms other results that the conditional probability distribution is dominantly bimodal.par The wind retrieval results improve on previous work at Aston, but do not match other neural network techniques that use spatial information in the inputs, which is to be expected given the ambiguity of the inverse problem. Current work uses the local inverse model for autonomous ambiguity removal in a principled Bayesian framework. Future directions in which these models may be improved are given.

Neural network-based wind vector retrieval from satellite scatterometer data

Obtaining wind vectors over the ocean is important for weather forecasting and ocean modelling. Several satellite systems used operationally by meteorological agencies utilise scatterometers to infer wind vectors over the oceans. In this paper we present the results of using novel neural network based techniques to estimate wind vectors from such data. The problem is partitioned into estimating wind speed and wind direction. Wind speed is modelled using a multi-layer perceptron (MLP) and a sum of squares error function. Wind direction is a periodic variable and a multi-valued function for a given set of inputs; a conventional MLP fails at this task, and so we model the full periodic probability density of direction conditioned on the satellite derived inputs using a Mixture Density Network (MDN) with periodic kernel functions. A committee of the resulting MDNs is shown to improve the results.

Short-term electricity demand and gas price forecasts using wavelet transforms and adaptive models

This paper presents some forecasting techniques for energy demand and price prediction, one day ahead. These techniques combine wavelet transform (WT) with fixed and adaptive machine learning/time series models (multi-layer perceptron (MLP), radial basis functions, linear regression, or GARCH). To create an adaptive model, we use an extended Kalman filter or particle filter to update the parameters continuously on the test set. The adaptive GARCH model is a new contribution, broadening the applicability of GARCH methods. We empirically compared two approaches of combining the WT with prediction models: multicomponent forecasts and direct forecasts. These techniques are applied to large sets of real data (both stationary and non-stationary) from the UK energy markets, so as to provide comparative results that are statistically stronger than those previously reported. The results showed that the forecasting accuracy is significantly improved by using the WT and adaptive models. The best models on the electricity demand/gas price forecast are the adaptive MLP/GARCH with the multicomponent forecast; their MSEs are 0.02314 and 0.15384 respectively.

The impact of architecture on the performance of artificial neural networks

A number of researchers have investigated the impact of network architecture on the performance of artificial neural networks. Particular attention has been paid to the impact on the performance of the multi-layer perceptron of architectural issues, and the use of various strategies to attain an optimal network structure. However, there are still perceived limitations with the multi-layer perceptron and networks that employ a different architecture to the multi-layer perceptron have gained in popularity in recent years, particularly, networks that implement a more localised solution, where the solution in one area of the problem space does not impact, or has a minimal impact, on other areas of the space. In this study, we discuss the major architectural issues affecting the performance of a multi-layer perceptron, before moving on to examine in detail the performance of a new localised network, namely the bumptree. The work presented here examines the impact on the performance of artificial neural networks of employing alternative networks to the long established multi-layer perceptron. In particular, networks that impose a solution where the impact of each parameter in the final network architecture has a localised impact on the problem space being modelled are examined. The alternatives examined are the radial basis function and bumptree neural networks, and the impact of architectural issues on the performance of these networks is examined. Particular attention is paid to the bumptree, with new techniques for both developing the bumptree structure and employing this structure to classify patterns being examined.

A comprehensive study of bankruptcy prediction: Accounting ratios, market variables, and microstructure

Bankruptcy prediction has been a fruitful area of research. Univariate analysis and discriminant analysis were the first methodologies used. While they perform relatively well at correctly classifying bankrupt and nonbankrupt firms, their predictive ability has come into question over time. Univariate analysis lacks the big picture that financial distress entails. Multivariate discriminant analysis requires stringent assumptions that are violated when dealing with accounting ratios and market variables. This has led to the use of more complex models such as neural networks. While the accuracy of the predictions has improved with the use of more technical models, there is still an important point missing. Accounting ratios are the usual discriminating variables used in bankruptcy prediction. However, accounting ratios are backward-looking variables. At best, they are a current snapshot of the firm. Market variables are forward-looking variables. They are determined by discounting future outcomes. Microstructure variables, such as the bid-ask spread, also contain important information. Insiders are privy to more information that the retail investor, so if any financial distress is looming, the insiders should know before the general public. Therefore, any model in bankruptcy prediction should include market and microstructure variables. That is the focus of this dissertation. The traditional models and the newer, more technical models were tested and compared to the previous literature by employing accounting ratios, market variables, and microstructure variables. Our findings suggest that the more technical models are preferable, and that a mix of accounting and market variables are best at correctly classifying and predicting bankrupt firms. Multi-layer perceptron appears to be the most accurate model following the results. The set of best discriminating variables includes price, standard deviation of price, the bid-ask spread, net income to sale, working capital to total assets, and current liabilities to total assets.

An intelligent decision support system for leukaemia diagnosis using microscopic blood images

This research proposes an intelligent decision support system for acute lymphoblastic leukaemia diagnosis from microscopic blood images. A novel clustering algorithm with stimulating discriminant measures (SDM) of both within- and between-cluster scatter variances is proposed to produce robust segmentation of nucleus and cytoplasm of lymphocytes/lymphoblasts. Specifically, the proposed between-cluster evaluation is formulated based on the trade-off of several between-cluster measures of well-known feature extraction methods. The SDM measures are used in conjuction with Genetic Algorithm for clustering nucleus, cytoplasm, and background regions. Subsequently, a total of eighty features consisting of shape, texture, and colour information of the nucleus and cytoplasm sub-images are extracted. A number of classifiers (multi-layer perceptron, Support Vector Machine (SVM) and Dempster-Shafer ensemble) are employed for lymphocyte/lymphoblast classification. Evaluated with the ALL-IDB2 database, the proposed SDM-based clustering overcomes the shortcomings of Fuzzy C-means which focuses purely on within-cluster scatter variance. It also outperforms Linear Discriminant Analysis and Fuzzy Compactness and Separation for nucleus-cytoplasm separation. The overall system achieves superior recognition rates of 96.72% and 96.67% accuracies using bootstrapping and 10-fold cross validation with Dempster-Shafer and SVM, respectively. The results also compare favourably with those reported in the literature, indicating the usefulness of the proposed SDM-based clustering method.

Investigation on the effect of cutting fluid pressure on surface quality measurement in high speed thread milling of brass alloy (C3600) and aluminium alloy (5083)

The quality of a machined finish plays a major role in the performance of milling operations, good surface quality can significantly improve fatigue strength, corrosion resistance, or creep behaviour as well as surface friction. In this study, the effect of cutting parameters and cutting fluid pressure on the quality measurement of the surface of the crest for threads milled during high speed milling operations has been scrutinised. Cutting fluid pressure, feed rate and spindle speed were the input parameters whilst minimising surface roughness on the crest of the thread was the target. The experimental study was designed using the Taguchi L32 array. Analysing and modelling the effective parameters were carried out using both a multi-layer perceptron (MLP) and radial basis function (RBF) artificial neural networks (ANNs). These were shown to be highly adept for such tasks. In this paper, the analysis of surface roughness at the crest of the thread in high speed thread milling using a high accuracy optical profile-meter is an original contribution to the literature. The experimental results demonstrated that the surface quality in the crest of the thread was improved by increasing cutting speed, feed rate ranging 0.41-0.45 m/min and cutting fluid pressure ranging 2-3.5 bars. These outcomes characterised the ANN as a promising application for surface profile modelling in precision machining.

Lung cancer prediction from microarray data by gene expression programming

Lung cancer is a leading cause of cancer-related death worldwide. The early diagnosis of cancer has demonstrated to be greatly helpful for curing the disease effectively. Microarray technology provides a promising approach of exploiting gene profiles for cancer diagnosis. In this study, the authors propose a gene expression programming (GEP)-based model to predict lung cancer from microarray data. The authors use two gene selection methods to extract the significant lung cancer related genes, and accordingly propose different GEP-based prediction models. Prediction performance evaluations and comparisons between the authors' GEP models and three representative machine learning methods, support vector machine, multi-layer perceptron and radial basis function neural network, were conducted thoroughly on real microarray lung cancer datasets. Reliability was assessed by the cross-data set validation. The experimental results show that the GEP model using fewer feature genes outperformed other models in terms of accuracy, sensitivity, specificity and area under the receiver operating characteristic curve. It is concluded that GEP model is a better solution to lung cancer prediction problems.

Design of multi-frequency microstrip antennas using multiple rings

An electromagnetically coupled feed arrangement is proposed for simultaneously exciting multiple concentric ring antennas for multi-frequency operation. This has a multi-layer dielectric configuration in which a transmission line is embedded below the layer containing radiating rings. Energy coupled to these rings from the line beneath is optimised by suitably adjusting the location and dimensions of stubs on the line. It has been shown that the resonant frequencies of these rings do not change as several of these single-frequency antennas are combined to form a multi-resonant antenna. Furthermore, all radiators are forced to operate at their primary mode and some harmonics of the lower resonant frequency rings appearing within the frequency range are suppressed when combined. The experimental prototype antenna has three resonant frequencies at which it has good radiation characteristics.

Oscillatory Instability of Rayleigh-Marangoni-Benard Convection in Two-Layer System

The convective instabilities in two or more superposed layers heated from below were studied extensively by many scientists due to several interfacial phenomena in nature and crystal growth application. Most works of them were performed mainly on the instability behaviors induced only by buoyancy force, especially on the oscillatory behavior at onset of convection (see Gershuni et. Al.(1982), Renardy et. Al. (1985,2000), Rasenat et. Al. (1989), and Colinet et. Al.(1994)) . But the unstable situations of multi-layer liquid convection will become more complicated and interesting while considering at the same time the buoyancy effect combined with thermocapillary effect. This is the case in the gravity reduced field or thin liquid layer where the thermocapillary effect is as important as buoyancy effect. The objective of this study was to investigate theoretically the interaction between Rayleigh-Bénard instability and pure Marangoni instability in a two-layer system, and more attention focus on the oscillatory instability both at the onset of convection and with increasing supercriticality. Oscillatory behavious of Rayleigh-Marangoni-Bénard convective instability (R-M-B instability) and flow patterns are presented in the two-layer system of Silicon Oil (10cSt) over Fluorinert (FC70) for a larger various range of two-layer depth ratios (Hr=Hupper/Hdown) from 0.2 to 5.0. Both linear instability analysis and 2D numerical simulation (A=L/H=10) show that the instability of the system depends strongly on the depth ratio of two-layer liquids. The oscillatory instability regime at the onset of R-M-B convection are found theoretically in different regions of layer thickness ratio for different two-layer depth H=12,6,4,3mm. The neutral stability curve of the system displaces to right while we consider the Marangoni effect at the interface in comparison with the Rayleigh-Bénard instability of the system without the Marangoni effect (Ma=0). The numerical results show different regimes of the developing of convection in the two-layer system for different thickness ratios and some differences at the onset of pure Marangoni convection and the onset of Rayleigh-Bénard convections in two-layer liquids. Both traveling wave and standing wave were detected in the oscillatory instability regime due to the competition between Rayleigh-Bénard instability and Marangoni effect. The mechanism of the standing wave formation in the system is presented numerically in this paper. The oscillating standing wave results in the competition of the intermediate Marangoni cell and the Rayleigh convective rolls. In the two-layer system of 47v2 silicone oil over water, a transition form the steady instability to the oscillatory instability of the Rayleigh-Marangoni-Bénard Convection was found numerically above the onset of convection for ε=0.9 and Hr=0.5. We propose that this oscillatory mechanism is possible to explain the experimental observation of Degen et. Al.(1998). Experimental work in comparison with our theoretical findings on the two-layer Rayleigh-Marangoni-Bénard convection with thinner depth for H<6mm will be carried out in the near future, and more attention will be paid to new oscillatory instability regimes possible in the influence of thermocapillary effects on the competition of two-layer liquids

Silicon-on-insulating multi-layers for total-dose irradiation hardness

Silicon-on-insulating multi-layer (SOIM) materials were fabricated by co-implantation of oxygen and nitrogen ions with different energies and doses. The multilayer microstructure was investigated by cross-sectional transmission electron microscopy. P-channel metal-oxide-semiconductor (PMOS) transistors and metal-semiconductor-insulator-semiconductor (MSIS) capacitors were produced by these materials. After the irradiated total dose reaches 3 x 10(5) rad (Si), the threshold voltage of the SOIM-based PMOS transistor only shifts 0.07 V, while thin silicon-on-insulating buried-oxide SIMOX-based PMOS transistors have a shift of 1.2V, where SIMOX represents the separated by implanted oxygen. The difference of capacitance of the SOIM-based MSIS capacitors before and after irradiation is less than that of the thin-box SIMOX-based MSIS capacitor. The results suggest that the SOIM materials have a more remarkable irradiation tolerance of total dose effect, compared to the thin-buried-oxide SIMOX materials.