Global optimisation of neural networks using a deterministic hybrid approach

Selection of the topology of a neural network and correct parameters for the learning algorithm is a tedious task for designing an optimal artificial neural network, which is smaller, faster and with a better generalization performance. In this paper we introduce a recently developed cutting angle method (a deterministic technique) for global optimization of connection weights. Neural networks are initially trained using the cutting angle method and later the learning is fine-tuned (meta-learning) using conventional gradient descent or other optimization techniques. Experiments were carried out on three time series benchmarks and a comparison was done using evolutionary neural networks. Our preliminary experimentation results show that the proposed deterministic approach could provide near optimal results much faster than the evolutionary approach.

Knowledge extraction from a mixed transfer function artificial neural network

One of the big problems with Artificial Neural Networks (ANN) is that their results are not intuitively clear. For example, if we use the traditional neurons, with a sigmoid activation function, we can approximate any function, including linear functions, but the coefficients (weights) in this approximation will be rather meaningless. To resolve this problem, this paper presents a novel kind of ANN with different transfer functions mixed together. The aim of such a network is to i) obtain a better generalization than current networks ii) to obtain knowledge from the networks without a sophisticated knowledge extraction algorithm iii) to increase the understanding and acceptance of ANNs. Transfer Complexity Ratio is defined to make a sense of the weights associated with the network. The paper begins with a review of the knowledge extraction from ANNs and then presents a Mixed Transfer Function Artificial Neural Network (MTFANN). A MTFANN contains different transfer functions mixed together rather than mono-transfer functions. This mixed presence has helped to obtain high level knowledge and similar generalization comparatively to monotransfer function nets in a global optimization context.

Handling of absorbent information by aggregation operators

We propose a generalization of the notion of the absorbent element of aggregation operators. Our construction involves tuples of values that are absorbent, that is, that decide the result of aggregation. We analyze some basic properties of this generalization and determine the absorbent tuples of some popular classes of aggregation operators.

Empirical generalisation as an inadequate cognitive scaffold to theoretical generalisation of a more complex concept

The impact of prior learning on new learning is highlighted by the case of Dean, a Year 8 student who developed his own method to find the sum of the interior angles of a polygon without knowing why his method worked. Enriched transcripts and visual displays of the cognitive, social (Dreyfus, Hershkowitz, & Schwarz, 2001) and affective elements (Williams, 2002) of Dean's interrupted abstraction process informed the identification of factors that inhibited Dean's constructing process. It was found Dean possessed an empirical, not theoretical, generalization (Davydov, 1990) about sums of interior angles of triangles that was an inadequate cognitive artifact for constructing the new more complex theoretical generalization. The study suggests use of tasks designed with the opportunity develop assumed knowledge in conjunction with new concepts.

A dimensionality reduction algorithm and its application for interactive visualization

Visualization is one of the most effective methods for analyzing how high-dimensional data are distributed. Dimensionality reduction techniques, such as PCA, can be used to map high dimensional data to a two- or three-dimensional space. In this paper, we propose an algorithm called HyperMap that can be effectively applied to visualization. Our algorithm can be seen as a generalization of FastMap. It preserves its linear computation complexity, and overcomes several main shortcomings, especially in visualization. Since there are more than two pivot objects in each axis of a target space, more distance information needs to be preserved in each dimension. Then in visualization, the number of pivot objects can go beyond the limitation of six (2-pivot objects × 3-dimensions). Our HyperMap algorithm also gives more flexibility to the target space, such that the data distribution can be observed from various viewpoints. Its effectiveness is confirmed by empirical evaluations on both real and synthetic datasets.

New ways to educate Asia-Pacific entrepreneurs

Asia-Pacific countries have huge numbers of young entrepreneurs. Yet the state of entrepreneurship education in this region has yet to come to grips with their needs. Elsewhere in the world, the growth and development in the curricula, textbooks, Websites and degree programs devoted to raising the level of enterprise and new venture creation has been remarkable.

The researcher undertook field study to examine best-practice models of enterprise education. He then carried out a content analysis of leading entrepreneurship textbooks to examine their applicability to the Asia-Pacific circumstance. Working with Thomson Learning Australia, he acquired the rights to re-write one leading textbook and entirely “asianised” it. He also produced a highly interactive Website to attract Internetnet savvy young entrepreneurs and students in the countries of the Asia-Pacific region.

The paper offers generalization that may prove helpful to educationalists and government policy planners about how to increase the supply of young people who launch their own businesses and social enterprises. The goal of this paper is to help universities in our region move toward launching entrepreneurship education in a relevant and interesting way.

Importance of the contribution of Australians to the penetration of China by the China Inland Mission in the period 1888-1953, with particular reference to the work of Australian women missionaries

Australians, in the main, are unaware of the role which Australia played in the evangelization of China in the late nineteenth and the first half of the twentieth century. Most would never have heard of the China Inland Mission (CIM), the largest of the Protestant bodies which penetrated the Middle Kingdom, and few would know of the contribution that its Australian contingent, which consistently comprised about a tenth of the CIM's numbers, made towards the Christianization of that vast country. This thesis aims to raise the level of awareness in this area. Academic researchers have not totally neglected to examine the proselytization of China, and historians of the stature of Latourette have not let it escape their attention. However, most of the studies which have not merely fleetingly focused on the subject while viewing a larger canvas, have been North American, singling out the efforts of United States and Canadian bodies in introducing Christianity to the Chinese. Here, authors like Amerding, Bacon, Creighton, Gates, Hawkes, Ho, Ko, Mensendiek, Michell and Quale have left their mark. In the case of the present thesis, the outlook from which events played out in China are viewed is firmly based in Australia rather than North America. Earlier Australian research has been scarce, and is dominated by Loane and Dixon. Loane, evidently primarily working from Australasian Council minutes, mainly concentrates on the efforts of the CIM's Home Council, examining its endeavours decade by decade against a backdrop of contemporaneous events in China, and briefly referring to aspects of the lives of a cross-section of Australasian missionaries, without providing much idea about what they actually did in the field or what they achieved there. Because of its preoccupation with the Home Council, which never admitted women into its ranks, Loane's treatise is systemically biased towards men, though the more prominent of the women, like Mary Reed and Susie Garland, are given due recognition. The current thesis looks in detail at what Australians did in the field, the level of success they achieved, and at the particular contribution of Australian women towards the evangelization of China. Dixon took upon herself the formidable task of examining the endeavours of all missions in China which contained Australian missionaries. Because of the magnitude of her task, she could not focus to any great extent on particular missions, nor pursue in any detail the work of individual Australian missionaries. Like Loane, she was unable to explore what they actually did in the field or what they achieved there. Neither could she delve to any depth into the work of Australian women missionaries, though on the basis of the information she had accumulated, she drew the conclusion that Australian women had largely only brought about some unintended feminist consequences amongst Chinese women. This sweeping generalization failed to take into account the other very real social changes for Chinese women the Australian female missionaries quite purposely helped to bring about, and this thesis makes good that omission. This thesis studies aspects of the Australian missionary endeavour which both Loane and Dixon have neglected, thereby breaking new ground, and sets out to correct erroneous impressions which Dixon's dissertation has left on the historical record. One of these impressions concerned the longevity of the effect of the Australian effort in China. She had the View, writing in 1978, that the Chinese Church was moribund (a view shared by Varg and Lacy) , and that therefore the effects attributable to the endeavours of any nationality had proved fruitless, whereas the author is able to show, using modern-day sources, that the church has burgeoned in recent years thanks to earlier missionary endeavours and later neo-evangelistic efforts like Gospel radio, and now has a complement of perhaps 50 million adherents, making it second only to the United States in the size of its Protestant evangelical population. Another impression she left was that the Australian input into the evangelization of China can be largely dismissed because no totally Australian organization emerged, leaving the direction of Australia's effort in other hands. Contrary to that impression, the author shows that the Australian impact in China was significant and that Australians enjoyed more power than Dixon ever imagined. The author also shows that Australians were accepted as the equal of other nationalities in the CIM once they had acquired the necessary field expertise, a factor which doubtless also applied in respect of other missions with Australian components in China. Marchant has suggested that it is a fiction perpetuated by mission periodicals that Christianity spread and progressed in a determined manner in China. This thesis establishes that within the CIM's bailiwick, though there was some patchiness, Christianity progressed steadily and inexorably. One mission alone, the CIM, is concentrated upon, firstly in order to render the data manageable, secondly because it was the largest mission in China and had a sizeable Australian (including female) contingent, and thirdly because it exemplified many of the problems which would have been faced by missions in that country and their Australian components. The methodology employed is multifaceted. The written testimony of the missionaries themselves, contained in CIM periodicals, Field Bulletins, Monthly Notes, Annual Reports, autobiographies, personal files, diaries and letters is used to illustrate various aspects of the CIM's work in which Australians were engaged. This approach is augmented by other sources such as China and Australasian Home Council Minutes, missionary conference reports, Candidates' Books, biographies, and other selected material from archival holdings in Australia, Singapore, the United Kingdom, America and Canada. Statistics, especially ratio analyses and growth rate comparisons are used to demonstrate the relative success of different missions, missionaries and genders. Also employed are reminiscences of missionaries and descendants obtained by personal interview, and these are aggregated to provide some general conclusions. Data from these various sources have been synthesized to serve the central objective of demonstrating the importance of the contribution of Australians to the penetration of China by the CIM in the period 1888-1953 with particular reference to the work of Australian women missionaries.

Fuzzy neural networks for control of dynamic systems

This thesis provides a unified and comprehensive treatment of the fuzzy neural networks as the intelligent controllers. This work has been motivated by a need to develop the solid control methodologies capable of coping with the complexity, the nonlinearity, the interactions, and the time variance of the processes under control. In addition, the dynamic behavior of such processes is strongly influenced by the disturbances and the noise, and such processes are characterized by a large degree of uncertainty. Therefore, it is important to integrate an intelligent component to increase the control system ability to extract the functional relationships from the process and to change such relationships to improve the control precision, that is, to display the learning and the reasoning abilities. The objective of this thesis was to develop a self-organizing learning controller for above processes by using a combination of the fuzzy logic and the neural networks. An on-line, direct fuzzy neural controller using the process input-output measurement data and the reference model with both structural and parameter tuning has been developed to fulfill the above objective. A number of practical issues were considered. This includes the dynamic construction of the controller in order to alleviate the bias/variance dilemma, the universal approximation property, and the requirements of the locality and the linearity in the parameters. Several important issues in the intelligent control were also considered such as the overall control scheme, the requirement of the persistency of excitation and the bounded learning rates of the controller for the overall closed loop stability. Other important issues considered in this thesis include the dependence of the generalization ability and the optimization methods on the data distribution, and the requirements for the on-line learning and the feedback structure of the controller. Fuzzy inference specific issues such as the influence of the choice of the defuzzification method, T-norm operator and the membership function on the overall performance of the controller were also discussed. In addition, the e-completeness requirement and the use of the fuzzy similarity measure were also investigated. Main emphasis of the thesis has been on the applications to the real-world problems such as the industrial process control. The applicability of the proposed method has been demonstrated through the empirical studies on several real-world control problems of industrial complexity. This includes the temperature and the number-average molecular weight control in the continuous stirred tank polymerization reactor, and the torsional vibration, the eccentricity, the hardness and the thickness control in the cold rolling mills. Compared to the traditional linear controllers and the dynamically constructed neural network, the proposed fuzzy neural controller shows the highest promise as an effective approach to such nonlinear multi-variable control problems with the strong influence of the disturbances and the noise on the dynamic process behavior. In addition, the applicability of the proposed method beyond the strictly control area has also been investigated, in particular to the data mining and the knowledge elicitation. When compared to the decision tree method and the pruned neural network method for the data mining, the proposed fuzzy neural network is able to achieve a comparable accuracy with a more compact set of rules. In addition, the performance of the proposed fuzzy neural network is much better for the classes with the low occurrences in the data set compared to the decision tree method. Thus, the proposed fuzzy neural network may be very useful in situations where the important information is contained in a small fraction of the available data.

Divide-the-Dollar game revisited

In the Divide-the-Dollar (DD) game, two players simultaneously make demands to divide a dollar. Each player receives his demand if the sum of the demands does not exceed one, a payoff of zero otherwise. Note that, in the latter case, both parties are punished severely. A major setback of DD is that each division of the dollar is a Nash equilibrium outcome. Observe that, when the sum of the two demands x and y exceeds one, it is as if Player 1's demand x (or his offer (1−x) to Player 2) suggests that Player 2 agrees to λx < 1 times his demand y so that Player 1's demand and Player 2's modified demand add up to exactly one; similarly, Player 2's demand y (or his offer (1−y) to Player 1) suggests that Player 1 agrees to λyx so that λyx+y = 1. Considering this fact, we change DD's payoff assignment rule when the sum of the demands exceeds one; here in this case, each player's payoff becomes his demand times his λ; i.e., each player has to make the sacrifice that he asks his opponent to make. We show that this modified version of DD has an iterated strict dominant strategy equilibrium in which each player makes the egalitarian demand 1/2. We also provide a natural N-person generalization of this procedure.

A particle swarm based hybrid system for imbalances medical data sampling

Background
Medical and biological data are commonly with small sample size, missing values, and most importantly, imbalanced class distribution. In this study we propose a particle swarm based hybrid system for remedying the class imbalance problem in medical and biological data mining. This hybrid system combines the particle swarm optimization (PSO) algorithm with multiple classifiers and evaluation metrics for evaluation fusion. Samples from the majority class are ranked using multiple objectives according to their merit in compensating the class imbalance, and then combined with the minority class to form a balanced dataset.

Results
One important finding of this study is that different classifiers and metrics often provide different evaluation results. Nevertheless, the proposed hybrid system demonstrates consistent improvements over several alternative methods with three different metrics. The sampling results also demonstrate good generalization on different types of classification algorithms, indicating the advantage of information fusion applied in the hybrid system.

Conclusion
The experimental results demonstrate that unlike many currently available methods which often perform unevenly with different datasets the proposed hybrid system has a better generalization property which alleviates the method-data dependency problem. From the biological perspective, the system provides indication for further investigation of the highly ranked samples, which may result in the discovery of new conditions or disease subtypes.

A multi-filter enhanced genetic ensemble system for gene selection and sample classification of microarray data

Background: Feature selection techniques are critical to the analysis of high dimensional datasets. This is especially true in gene selection from microarray data which are commonly with extremely high feature-to-sample ratio. In addition to the essential objectives such as to reduce data noise, to reduce data redundancy, to improve sample classification accuracy, and to improve model generalization property, feature selection also helps biologists to focus on the selected genes to further validate their biological hypotheses.
Results: In this paper we describe an improved hybrid system for gene selection. It is based on a recently proposed genetic ensemble (GE) system. To enhance the generalization property of the selected genes or gene subsets and to overcome the overfitting problem of the GE system, we devised a mapping strategy to fuse the goodness information of each gene provided by multiple filtering algorithms. This information is then used for initialization and mutation operation of the genetic ensemble system.
Conclusion: We used four benchmark microarray datasets (including both binary-class and multi-class classification problems) for concept proving and model evaluation. The experimental results indicate that the proposed multi-filter enhanced genetic ensemble (MF-GE) system is able to improve sample classification accuracy, generate more compact gene subset, and converge to the selection results more quickly. The MF-GE system is very flexible as various combinations of multiple filters and classifiers can be incorporated based on the data characteristics and the user preferences.

Activity recognition and abnormality detection with the switching hidden semi-Markov model

This paper addresses the problem of learning and recognizing human activities of daily living (ADL), which is an important research issue in building a pervasive and smart environment. In dealing with ADL, we argue that it is beneficial to exploit both the inherent hierarchical organization of the activities and their typical duration. To this end, we introduce the Switching Hidden Semi-Markov Model (S-HSMM), a two-layered extension of the hidden semi-Markov model (HSMM) for the modeling task. Activities are modeled in the S-HSMM in two ways: the bottom layer represents atomic activities and their duration using HSMMs; the top layer represents a sequence of high-level activities where each high-level activity is made of a sequence of atomic activities. We consider two methods for modeling duration: the classic explicit duration model using multinomial distribution, and the novel use of the discrete Coxian distribution. In addition, we propose an effective scheme to detect abnormality without the need for training on abnormal data. Experimental results show that the S-HSMM performs better than existing models including the flat HSMM and the hierarchical hidden Markov model in both classification and abnormality detection tasks, alleviating the need for presegmented training data. Furthermore, our discrete Coxian duration model yields better computation time and generalization error than the classic explicit duration model.

A hybrid neural network classifier combining ordered fuzzy ARTMAP and the dynamic decay adjustment algorithm

This paper presents a novel conflict-resolving neural network classifier that combines the ordering algorithm, fuzzy ARTMAP (FAM), and the dynamic decay adjustment (DDA) algorithm, into a unified framework. The hybrid classifier, known as Ordered FAMDDA, applies the DDA algorithm to overcome the limitations of FAM and ordered FAM in achieving a good generalization/performance. Prior to network learning, the ordering algorithm is first used to identify a fixed order of training patterns. The main aim is to reduce and/or avoid the formation of overlapping prototypes of different classes in FAM during learning. However, the effectiveness of the ordering algorithm in resolving overlapping prototypes of different classes is compromised when dealing with complex datasets. Ordered FAMDDA not only is able to determine a fixed order of training patterns for yielding good generalization, but also is able to reduce/resolve overlapping regions of different classes in the feature space for minimizing misclassification during the network learning phase. To illustrate the effectiveness of Ordered FAMDDA, a total of ten benchmark datasets are experimented. The results are analyzed and compared with those from FAM and Ordered FAM. The outcomes demonstrate that Ordered FAMDDA, in general, outperforms FAM and Ordered FAM in tackling pattern classification problems.

Misconceptions about density of decimals : insights from Indonesian pre-service teachers’ work

Extensive studies have documented various difficulties with, and misconceptions about, decimal numeration across different levels of education. This paper reports on pre-service teachers’ misconceptions about the density of decimals. Written test data from 140 Indonesian pre-service teachers, observation of group and classroom discussions provided evidence of pre-service teachers’ difficulties in grasping the density notion of decimals. This research was situated in a teacher education university in Yogyakarta, Indonesia. Incorrect analogies resulting from over generalization of knowledge about whole numbers and fractions were identified. Teaching ideas to resolve these difficulties and challenges in resolving pre-service teachers’ misconceptions are discussed. Evidence from this research indicates that it is possible to remove misconceptions about density of decimals.

The application of an ensemble of boosted elman networks to time series prediction : a benchmark study

In this paper, the application of multiple Elman neural networks to time series data regression problems is studied. An ensemble of Elman networks is formed by boosting to enhance the performance of the individual networks. A modified version of the AdaBoost algorithm is employed to integrate the predictions from multiple networks. Two benchmark time series data sets, i.e., the Sunspot and Box-Jenkins gas furnace problems, are used to assess the effectiveness of the proposed system. The simulation results reveal that an ensemble of boosted Elman networks can achieve a higher degree of generalization as well as performance than that of the individual networks. The results are compared with those from other learning systems, and implications of the performance are discussed.