Evolutionary structure learning algorithm for Bayesian network and penalized mutual information metric

This paper formulates the problem of learning Bayesian network structures from data as determining the structure that best approximates the probability distribution indicated by the data. A new metric, Penalized Mutual Information metric, is proposed, and a evolutionary algorithm is designed to search for the best structure among alternatives. The experimental results show that this approach is reliable and promising.

An objective function based on Bayesian likelihoods of necessity and sufficiency for concept learning in the absence of labeled counter-examples

Supervised machine learning techniques generally require that the training set on which learning is based contain sufficient examples representative of the target concept, as well as known counter-examples of the concept; however, in many application domains it is not possible to supply a set of labeled counter-examples. This paper proposes an objective function based on Bayesian likelihoods of necessity and sufficiency. This function can be used to guide search towards the discovery of a concept description given only a set of labeled positive examples of the target concept, and as a corpus of unlabeled examples. Results of experiments performed on several datasets from the VCI repository show that the technique achieves comparable accuracy to conventional supervised learning techniques, despite the fact that the latter require a set of labeled counter-examples to be supplied. The technique can be applied in many domains in which the provision of labeled counter-examples is problematic.

Comparison of MLP and Bayesian approaches on mineral prospectivity mapping tasks

Mineral Prospectivity Mapping is the process of combining maps containing different geoscientific data sets to produce a single map depicting areas ranked according to their potential to host mineral deposits of a particular type. This paper outlines two approaches for deriving a function which can be used to assign to each cell in the study area a value representing the posterior probability that the cell contains a deposit of the sought-after mineral. One approach is based on estimating probability density functions (pdfs); the second uses multilayer perceptrons (MLPs). Results are provided from applying these approaches to geoscientific datasets covering a region in North Western Victoria, Australia. The results demonstrate that while both the Bayesian approach and the MLP approach yield similar results when the number of input dimensions is small, the Bayesian approach rapidly becomes unstable as the number of input dimensions increases, with the resulting maps displaying high sensitivity to the number of mixtures used to model the distributions. However, despite the fact that Bayesian assigned values cannot be interpreted as posterior probabilities in high dimensional input spaces, the pixel favorability rankings produced by the two methods is similar.

Bayesian clustering with AutoClass explicitly recognises uncertainties in landscape classification

Clustering of multivariate data is a commonly used technique in ecology, and many approaches to clustering are available. The results from a clustering algorithm are uncertain, but few clustering approaches explicitly acknowledge this uncertainty. One exception is Bayesian mixture modelling, which treats all results probabilistically, and allows comparison of multiple plausible classifications of the same data set. We used this method, implemented in the AutoClass program, to classify catchments (watersheds) in the Murray Darling Basin (MDB), Australia, based on their physiographic characteristics (e.g. slope, rainfall, lithology). The most likely classification found nine classes of catchments. Members of each class were aggregated geographically within the MDB. Rainfall and slope were the two most important variables that defined classes. The second-most likely classification was very similar to the first, but had one fewer class. Increasing the nominal uncertainty of continuous data resulted in a most likely classification with five classes, which were again aggregated geographically. Membership probabilities suggested that a small number of cases could be members of either of two classes. Such cases were located on the edges of groups of catchments that belonged to one class, with a group belonging to the second-most likely class adjacent. A comparison of the Bayesian approach to a distance-based deterministic method showed that the Bayesian mixture model produced solutions that were more spatially cohesive and intuitively appealing. The probabilistic presentation of results from the Bayesian classification allows richer interpretation, including decisions on how to treat cases that are intermediate between two or more classes, and whether to consider more than one classification. The explicit consideration and presentation of uncertainty makes this approach useful for ecological investigations, where both data and expectations are often highly uncertain.

Bayesian classification of catchments using spatial data: a first step to improved modelling of catchment effects on stream ecological condition

A major challenge facing freshwater ecologists and managers is the development of models that link stream ecological condition to catchment scale effects, such as land use. Previous attempts to make such models have followed two general approaches. The bottom-up approach employs mechanistic models, which can quickly become too complex to be useful. The top-down approach employs empirical models derived from large data sets, and has often suffered from large amounts of unexplained variation in stream condition.

We believe that the lack of success of both modelling approaches may be at least partly explained by scientists considering too wide a breadth of catchment type. Thus, we believe that by stratifying large sets of catchments into groups of similar types prior to modelling, both types of models may be improved. This paper describes preliminary work using a Bayesian classification software package, ‘Autoclass’ (Cheeseman and Stutz 1996) to create classes of catchments within the Murray Darling Basin based on physiographic data.

Autoclass uses a model-based classification method that employs finite mixture modelling and trades off model fit versus complexity, leading to a parsimonious solution. The software provides information on the posterior probability that the classification is ‘correct’ and also probabilities for alternative classifications. The importance of each attribute in defining the individual classes is calculated and presented, assisting description of the classes. Each case is ‘assigned’ to a class based on membership probability, but the probability of membership of other classes is also provided. This feature deals very well with cases that do not fit neatly into a larger class. Lastly, Autoclass requires the user to specify the measurement error of continuous variables.

Catchments were derived from the Australian digital elevation model. Physiographic data werederived from national spatial data sets. There was very little information on measurement errors for the spatial data, and so a conservative error of 5% of data range was adopted for all continuous attributes. The incorporation of uncertainty into spatial data sets remains a research challenge.

The results of the classification were very encouraging. The software found nine classes of catchments in the Murray Darling Basin. The classes grouped together geographically, and followed altitude and latitude gradients, despite the fact that these variables were not included in the classification. Descriptions of the classes reveal very different physiographic environments, ranging from dry and flat catchments (i.e. lowlands), through to wet and hilly catchments (i.e. mountainous areas). Rainfall and slope were two important discriminators between classes. These two attributes, in particular, will affect the ways in which the stream interacts with the catchment, and can thus be expected to modify the effects of land use change on ecological condition. Thus, realistic models of the effects of land use change on streams would differ between the different types of catchments, and sound management practices will differ.

A small number of catchments were assigned to their primary class with relatively low probability. These catchments lie on the boundaries of groups of catchments, with the second most likely class being an adjacent group. The locations of these ‘uncertain’ catchments show that the Bayesian classification dealt well with cases that do not fit neatly into larger classes.

Although the results are intuitive, we cannot yet assess whether the classifications described in this paper would assist the modelling of catchment scale effects on stream ecological condition. It is most likely that catchment classification and modelling will be an iterative process, where the needs of the model are used to guide classification, and the results of classifications used to suggest further refinements to models.

Evaluation of Bayesian belief networks for diagnosis of psychotic disorders

The thesis examined the inter-rater reliability and procedural validity of four computerised Bayesian belief networks (BBNs) which were developed to assist with the diagnosis of psychotic disorders. The results of this research indicated that BBNs can significantly improve diagnostic reliability and may represent an important advance over current diagnostic methods. The professional portfolio investigated, through the presentation of case studies and review of literature relevant to each case study, how comorbidity and context of depression may impact on cognitive behavioural therapy treatment.

Learning other agents' preferences in multi-agent negotiation using the bayesian classifier

In multi-agent systems, most of the time, an agent does not have complete information about the preferences and decision making processes of other agents. This prevents even the cooperative agents from making coordinated choices, purely due to their ignorance of what others want. To overcome this problem, traditional coordination methods rely heavily on inter-agent communication, and thus become very inefficient when communication is costly or simply not desirable (e.g. to preserve privacy). In this paper, we propose the use of learning to complement communication in acquiring knowledge about other agents. We augment the communication-intensive negotiating agent architecture with a learning module, implemented as a Bayesian classifier. This allows our agents to incrementally update models of other agents' preferences from past negotiations with them. Based on these models, the agents can make sound predictions about others' preferences, thus reducing the need for communication in their future interactions.

Bayesian networks and decision trees in the diagnosis of female urinary incontinence

This study compares the effectiveness of Bayesian networks versus Decision Trees in modeling the Integral Theory of Female Urinary Incontinence diagnostic algorithm. Bayesian networks and Decision Trees were developed and trained using data from 58 adult women presenting with urinary incontinence symptoms. A Bayesian Network was developed in collaboration with an expert specialist who regularly utilizes a non-automated diagnostic algorithm in clinical practice. The original Bayesian network was later refined using a more connected approach. Diagnoses determined from all automated approaches were compared with the diagnoses of a single human expert. In most cases, Bayesian networks were found to be at least as accurate as the Decision Tree approach. The refined Connected Bayesian Network was found to be more accurate than the Original Bayesian Network accurately discriminated between diagnoses despite the small sample size. In contrast, the Connected and Decision Tree approaches were less able to discriminate between diagnoses. The Original Bayesian Network was found to provide an excellent basis for graphically communicating the correlation between symptoms and laxity defects in a given anatomical zone. Performance measures in both networks indicate that Bayesian networks could provide a potentially useful tool in the management of female pelvic floor dysfunction. Before the technique can be utilized in practice, well-established learning algorithms should be applied to improve network structure. A larger training data set should also improve network accuracy, sensitivity, and specificity.

A Bayesian framework for learning shared and individual subspaces from multiple data sources

This paper presents a novel Bayesian formulation to exploit shared structures across multiple data sources, constructing foundations for effective mining and retrieval across disparate domains. We jointly analyze diverse data sources using a unifying piece of metadata (textual tags). We propose a method based on Bayesian Probabilistic Matrix Factorization (BPMF) which is able to explicitly model the partial knowledge common to the datasets using shared subspaces and the knowledge specific to each dataset using individual subspaces. For the proposed model, we derive an efficient algorithm for learning the joint factorization based on Gibbs sampling. The effectiveness of the model is demonstrated by social media retrieval tasks across single and multiple media. The proposed solution is applicable to a wider context, providing a formal framework suitable for exploiting individual as well as mutual knowledge present across heterogeneous data sources of many kinds.

A neural network-based multi-agent classifier system with a Bayesian formalism for trust measurement

In this paper, a neural network (NN)-based multi-agent classifier system (MACS) utilising the trust-negotiation-communication (TNC) reasoning model is proposed. A novel trust measurement method, based on the combination of Bayesian belief functions, is incorporated into the TNC model. The Fuzzy Min-Max (FMM) NN is used as learning agents in the MACS, and useful modifications of FMM are proposed so that it can be adopted for trust measurement. Besides, an auctioning procedure, based on the sealed bid method, is applied for the negotiation phase of the TNC model. Two benchmark data sets are used to evaluate the effectiveness of the proposed MACS. The results obtained compare favourably with those from a number of machine learning methods. The applicability of the proposed MACS to two industrial sensor data fusion and classification tasks is also demonstrated, with the implications analysed and discussed.