Incremental learning from stream data

Recent years have witnessed an incredibly increasing interest in the topic of incremental learning. Unlike conventional machine learning situations, data flow targeted by incremental learning becomes available continuously over time. Accordingly, it is desirable to be able to abandon the traditional assumption of the availability of representative training data during the training period to develop decision boundaries. Under scenarios of continuous data flow, the challenge is how to transform the vast amount of stream raw data into information and knowledge representation, and accumulate experience over time to support future decision-making process. In this paper, we propose a general adaptive incremental learning framework named ADAIN that is capable of learning from continuous raw data, accumulating experience over time, and using such knowledge to improve future learning and prediction performance. Detailed system level architecture and design strategies are presented in this paper. Simulation results over several real-world data sets are used to validate the effectiveness of this method.

Computerised Diagnostic Decision Support System for the Classification of Pre-Invasive Cervical Squamous Lesions.

Previous studies have revealed considerable interobserver and intraobserver variation in the histological classification of preinvasive cervical squamous lesions. The aim of the present study was to develop a decision support system (DSS) for the histological interpretation of these lesions. Knowledge and uncertainty were represented in the form of a Bayesian belief network that permitted the storage of diagnostic knowledge and, for a given case, the collection of evidence in a cumulative manner that provided a final probability for the possible diagnostic outcomes. The network comprised 8 diagnostic histological features (evidence nodes) that were each independently linked to the diagnosis (decision node) by a conditional probability matrix. Diagnostic outcomes comprised normal; koilocytosis; and cervical intraepithelial neoplasia (CIN) 1, CIN II, and CIN M. For each evidence feature, a set of images was recorded that represented the full spectrum of change for that feature. The system was designed to be interactive in that the histopathologist was prompted to enter evidence into the network via a specifically designed graphical user interface (i-Path Diagnostics, Belfast, Northern Ireland). Membership functions were used to derive the relative likelihoods for the alternative feature outcomes, the likelihood vector was entered into the network, and the updated diagnostic belief was computed for the diagnostic outcomes and displayed. A cumulative probability graph was generated throughout the diagnostic process and presented on screen. The network was tested on 50 cervical colposcopic biopsy specimens, comprising 10 cases each of normal, koilocytosis, CIN 1, CIN H, and CIN III. These had been preselected by a consultant gynecological pathologist. Using conventional morphological assessment, the cases were classified on 2 separate occasions by 2 consultant and 2 junior pathologists. The cases were also then classified using the DSS on 2 occasions by the 4 pathologists and by 2 medical students with no experience in cervical histology. Interobserver and intraobserver agreement using morphology and using the DSS was calculated with K statistics. Intraobserver reproducibility using conventional unaided diagnosis was reasonably good (kappa range, 0.688 to 0.861), but interobserver agreement was poor (kappa range, 0.347 to 0.747). Using the DSS improved overall reproducibility between individuals. Using the DSS, however, did not enhance the diagnostic performance of junior pathologists when comparing their DSS-based diagnosis against an experienced consultant. However, the generation of a cumulative probability graph also allowed a comparison of individual performance, how individual features were assessed in the same case, and how this contributed to diagnostic disagreement between individuals. Diagnostic features such as nuclear pleomorphism were shown to be particularly problematic and poorly reproducible. DSSs such as this therefore not only have a role to play in enhancing decision making but also in the study of diagnostic protocol, education, self-assessment, and quality control. (C) 2003 Elsevier Inc. All rights reserved.

High precision C-14 dating of Holocene peat deposits: A comparison of Bayesian calibration and wiggle-matching approaches

The chronologies of five northern European ombrotrophic peat bogs subjected to a large ANIS C-14 dating effort (32-44 dates/site) are presented here. The results of Bayesian calibration (BCal) of dates with a prior assumption of chronological ordering were compared with a Bayesian wiggle-match approach (Bpeat) which assumes constant linear accumulation over sections of the peat profile. Interpolation of BCal age estimates of dense sequences of C-14 dates showed variable patterns of peat accumulation with time, with changes in accumulation occurring at intervals ranging from 20 to 50 cm. Within these intervals, peat accumulation appeared to be relatively linear. Close analysis suggests that some of the inferred variations in accumulation rate were related to the plant macrofossil composition of the peat. The wiggle-matched age-depth models had relatively high chronological uncertainty within intervals of closely spaced 14 C dates, suggesting that the premise of constant linear accumulation over large sections of the peat profile is unrealistic. Age models based on the assumption of linear accumulation over large parts of a peat core (and therefore only effective over millennial timescales), are not compatible with studies examining environmental change during the Holocene, where variability often occurs at decadal to centennial time-scales. Ideally, future wiggle-match age models should be constrained, with boundaries between sections based on the plant macrofossil composition of the peat and physical-chemical parameters such as the degree of decomposition. Strategies for the selection of material for dating should be designed so that there should be enough C-14 dates to accurately reconstruct the peat accumulation rate of each homogeneous stratigraphic unit. (c) 2006 Elsevier Ltd. All rights reserved.

Constraining the age of Lateglacial and early Holocene pollen zones and tephra horizons in southern Sweden with Bayesian probability methods

The sediment sequence from Hasseldala port in southeastern Sweden provides a unique Lateglacial/early Holocene record that contains five different tephra layers. Three of these have been geochemically identified as the Borrobol Tephra, the Hasseldalen Tephra and the 10-ka Askja Tephra. Twenty-eight high-resolution C-14 measurements have been obtained and three different age models based on Bayesian statistics are employed to provide age estimates for the five different tephra layers. The chrono- and pollen stratigraphic framework supports the stratigraphic position of the Borrobol Tephra as found in Sweden at the very end of the Older Dryas pollen zone and provides the first age estimates for the Askja and Hasseldalen tephras. Our results, however, highlight the limitations that arise in attempting to establish a robust, chronologically independent lacustrine sequence that can be correlated in great detail to ice core or marine records. Radiocarbon samples are prone to error and sedimentation rates in lake basins may vary considerably due to a number of factors. Any type of valid and 'realistic' age model, therefore, has to take these limitations into account and needs to include this information in its prior assumptions. As a result, the age ranges for the specific horizons at Hasseldala port are large and calendar year estimates differ according to the assumptions of the age-model. Not only do these results provide a cautionary note for overdependence on one age-model for the derivation of age estimates for specific horizons, but they also demonstrate that precise correlations to other palaeoarchives to detect leads or lags is problematic. Given the uncertainties associated with establishing age-depth models for sedimentary sequences spanning the Lateglacial period, however, this exercise employing Bayesian probability methods represents the best possible approach and provides the most statistically significant age estimates for the pollen zone boundaries and tephra horizons. Copyright (C) 2006 John Wiley & Sons, Ltd.

An Affordance Based Approach to Decision Making in Sport: Discussing a Novel Methodological Framework

Decision making is a fundamental clement of any sport, particularly open, fast, dynamic team sports such as football, basketball and rugby. At the elite level, athletes appear to consistently make good decisions in situations that are highly temporally constrained. To further understand how this is done has been the aim of researchers within the perception-action field for several decades. The purpose of this article is to present novel contributions, both theoretical and methodological, that are pushing the boundaries of this area of research. The theoretical framework (Ecological psychology) within which the work is posited will be described, followed by a description of Virtual Reality (VR) technology and how it relates to the theoretical aims. Finally, an applied example will be summarised in order to demonstrate how the theoretical approach and the methodological approach come together in practice.

Risk-informed decision-making in the presence of epistemic uncertainty

An important issue in risk analysis is the distinction between epistemic and aleatory uncertainties. In this paper, the use of distinct representation formats for aleatory and epistemic uncertainties is advocated, the latter being modelled by sets of possible values. Modern uncertainty theories based on convex sets of probabilities are known to be instrumental for hybrid representations where aleatory and epistemic components of uncertainty remain distinct. Simple uncertainty representation techniques based on fuzzy intervals and p-boxes are used in practice. This paper outlines a risk analysis methodology from elicitation of knowledge about parameters to decision. It proposes an elicitation methodology where the chosen representation format depends on the nature and the amount of available information. Uncertainty propagation methods then blend Monte Carlo simulation and interval analysis techniques. Nevertheless, results provided by these techniques, often in terms of probability intervals, may be too complex to interpret for a decision-maker and we, therefore, propose to compute a unique indicator of the likelihood of risk, called confidence index. It explicitly accounts for the decisionmaker’s attitude in the face of ambiguity. This step takes place at the end of the risk analysis process, when no further collection of evidence is possible that might reduce the ambiguity due to epistemic uncertainty. This last feature stands in contrast with the Bayesian methodology, where epistemic uncertainties on input parameters are modelled by single subjective probabilities at the beginning of the risk analysis process.

PROSTATIC INTRAEPITHELIAL NEOPLASIA (PIN) - PERFORMANCE OF BAYESIAN BELIEF NETWORK FOR DIAGNOSIS AND GRADING

Prostatic intraepithelial neoplasia (PIN) diagnosis and grading are affected by uncertainties which arise from the fact that almost all knowledge of PIN histopathology is expressed in concepts, descriptive linguistic terms, and words. A Bayesian belief network (BBN) was therefore used to reduce the problem of uncertainty in diagnostic clue assessment, while still considering the dependences between elements in the reasoning sequence. A shallow network was used with an open-tree topology, with eight first-level descendant nodes for the diagnostic clues (evidence nodes), each independently linked by a conditional probability matrix to a root node containing the diagnostic alternatives (decision node). One of the evidence nodes was based on the tissue architecture and the others were based on cell features. The system was designed to be interactive, in that the histopathologist entered evidence into the network in the form of likelihood ratios for outcomes at each evidence node. The efficiency of the network was tested on a series of 110 prostate specimens, subdivided as follows: 22 cases of non-neoplastic prostate or benign prostatic tissue (NP), 22 PINs of low grade (PINlow), 22 PINs of high grade (PINhigh), 22 prostatic adenocarcinomas with cribriform pattern (PACcri), and 22 prostatic adenocarcinomas with large acinar pattern (PAClgac). The results obtained in the benign and malignant categories showed that the belief for the diagnostic alternatives is very high, the values being in general more than 0.8 and often close to 1.0. When considering the PIN lesions, the network classified and graded most of the cases with high certainty. However, there were some cases which showed values less than 0.8 (13 cases out of 44), thus indicating that there are situations in which the feature changes are intermediate between contiguous categories or grades. Discrepancy between morphological grading and the BBN results was observed in four out of 44 PIN cases: one PINlow was classified as PINhigh and three PINhigh were classified as PINlow. In conclusion, the network can grade PlN lesions and differentiate them from other prostate lesions with certainty. In particular, it offers a descriptive classifier which is readily implemented and which allows the use of linguistic, fuzzy variables.

EXPERT-SYSTEM SUPPORT USING BAYESIAN BELIEF NETWORKS IN THE DIAGNOSIS OF FINE-NEEDLE ASPIRATION BIOPSY SPECIMENS OF THE BREAST

Aim-To develop an expert system model for the diagnosis of fine needle aspiration cytology (FNAC) of the breast.

Methods-Knowledge and uncertainty were represented in the form of a Bayesian belief network which permitted the combination of diagnostic evidence in a cumulative manner and provided a final probability for the possible diagnostic outcomes. The network comprised 10 cytological features (evidence nodes), each independently linked to the diagnosis (decision node) by a conditional probability matrix. The system was designed to be interactive in that the cytopathologist entered evidence into the network in the form of likelihood ratios for the outcomes at each evidence node.

Results-The efficiency of the network was tested on a series of 40 breast FNAC specimens. The highest diagnostic probability provided by the network agreed with the cytopathologists' diagnosis in 100% of cases for the assessment of discrete, benign, and malignant aspirates. A typical probably benign cases were given probabilities in favour of a benign diagnosis. Suspicious cases tended to have similar probabilities for both diagnostic outcomes and so, correctly, could not be assigned as benign or malignant. A closer examination of cumulative belief graphs for the diagnostic sequence of each case provided insight into the diagnostic process, and quantitative data which improved the identification of suspicious cases.

Conclusion-The further development of such a system will have three important roles in breast cytodiagnosis: (1) to aid the cytologist in making a more consistent and objective diagnosis; (2) to provide a teaching tool on breast cytological diagnosis for the non-expert; and (3) it is the first stage in the development of a system capable of automated diagnosis through the use of expert system machine vision.

Approximate credal network updating by linear programming with applications to decision making

Credal nets are probabilistic graphical models which extend Bayesian nets to cope with sets of distributions. An algorithm for approximate credal network updating is presented. The problem in its general formulation is a multilinear optimization task, which can be linearized by an appropriate rule for fixing all the local models apart from those of a single variable. This simple idea can be iterated and quickly leads to accurate inferences. A transformation is also derived to reduce decision making in credal networks based on the maximality criterion to updating. The decision task is proved to have the same complexity of standard inference, being NPPP-complete for general credal nets and NP-complete for polytrees. Similar results are derived for the E-admissibility criterion. Numerical experiments confirm a good performance of the method.

Decision Making by Credal Nets

Credal nets are probabilistic graphical models which extend Bayesian nets to cope with sets of distributions. This feature makes the model particularly suited for the implementation of classifiers and knowledge-based systems. When working with sets of (instead of single) probability distributions, the identification of the optimal option can be based on different criteria, some of them eventually leading to multiple choices. Yet, most of the inference algorithms for credal nets are designed to compute only the bounds of the posterior probabilities. This prevents some of the existing criteria from being used. To overcome this limitation, we present two simple transformations for credal nets which make it possible to compute decisions based on the maximality and E-admissibility criteria without any modification in the inference algorithms. We also prove that these decision problems have the same complexity of standard inference, being NP^PP-hard for general credal nets and NP-hard for polytrees.

The Inferential Complexity of Bayesian and Credal Networks

This paper presents new results on the complexity of graph-theoretical models that represent probabilities (Bayesian networks) and that represent interval and set valued probabilities (credal networks). We define a new class of networks with bounded width, and introduce a new decision problem for Bayesian networks, the maximin a posteriori. We present new links between the Bayesian and credal networks, and present new results both for Bayesian networks (most probable explanation with observations, maximin a posteriori) and for credal networks (bounds on probabilities a posteriori, most probable explanation with and without observations, maximum a posteriori).

Towards a knowledge-based approach for effective decision-making in railway safety

Purpose – This paper aims to contribute towards understanding how safety knowledge can be elicited from railway experts for the purposes of supporting effective decision-making. Design/methodology/approach – A consortium of safety experts from across the British railway industry is formed. Collaborative modelling of the knowledge domain is used as an approach to the elicitation of safety knowledge from experts. From this, a series of knowledge models is derived to inform decision-making. This is achieved by using Bayesian networks as a knowledge modelling scheme, underpinning a Safety Prognosis tool to serve meaningful prognostics information and visualise such information to predict safety violations. Findings – Collaborative modelling of safety-critical knowledge is a valid approach to knowledge elicitation and its sharing across the railway industry. This approach overcomes some of the key limitations of existing approaches to knowledge elicitation. Such models become an effective tool for prediction of safety cases by using railway data. This is demonstrated using passenger–train interaction safety data. Practical implications – This study contributes to practice in two main directions: by documenting an effective approach to knowledge elicitation and knowledge sharing, while also helping the transport industry to understand safety. Social implications – By supporting the railway industry in their efforts to understand safety, this research has the potential to benefit railway passengers, staff and communities in general, which is a priority for the transport sector. Originality/value – This research applies a knowledge elicitation approach to understanding safety based on collaborative modelling, which is a novel approach in the context of transport.