Machine learning classification of surgical pathology reports and chunk recognition for information extraction noise reduction

Background and aims: Machine learning techniques for the text mining of cancer-related clinical documents have not been sufficiently explored. Here some techniques are presented for the pre-processing of free-text breast cancer pathology reports, with the aim of facilitating the extraction of information relevant to cancer staging.

Materials and methods: The first technique was implemented using the freely available software RapidMiner to classify the reports according to their general layout: ‘semi-structured’ and ‘unstructured’. The second technique was developed using the open source language engineering framework GATE and aimed at the prediction of chunks of the report text containing information pertaining to the cancer morphology, the tumour size, its hormone receptor status and the number of positive nodes. The classifiers were trained and tested respectively on sets of 635 and 163 manually classified or annotated reports, from the Northern Ireland Cancer Registry.

Results: The best result of 99.4% accuracy – which included only one semi-structured report predicted as unstructured – was produced by the layout classifier with the k nearest algorithm, using the binary term occurrence word vector type with stopword filter and pruning. For chunk recognition, the best results were found using the PAUM algorithm with the same parameters for all cases, except for the prediction of chunks containing cancer morphology. For semi-structured reports the performance ranged from 0.97 to 0.94 and from 0.92 to 0.83 in precision and recall, while for unstructured reports performance ranged from 0.91 to 0.64 and from 0.68 to 0.41 in precision and recall. Poor results were found when the classifier was trained on semi-structured reports but tested on unstructured.

Conclusions: These results show that it is possible and beneficial to predict the layout of reports and that the accuracy of prediction of which segments of a report may contain certain information is sensitive to the report layout and the type of information sought.

Speech Recognition with unknown partial feature corruption - a review of the union model

This paper provides a summary of our studies on robust speech recognition based on a new statistical approach – the probabilistic union model. We consider speech recognition given that part of the acoustic features may be corrupted by noise. The union model is a method for basing the recognition on the clean part of the features, thereby reducing the effect of the noise on recognition. To this end, the union model is similar to the missing feature method. However, the two methods achieve this end through different routes. The missing feature method usually requires the identity of the noisy data for noise removal, while the union model combines the local features based on the union of random events, to reduce the dependence of the model on information about the noise. We previously investigated the applications of the union model to speech recognition involving unknown partial corruption in frequency band, in time duration, and in feature streams. Additionally, a combination of the union model with conventional noise-reduction techniques was studied, as a means of dealing with a mixture of known or trainable noise and unknown unexpected noise. In this paper, a unified review, in the context of dealing with unknown partial feature corruption, is provided into each of these applications, giving the appropriate theory and implementation algorithms, along with an experimental evaluation.

The application of a Trous wave filtering and Monte Carlo analysis on secis 2001 solar eclipse observations

Eight thousand images of the solar corona were captured during the June 2001 total solar eclipse. New software for the alignment of the images and an automated technique for detecting intensity oscillations using multi-scale wavelet analysis were developed. Large areas of the images covered by the Moon and the upper corona were scanned for oscillations and the statistical properties of the atmospheric effects were determined. The a Trous wavelet transform was used for noise reduction and Monte Carlo analysis as a significance test of the detections. The effectiveness of those techniques is discussed in detail.

A frequency measure robust to linear filtering

It is noted that the determination of an oscillation frequency by used of the power spectrum of measured time series is susceptible to filtering of the signal. Similarly, frequency measurements made by period counting can yield different, results depending on how the signal is filtered for noise reduction. In an attempt to eliminate these ambiguities, a new measure of frequency, based on an approximate reconstruction of the phase-space trajectory of the oscillator from the signal, is introduced. This measure is shown to be invariant under linear filtering. For this reason, it is also inaccessible by spectral methods. The effect of filtering on frequency for weakly nonlinear, noisy oscillators, to which this definition applies only imperfectly, is quantified. This work provides the theoretical basis for frequency measurements employing MIRVA filtering.

Analyzing laser plasma interferograms with a continuous wavelet transform ridge extraction technique: the method

Laser plasma interferograms are currently analyzed by extraction of the phase-shift map with fast Fourier transform (FFT) techniques [Appl. Opt. 18, 3101 (1985)]. This methodology works well when interferograms are only marginally affected by noise and reduction of fringe visibility, but it can fail to produce accurate phase-shift maps when low-quality images are dealt with. We present a novel procedure for a phase-shift map computation that makes extensive use of the ridge extraction in the continuous wavelet transform (CWT) framework. The CWT tool is flexible because of the wide adaptability of the analyzing basis, and it can be accurate because of the intrinsic noise reduction in the ridge extraction. A comparative analysis of the accuracy performances of them new tool and the FFT-based one shows that the CWT-based tool produces phase maps considerably less noisy and that it can better resolve local inhomogeneties. (C) 2001 Optical Society of America.