Data visualisation with missing data: A non-linear approach

Exploratory analysis of data in all sciences seeks to find common patterns to gain insights into the structure and distribution of the data. Typically visualisation methods like principal components analysis are used but these methods are not easily able to deal with missing data nor can they capture non-linear structure in the data. One approach to discovering complex, non-linear structure in the data is through the use of linked plots, or brushing, while ignoring the missing data. In this technical report we discuss a complementary approach based on a non-linear probabilistic model. The generative topographic mapping enables the visualisation of the effects of very many variables on a single plot, which is able to incorporate far more structure than a two dimensional principal components plot could, and deal at the same time with missing data. We show that using the generative topographic mapping provides us with an optimal method to explore the data while being able to replace missing values in a dataset, particularly where a large proportion of the data is missing.

Addressing missing data in geochemistry: a non-linear approach

Exploratory analysis of petroleum geochemical data seeks to find common patterns to help distinguish between different source rocks, oils and gases, and to explain their source, maturity and any intra-reservoir alteration. However, at the outset, one is typically faced with (a) a large matrix of samples, each with a range of molecular and isotopic properties, (b) a spatially and temporally unrepresentative sampling pattern, (c) noisy data and (d) often, a large number of missing values. This inhibits analysis using conventional statistical methods. Typically, visualisation methods like principal components analysis are used, but these methods are not easily able to deal with missing data nor can they capture non-linear structure in the data. One approach to discovering complex, non-linear structure in the data is through the use of linked plots, or brushing, while ignoring the missing data. In this paper we introduce a complementary approach based on a non-linear probabilistic model. Generative topographic mapping enables the visualisation of the effects of very many variables on a single plot, while also dealing with missing data. We show how using generative topographic mapping also provides an optimal method with which to replace missing values in two geochemical datasets, particularly where a large proportion of the data is missing.

Handling varying amounts of missing data when classifying mental-health risk levels

One of the main challenges of classifying clinical data is determining how to handle missing features. Most research favours imputing of missing values or neglecting records that include missing data, both of which can degrade accuracy when missing values exceed a certain level. In this research we propose a methodology to handle data sets with a large percentage of missing values and with high variability in which particular data are missing. Feature selection is effected by picking variables sequentially in order of maximum correlation with the dependent variable and minimum correlation with variables already selected. Classification models are generated individually for each test case based on its particular feature set and the matching data values available in the training population. The method was applied to real patients' anonymous mental-health data where the task was to predict the suicide risk judgement clinicians would give for each patient's data, with eleven possible outcome classes: zero to ten, representing no risk to maximum risk. The results compare favourably with alternative methods and have the advantage of ensuring explanations of risk are based only on the data given, not imputed data. This is important for clinical decision support systems using human expertise for modelling and explaining predictions.

GTM-based data visualisation with incomplete data

We analyse how the Generative Topographic Mapping (GTM) can be modified to cope with missing values in the training data. Our approach is based on an Expectation -Maximisation (EM) method which estimates the parameters of the mixture components and at the same time deals with the missing values. We incorporate this algorithm into a hierarchical GTM. We verify the method on a toy data set (using a single GTM) and a realistic data set (using a hierarchical GTM). The results show our algorithm can help to construct informative visualisation plots, even when some of the training points are corrupted with missing values.

Visualisation of heterogeneous data with the generalised generative topographic mapping

Heterogeneous and incomplete datasets are common in many real-world visualisation applications. The probabilistic nature of the Generative Topographic Mapping (GTM), which was originally developed for complete continuous data, can be extended to model heterogeneous (i.e. containing both continuous and discrete values) and missing data. This paper describes and assesses the resulting model on both synthetic and real-world heterogeneous data with missing values.

Bilateral visual outcomes and service utilization of patients treated for 3 years with ranibizumab for neovascular age-related macular degeneration

Background: The aim of this study was to describe bilateral visual outcomes and the effect of incomplete follow-up after 3 years of ranibizumab therapy for neovascular age-related macular degeneration. Secondarily, the demands on service provision over a 3-year period were described. Methods: Data on visual acuity, hospital visits, and injections were collected over 36 months on consecutive patients commencing treatment over a 9-month period. Visual outcome was determined for 1) all patients, using last observation carried forward for missed visits due to early discontinuation and 2) only those patients completing full 36-month follow-up. Results: Over 3 years, 120 patients cumulatively attended hospital for 1,823 noninjection visits and 1,365 injection visits. A visual acuity loss of <15 letters (L) was experienced by 78.2% of patients. For all patients (n=120), there was a mean loss of 1.68 L using last observation carried forward for missing values. Excluding five patients who died and 30 who discontinued follow-up, mean gain was 1.47 L. In bilateral cases, final acuity was on average 9 L better in second eyes compared to first eyes. Also, 91% of better-seeing eyes continued to be the better-seeing eye. Conclusion: We have demonstrated our approach to describing the long-term service provision and visual outcomes of ranibizumab therapy for neovascular age-related macular degeneration in a consecutive cohort of patients. Although there was a heavy burden with very frequent injections and clinic visits, patients can expect a good level of visual stability and a very high chance of maintaining their better-seeing eye for up to 3 years. © 2014 Chavan et al. This work is published by Dove Medical Press Limited.