An overview of interactive visual data mining techniques for knowledge discovery

n the past decade, the analysis of data has faced the challenge of dealing with very large and complex datasets and the real-time generation of data. Technologies to store and access these complex and large datasets are in place. However, robust and scalable analysis technologies are needed to extract meaningful information from these datasets. The research field of Information Visualization and Visual Data Analytics addresses this need. Information visualization and data mining are often used complementary to each other. Their common goal is the extraction of meaningful information from complex and possibly large data. However, though data mining focuses on the usage of silicon hardware, visualization techniques also aim to access the powerful image-processing capabilities of the human brain. This article highlights the research on data visualization and visual analytics techniques. Furthermore, we highlight existing visual analytics techniques, systems, and applications including a perspective on the field from the chemical process industry.

Artisanal mining's democratizing directions and deviations

Artisanal miners have tended to be portrayed in the literature and media as people who work hard and play hard, not infrequently depicted as ‘rough diamonds’ likely to cross the boundaries of appropriate behaviour through pursuit of wealth and flamboyant living, often at the cost of local environmental damage. A popular alternative image is that of marginalised labourers, driven by poverty to toil in harsh conditions and pursuing mining livelihoods in the face of national governments and large-scale mining companies’ subversion of their land and mineral rights. Both views reflect partial realities, but are inclined to exaggerate the position of miners as mischief-making rogues or victims. Through documentation of the multi-faceted nature of Tanzanian artisanal miners’ work and home lives during the country’s on-going economic mineralisation, we endeavour to convey a balanced rendering of their aspirations, occupational identity and social ties. Our emphasis is on their working lives as artisans, how they organise themselves and contend with the risks of their occupation, including their engagement with government policy and large-scale mining interests.

Distribution and mobility of trace elements in soils and vegetation around the mining and smelting areas of Tharsis, Riotinto and Huelva, Iberian Pyrite Belt, SW Spain

Trace elements may present an environmental hazard in the vicinity of mining and smelting activities. However, the factors controlling their distribution and transfer within the soil and vegetation systems are not always well defined. Total concentrations of up to 15,195 mg center dot kg (-1) As, 6,690 mg center dot kg(-1) Cu, 24,820 mg center dot kg(-1) Pb and 9,810 mg center dot kg(-1) Zn in soils, and 62 mg center dot kg(-1) As, 1,765 mg center dot kg(-1) Cu, 280 mg center dot kg(-1) Pb and 3,460 mg center dot kg (-1) Zn in vegetation were measured. However, unusually for smelters and mines of a similar size, the elevated trace element concentrations in soils were found to be restricted to the immediate vicinity of the mines and smelters (maximum 2-3 km). Parent material, prevailing wind direction, and soil physical and chemical characteristics were found to correlate poorly with the restricted trace element distributions in soils. Hypotheses are given for this unusual distribution: (1) the contaminated soils were removed by erosion or (2) mines and smelters released large heavy particles that could not have been transported long distances. Analyses of the accumulation of trace elements in vegetation (median ratios: As 0.06, Cu 0.19, Pb 0.54 and Zn 1.07) and the percentage of total trace elements being DTPA extractable in soils (median percentages: As 0.06%, Cu 15%, Pb 7% and Zn 4%) indicated higher relative trace element mobility in soils with low total concentrations than in soils with elevated concentrations.

Trace element partitioning and soil particle characterisation around mining and smelting areas at Tharsis, Riotinto and Huelva, SW Spain

Trace elements may present an environmental hazard in the vicinity of mining and smelting activities. However, the factors controlling trace element distribution in soils around ancient and modem mining and smelting areas are not always clear. Tharsis, Riotinto and Huelva are located in the Iberian Pyrite Belt in SW Spain. Tharsis and Riotinto mines have been exploited since 2500 B.C., with intensive smelting taking place. Huelva, established in 1970 and using the Flash Furnace Outokumpu process, is currently one of the largest smelter in the world. Pyrite and chalcopyrite ore have been intensively smelted for Cu. However, unusually for smelters and mines of a similar size, the elevated trace element concentrations in soils were found to be restricted to the immediate vicinity of the mines and smelters, being found up to a maximum of 2 kin from the mines and smelters at Tharsis, Riotinto and Huelva. Trace element partitioning (over 2/3 of trace elements found in the residual immobile fraction of soils at Tharsis) and soil particles examination by SEM-EDX showed that trace elements were not adsorbed onto soil particles, but were included within the matrix of large trace element-rich Fe silicate slag particles (i.e. 1 min circle divide at least 1 wt.% As, Cu and Zn, and 2 wt.% Pb). Slag particle large size (I mm 0) was found to control the geographically restricted trace element distribution in soils at Tharsis, Riotinto and Huelva, since large heavy particles could not have been transported long distances. Distribution and partitioning indicated that impacts to the environment as a result of mining and smelting should remain minimal in the region. (c) 2006 Elsevier B.V. All rights reserved.

A preliminary investigation into mining and smelting impacts on trace element concentrations in the soils and vegetation around Tharsis, SW Spain

Toxic trace elements present an environmental hazard in the vicinity of mining and smelting activities. However. the processes of transfer of these elements to groundwater and to plants are not always clear. Tharsis mine. in the Iberian pyrite belt (SW Spain), has been exploited since 2500 BC, with extensive smelting, taking place front the 1850S until the 1920s. Sixty four soil (mainly topsoils) and vegetation samples were collected in February 2001 and analysed by ICP-AES for 23 elements. Concentrations are 6-6300 mg kg(-1) As and 14-24800 mg kg(-1) Pb in soils, and 0.20-9 mg kg(-1) As and 2-195 mg Pb in vegetation. Trace element concentrations decrease rapidly away from the mine. with As and Pb concentrations in the range 6-1850 mg kg(-1) (median 22 mg kg(-1)) and 14-31 mg, kg(-1) (median 43 mg, kg(-1)), respectively, 1 km away from the mine. These concentrations are low when compared to other well-studied mining and smelting areas (e.g. 600 mg kg(-1) As at 8 km from Yellowknife smelter, Canada; >100 mg kg(-1) Pb over 270 km(2) around the Pb-Zn Port Pirie smelter. South Australia: mean of 1419 mg kg(-1) Pb around Aberystwyth smelter, Wales, UK). The high metal content of the vegetation and the low soil pH (mean pH 4.93) indicate the potential for trace element mobility which Could explain the relatively low concentration of metals in Tharsis topsoils and cause threats to plans to redevelop the Tharsis area as an orange plantation.

A geostatistical analysis of soil, vegetation, and image data characterizing land surface variation

The elucidation of spatial variation in the landscape can indicate potential wildlife habitats or breeding sites for vectors, such as ticks or mosquitoes, which cause a range of diseases. Information from remotely sensed data could aid the delineation of vegetation distribution on the ground in areas where local knowledge is limited. The data from digital images are often difficult to interpret because of pixel-to-pixel variation, that is, noise, and complex variation at more than one spatial scale. Landsat Thematic Mapper Plus (ETM+) and Satellite Pour l'Observation de La Terre (SPOT) image data were analyzed for an area close to Douna in Mali, West Africa. The variograms of the normalized difference vegetation index (NDVI) from both types of image data were nested. The parameters of the nested variogram function from the Landsat ETM+ data were used to design the sampling for a ground survey of soil and vegetation data. Variograms of the soil and vegetation data showed that their variation was anisotropic and their scales of variation were similar to those of NDVI from the SPOT data. The short- and long-range components of variation in the SPOT data were filtered out separately by factorial kriging. The map of the short-range component appears to represent the patterns of vegetation and associated shallow slopes and drainage channels of the tiger bush system. The map of the long-range component also appeared to relate to broader patterns in the tiger bush and to gentle undulations in the topography. The results suggest that the types of image data analyzed in this study could be used to identify areas with more moisture in semiarid regions that could support wildlife and also be potential vector breeding sites.

Dynamic load balancing for the distributed mining of molecular structures

In molecular biology, it is often desirable to find common properties in large numbers of drug candidates. One family of methods stems from the data mining community, where algorithms to find frequent graphs have received increasing attention over the past years. However, the computational complexity of the underlying problem and the large amount of data to be explored essentially render sequential algorithms useless. In this paper, we present a distributed approach to the frequent subgraph mining problem to discover interesting patterns in molecular compounds. This problem is characterized by a highly irregular search tree, whereby no reliable workload prediction is available. We describe the three main aspects of the proposed distributed algorithm, namely, a dynamic partitioning of the search space, a distribution process based on a peer-to-peer communication framework, and a novel receiverinitiated load balancing algorithm. The effectiveness of the distributed method has been evaluated on the well-known National Cancer Institute’s HIV-screening data set, where we were able to show close-to linear speedup in a network of workstations. The proposed approach also allows for dynamic resource aggregation in a non dedicated computational environment. These features make it suitable for large-scale, multi-domain, heterogeneous environments, such as computational grids.

A customizable multi-agent system for distributed data mining

We present a general Multi-Agent System framework for distributed data mining based on a Peer-to-Peer model. Agent protocols are implemented through message-based asynchronous communication. The framework adopts a dynamic load balancing policy that is particularly suitable for irregular search algorithms. A modular design allows a separation of the general-purpose system protocols and software components from the specific data mining algorithm. The experimental evaluation has been carried out on a parallel frequent subgraph mining algorithm, which has shown good scalability performances.

The role of satellite image-processing for national-scale estimates of gene flow from genetically modified crops: rapeseed in the UK as a model.

1. There is concern over the possibility of unwanted environmental change following transgene movement from genetically modified (GM) rapeseed Brassica napus to its wild and weedy relatives. 2. The aim of this research was to develop a remote sensing-assisted methodology to help quantify gene flow from crops to their wild relatives over wide areas. Emphasis was placed on locating sites of sympatry, where the frequency of gene flow is likely to be highest, and on measuring the size of rapeseed fields to allow spatially explicit modelling of wind-mediated pollen-dispersal patterns. 3. Remote sensing was used as a tool to locate rapeseed fields, and a variety of image-processing techniques was adopted to facilitate the compilation of a spatially explicit profile of sympatry between the crop and Brassica rapa. 4. Classified satellite images containing rapeseed fields were first used to infer the spatial relationship between donor rapeseed fields and recipient riverside B. rapa populations. Such images also have utility for improving the efficiency of ground surveys by identifying probable sites of sympatry. The same data were then also used for the calculation of mean field size. 5. This paper forms a companion paper to Wilkinson et al. (2003), in which these elements were combined to produce a spatially explicit profile of hybrid formation over the UK. The current paper demonstrates the value of remote sensing and image processing for large-scale studies of gene flow, and describes a generic method that could be applied to a variety of crops in many countries. 6. Synthesis and applications. The decision to approve or prevent the release of a GM cultivar is made at a national rather than regional level. It is highly desirable that data relating to the decision-making process are collected at the same scale, rather than relying on extrapolation from smaller experiments designed at the plot, field or even regional scale. It would be extremely difficult and labour intensive to attempt to carry out such large-scale investigations without the use of remote-sensing technology. This study used rapeseed in the UK as a model to demonstrate the value of remote sensing in assembling empirical information at a national level.

Discriminative pattern mining in software fault detection

We present a method to enhance fault localization for software systems based on a frequent pattern mining algorithm. Our method is based on a large set of test cases for a given set of programs in which faults can be detected. The test executions are recorded as function call trees. Based on test oracles the tests can be classified into successful and failing tests. A frequent pattern mining algorithm is used to identify frequent subtrees in successful and failing test executions. This information is used to rank functions according to their likelihood of containing a fault. The ranking suggests an order in which to examine the functions during fault analysis. We validate our approach experimentally using a subset of Siemens benchmark programs.

A hierarchical distributed approach for mining molecular fragments

Recently, two approaches have been introduced that distribute the molecular fragment mining problem. The first approach applies a master/worker topology, the second approach, a completely distributed peer-to-peer system, solves the scalability problem due to the bottleneck at the master node. However, in many real world scenarios the participating computing nodes cannot communicate directly due to administrative policies such as security restrictions. Thus, potential computing power is not accessible to accelerate the mining run. To solve this shortcoming, this work introduces a hierarchical topology of computing resources, which distributes the management over several levels and adapts to the natural structure of those multi-domain architectures. The most important aspect is the load balancing scheme, which has been designed and optimized for the hierarchical structure. The approach allows dynamic aggregation of heterogenous computing resources and is applied to wide area network scenarios.

Distributed mining of molecular fragments

In real world applications sequential algorithms of data mining and data exploration are often unsuitable for datasets with enormous size, high-dimensionality and complex data structure. Grid computing promises unprecedented opportunities for unlimited computing and storage resources. In this context there is the necessity to develop high performance distributed data mining algorithms. However, the computational complexity of the problem and the large amount of data to be explored often make the design of large scale applications particularly challenging. In this paper we present the first distributed formulation of a frequent subgraph mining algorithm for discriminative fragments of molecular compounds. Two distributed approaches have been developed and compared on the well known National Cancer Institute’s HIV-screening dataset. We present experimental results on a small-scale computing environment.