Exploring the integration of data mining and data visualization

Due to the rapid advances in computing and sensing technologies, enormous amounts of data are being generated everyday in various applications. The integration of data mining and data visualization has been widely used to analyze these massive and complex data sets to discover hidden patterns. For both data mining and visualization to be effective, it is important to include the visualization techniques in the mining process and to generate the discovered patterns for a more comprehensive visual view. In this dissertation, four related problems: dimensionality reduction for visualizing high dimensional datasets, visualization-based clustering evaluation, interactive document mining, and multiple clusterings exploration are studied to explore the integration of data mining and data visualization. In particular, we 1) propose an efficient feature selection method (reliefF + mRMR) for preprocessing high dimensional datasets; 2) present DClusterE to integrate cluster validation with user interaction and provide rich visualization tools for users to examine document clustering results from multiple perspectives; 3) design two interactive document summarization systems to involve users efforts and generate customized summaries from 2D sentence layouts; and 4) propose a new framework which organizes the different input clusterings into a hierarchical tree structure and allows for interactive exploration of multiple clustering solutions.

Efficient storage and domain-specific information discovery on semistructured documents

The increasing amount of available semistructured data demands efficient mechanisms to store, process, and search an enormous corpus of data to encourage its global adoption. Current techniques to store semistructured documents either map them to relational databases, or use a combination of flat files and indexes. These two approaches result in a mismatch between the tree-structure of semistructured data and the access characteristics of the underlying storage devices. Furthermore, the inefficiency of XML parsing methods has slowed down the large-scale adoption of XML into actual system implementations. The recent development of lazy parsing techniques is a major step towards improving this situation, but lazy parsers still have significant drawbacks that undermine the massive adoption of XML. ^ Once the processing (storage and parsing) issues for semistructured data have been addressed, another key challenge to leverage semistructured data is to perform effective information discovery on such data. Previous works have addressed this problem in a generic (i.e. domain independent) way, but this process can be improved if knowledge about the specific domain is taken into consideration. ^ This dissertation had two general goals: The first goal was to devise novel techniques to efficiently store and process semistructured documents. This goal had two specific aims: We proposed a method for storing semistructured documents that maps the physical characteristics of the documents to the geometrical layout of hard drives. We developed a Double-Lazy Parser for semistructured documents which introduces lazy behavior in both the pre-parsing and progressive parsing phases of the standard Document Object Model’s parsing mechanism. ^ The second goal was to construct a user-friendly and efficient engine for performing Information Discovery over domain-specific semistructured documents. This goal also had two aims: We presented a framework that exploits the domain-specific knowledge to improve the quality of the information discovery process by incorporating domain ontologies. We also proposed meaningful evaluation metrics to compare the results of search systems over semistructured documents. ^

Field programmable gate array based target detection and gesture recognition

The move from Standard Definition (SD) to High Definition (HD) represents a six times increases in data, which needs to be processed. With expanding resolutions and evolving compression, there is a need for high performance with flexible architectures to allow for quick upgrade ability. The technology advances in image display resolutions, advanced compression techniques, and video intelligence. Software implementation of these systems can attain accuracy with tradeoffs among processing performance (to achieve specified frame rates, working on large image data sets), power and cost constraints. There is a need for new architectures to be in pace with the fast innovations in video and imaging. It contains dedicated hardware implementation of the pixel and frame rate processes on Field Programmable Gate Array (FPGA) to achieve the real-time performance. ^ The following outlines the contributions of the dissertation. (1) We develop a target detection system by applying a novel running average mean threshold (RAMT) approach to globalize the threshold required for background subtraction. This approach adapts the threshold automatically to different environments (indoor and outdoor) and different targets (humans and vehicles). For low power consumption and better performance, we design the complete system on FPGA. (2) We introduce a safe distance factor and develop an algorithm for occlusion occurrence detection during target tracking. A novel mean-threshold is calculated by motion-position analysis. (3) A new strategy for gesture recognition is developed using Combinational Neural Networks (CNN) based on a tree structure. Analysis of the method is done on American Sign Language (ASL) gestures. We introduce novel point of interests approach to reduce the feature vector size and gradient threshold approach for accurate classification. (4) We design a gesture recognition system using a hardware/ software co-simulation neural network for high speed and low memory storage requirements provided by the FPGA. We develop an innovative maximum distant algorithm which uses only 0.39% of the image as the feature vector to train and test the system design. Database set gestures involved in different applications may vary. Therefore, it is highly essential to keep the feature vector as low as possible while maintaining the same accuracy and performance^

Efficient Storage and Domain-Specific Information Discovery on Semistructured Documents

The increasing amount of available semistructured data demands efficient mechanisms to store, process, and search an enormous corpus of data to encourage its global adoption. Current techniques to store semistructured documents either map them to relational databases, or use a combination of flat files and indexes. These two approaches result in a mismatch between the tree-structure of semistructured data and the access characteristics of the underlying storage devices. Furthermore, the inefficiency of XML parsing methods has slowed down the large-scale adoption of XML into actual system implementations. The recent development of lazy parsing techniques is a major step towards improving this situation, but lazy parsers still have significant drawbacks that undermine the massive adoption of XML. Once the processing (storage and parsing) issues for semistructured data have been addressed, another key challenge to leverage semistructured data is to perform effective information discovery on such data. Previous works have addressed this problem in a generic (i.e. domain independent) way, but this process can be improved if knowledge about the specific domain is taken into consideration. This dissertation had two general goals: The first goal was to devise novel techniques to efficiently store and process semistructured documents. This goal had two specific aims: We proposed a method for storing semistructured documents that maps the physical characteristics of the documents to the geometrical layout of hard drives. We developed a Double-Lazy Parser for semistructured documents which introduces lazy behavior in both the pre-parsing and progressive parsing phases of the standard Document Object Model's parsing mechanism. The second goal was to construct a user-friendly and efficient engine for performing Information Discovery over domain-specific semistructured documents. This goal also had two aims: We presented a framework that exploits the domain-specific knowledge to improve the quality of the information discovery process by incorporating domain ontologies. We also proposed meaningful evaluation metrics to compare the results of search systems over semistructured documents.

Using Florida Keys Reference Sites As a Standard for Restoration of Forest Structure in Everglades Tree Islands

In south Florida, tropical hardwood forests (hammocks) occur in Everglades tree islands and as more extensive forests in coastal settings in the nearby Florida Keys. Keys hammocks have been less disturbed by humans, and many qualify as “old-growth,” while Everglades hammocks have received much heavier use. With improvement of tree island condition an important element in Everglades restoration efforts, we examined stand structure in 23 Keys hammocks and 69 Everglades tree islands. Based on Stand Density Index and tree diameter distributions, many Everglades hammocks were characterized by low stocking and under-representation in the smaller size classes. In contrast, most Keys forests had the dense canopies and open understories usually associated with old-growth hardwood hammocks. Subject to the same caveats that apply to off-site references elsewhere, structural information from mature Keys hammocks can be helpful in planning and implementing forest restoration in Everglades tree islands. In many of these islands, such restoration might involve supplementing tree stocking by planting native trees to produce more complete site utilization and a more open understory.

Ecosystem structure, nutrient dynamics, and hydrologic relationships in tree islands of the southern Everglades, Florida, USA

Tree islands are an important structural component of many graminoid-dominated wetlands because they increase ecological complexity in the landscape. Tree island area has been drastically reduced with hydrologic modifications within the Everglades ecosystem, yet still little is known about the ecosystem ecology of Everglades tree islands. As part of an ongoing study to investigate the effects of hydrologic restoration on short hydroperiod marshes of the southern Everglades, we report an ecosystem characterization of seasonally flooded tree islands relative to locations described by variation in freshwater flow (i.e. locally enhanced freshwater flow by levee removal). We quantified: (1) forest structure, litterfall production, nutrient utilization, soil dynamics, and hydrologic properties of six tree islands and (2) soil and surface water physico-chemical properties of adjacent marshes. Tree islands efficiently utilized both phosphorus and nitrogen, but indices of nutrient-use efficiency indicated stronger P than N limitation. Tree islands were distinct in structure and biogeochemical properties from the surrounding marsh, maintaining higher organically bound P and N, but lower inorganic N. Annual variation resulting in increased hydroperiod and lower wet season water levels not only increased nitrogen use by tree species and decreased N:P values of the dominant plant species (Chrysobalanus icaco), but also increased soil pH and decreased soil temperature. When compared with other forested wetlands, these Everglades tree islands were among the most nutrient efficient, likely a function of nutrient immobilization in soils and the calcium carbonate bedrock. Tree islands of our study area are defined by: (1) unique biogeochemical properties when compared with adjacent short hydroperiod marshes and other forested wetlands and (2) an intricate relationship with marsh hydrology. As such, they may play an important and disproportionate role in nutrient and carbon cycling in Everglades wetlands. With the loss of tree islands that has occurred with the degradation of the Everglades system, these landscape processes may have been altered. With this baseline dataset, we have established a long-term ecosystem-scale experiment to follow the ecosystem trajectory of seasonally flooded tree islands in response to hydrologic restoration of the southern Everglades.

Tree Islands in Everglades Landscapes: Current Status, Historical Changes, and Hydrologic Impacts on Population Dynamics and Moisture Relations, First Annual Report

In 2005 we initiated a project designed to better understand tree island structure and function in the Everglades and the wetlands bordering it. Focus was on the raised portions at the upstream end of the islands, where tropical hardwood species adapted to well-drained conditions usually are the most prominent component of the vegetation. The study design is hierarchical, with four levels; in general, a large number of sites is to be surveyed once for a limited set of parameters, and increasingly small sets of islands are to be sampled more intensively, more frequently, and for more aspects of ecosystem function. During the first year of the 3-year study, we completed surveys of 41 Level 1 (i.e., the least intensive level) islands, and established permanent plots in two and three islands of Levels 2 and 4 intensity, respectively. Tree species richness and structural complexity was highest in Shark Slough “hammocks”, while islands in Northeast Shark Slough and Water Conservation Area 3B, which receive heavy human use, were simpler, more park-like communities. Initial monitoring of soil moisture in Level 4 hammocks indicated considerable local variation, presumably associated with antecedent rainfall and current water levels in the adjacent marsh. Tree islands throughout the study area were impacted significantly by Hurricanes Katrina and Wilma in 2005, but appear to be recovering rapidly. As the project continues to include more islands and repeated measurements, we expect to develop a better grasp of tree island dynamics across the Everglades ecosystem, especially with respect to moisture relations and water levels in the adjacent marsh. The detailed progress report which follows is also available online at http://www.fiu.edu/~serp1/projects/treeislands/tree_islands_2005_annual_report.pd

Monitoring of Tree Island Condition in the Southern Everglades: Hydrologic Driven Decadal Changes in Tree Island Woody Vegetation Structure and Composition: 2012 Annual Report

This report examines the interaction between hydrology and vegetation over a 10-year period, between 2001/02 and 2012 within six permanent tree island plots located on three tree islands, two plots each per tree island, established in 2001/02, along a hydrologic and productivity gradient. We hypothesize that: (H1) hydrologic differences within plots between census dates will result in marked differences in a) tree and sapling densities, b) tree basal area, and c) forest structure, i.e., canopy volume and height, and (H2) tree island growth, development, and succession is dependent on hydrologic fluxes, particularly during periods of prolonged droughts or below average hydroperiods.

Genetic Structure of the Florida Key Tree Cactus, Pilosocereus robinii, using Restriction Site associated DNA (RAD) markers

Rare plant conservation efforts must utilize current genetic methods to ensure the evolutionary potential of populations is preserved. One such effort involves the Key Tree Cactus, Pilosocereus robinii, which is an endangered columnar cactus native to the Florida Keys. The populations have precipitously declined over the past decade because of habitat loss and increasing soil salinity from rising sea levels and storm surge. Next-generation DNA sequencing was used to assess the genetic structure of the populations. Twenty individuals representative of both wild and extirpated cacti were chosen for Restriction Site Associated DNA (RAD) analysis. Samples processed using the HindIII and NotIII restriction enzymes produced 82,382,440 high quality reads used for genetic mapping, from which 5,265 Single Nucleotide Polymorphisms (SNPs) were discovered. The analysis revealed that the Keys’ populations are closely related with little population differentiation. In addition, the populations display evidence of inbreeding and low genetic diversity.

Tree Islands in the Shark Slough Landscape: Interactions of Vegetation, Hydrology, and Soils

Executive Summary: This report presents what we have learned about tree islands of Shark Slough and adjacent marshes of Everglades National Park (ENP), based on ecological studies carried out in these wetlands during the period 2000-2003. The tree islands of Shark Slough share many features with tree islands elsewhere in the Everglades. Their current composition and community structure is determined to a large extent by recent hydrology, as well as by disturbances (fire, freezes, hurricanes, man). Tree islands have historical, cultural, and biological values that are recognized by nearly all parties to the Comprehensive Everglades Restoration Plan (CERP). Maintaining and/or restoring the health of tree islands are major objectives of CERP. Consequently, there is a need within CERP for tools to assess the health of tree islands, and to relate these measures to the hydrologic regime to which they are exposed.

Effects of irradiance and spectral quality on leaf structure and function in seedlings of two Southeast Asian Hopea species

We studied the development of leaf characters in two Southeast Asian dipterocarp forest trees under different photosynthetic photon flux densities (PFD) and spectral qualities (red to far-red, R:FR). The two species, Hopea helferi and H. odorata, are taxonomically closely related but differ in their ecological requirements; H. helferi is more drought tolerant and H. odorata more shade tolerant. Seedlings were grown in replicated shadehouse treatments of differing PFD and R:FR. We measured or calculated (1) leaf and tissue thicknesses; (2) mesophyll parenchyma, air space, and lignified tissue volumes; (3) mesophyll air volumes (Vmes/Asurf) and surfaces (Ames/Asurf); (4) palisade cell length and width; (5) chlorophyll/cm2 and a/ b; (6) leaf absorption; and (7) attenuance/absorbance at 652 and 550 nm. These characters varied in response to light conditions in both taxa. Characters were predominantly affected by PFD, and R:FR slightly influenced many characters. Leaf characters of H. odorata were more plastic in response to treatment conditions. Characters were correlated with each other in a complex fashion. Variation in leaf anatomy is most likely a consequence of increasing leaf thickness in both taxa, which may increase mechanical strength and defense against herbivory in more exposed environments. Variation in leaf optical properties was most likely affected by pigment photo-bleaching in treatments of more intense PFD and was not correlated with Amax. The greater plasticity of leaf responses in H. odorata helps explain the acclimation over the range of light conditions encountered by this shade-tolerant taxon. The dense layer of scales on the leaf undersurface and other anatomical characters in H. helferi reduced gas exchange and growth in this drought-tolerant tree.

Tobacco use prevention education (TUPE) programs in the state of Florida: Teachers' perceptions of program structure and effectiveness

This study described teacher perceptions of TUPE program effectiveness in Florida in an attempt to improve programs by identifying factors that might influence teacher motivation and performance. Very little work has been done to examine how teachers' perceptions are related to the effectiveness of TUPE programs. A statewide survey provided information about how teachers' perceptions of program effectiveness are affected by variables such as: program structure, barriers, tobacco use norms, and training variables. Data were obtained from a telephone survey conducted in Florida as part of the Tobacco Pilot Project (TPP). The sample included 296 middle school teachers and 282 high school teachers as well as 193 middle school principals and 190 high school principals. Correlational and hierarchical regression analyses identified correlates and predictors of teachers' ratings of effectiveness. Results suggest that use of peer leaders, more frequent evaluations, a higher degree of parent involvement, fewer barriers, greater student interest, and lower tolerance for tobacco use were correlated with higher ratings of program effectiveness. Furthermore, student interest, peer, staff, and community tolerance norms, peer leaders, program evaluation, and parent involvement predicted middle school teachers' perceptions. Parent tolerance, student interest, number of barriers, and more frequent program evaluation predicted high school teachers' perceptions. In addition, middle school teachers who reported a lower number of factors negatively associated with teacher receptivity were more likely to view TUPE programs more favorably than teachers who reported a greater number of these risk factors. This relationship was not as robust among the high school teacher sample. Differences between the middle and high school sample were found in the magnitude and number of significant correlations, the proportion of variance accounted for by predictor variables, and the strength of the relationship between the number of factors negatively associated with teacher receptivity and teachers' perceptions of TUPE effectiveness. These findings highlighted the importance of the timing, program features, and the external environment for enhancing or minimizing teachers' ratings of TUPE program effectiveness. In conclusion, significant increases in TUPE teachers' self-efficacy will occur through the participation of peers, parents, staff, and community leaders in different aspects of TUPE programs. ^

An indexing structure and application model for vehicles moving on road networks

Moving objects database systems are the most challenging sub-category among Spatio-Temporal database systems. A database system that updates in real-time the location information of GPS-equipped moving vehicles has to meet even stricter requirements. Currently existing data storage models and indexing mechanisms work well only when the number of moving objects in the system is relatively small. This dissertation research aimed at the real-time tracking and history retrieval of massive numbers of vehicles moving on road networks. A total solution has been provided for the real-time update of the vehicles' location and motion information, range queries on current and history data, and prediction of vehicles' movement in the near future. ^ To achieve these goals, a new approach called Segmented Time Associated to Partitioned Space (STAPS) was first proposed in this dissertation for building and manipulating the indexing structures for moving objects databases. ^ Applying the STAPS approach, an indexing structure of associating a time interval tree to each road segment was developed for real-time database systems of vehicles moving on road networks. The indexing structure uses affordable storage to support real-time data updates and efficient query processing. The data update and query processing performance it provides is consistent without restrictions such as a time window or assuming linear moving trajectories. ^ An application system design based on distributed system architecture with centralized organization was developed to maximally support the proposed data and indexing structures. The suggested system architecture is highly scalable and flexible. Finally, based on a real-world application model of vehicles moving in region-wide, main issues on the implementation of such a system were addressed. ^

Hurricane disturbance and recovery of energy balance, CO2 fluxes and canopy structure in a mangrove forest of the Florida Everglades

Eddy covariance (EC) estimates of carbon dioxide (CO2) fluxes and energy balance are examined to investigate the functional responses of a mature mangrove forest to a disturbance generated by Hurricane Wilma on October 24, 2005 in the Florida Everglades. At the EC site, high winds from the hurricane caused nearly 100% defoliation in the upper canopy and widespread tree mortality. Soil temperatures down to -50 cm increased, and air temperature lapse rates within the forest canopy switched from statically stable to statically unstable conditions following the disturbance. Unstable conditions allowed more efficient transport of water vapor and CO2 from the surface up to the upper canopy layer. Significant increases in latent heat fluxes (LE) and nighttime net ecosystem exchange (NEE) were also observed and sensible heat fluxes (H) as a proportion of net radiation decreased significantly in response to the disturbance. Many of these impacts persisted through much of the study period through 2009. However, local albedo and MODIS (Moderate Resolution Imaging Spectro-radiometer) data (the Enhanced Vegetation Index) indicated a substantial proportion of active leaf area recovered before the EC measurements began 1 year after the storm. Observed changes in the vertical distribution and the degree of clumping in newly emerged leaves may have affected the energy balance. Direct comparisons of daytime NEE values from before the storm and after our measurements resumed did not show substantial or consistent differences that could be attributed to the disturbance. Regression analyses on seasonal time scales were required to differentiate the storm's impact on monthly average daytime NEE from the changes caused by interannual variability in other environmental drivers. The effects of the storm were apparent on annual time scales, and CO2 uptake remained approximately 250 g C m-2 yr-1 lower in 2009 compared to the average annual values measured in 2004-2005. Dry season CO2 uptake was relatively more affected by the disturbance than wet season values. Complex leaf regeneration dynamics on damaged trees during ecosystem recovery are hypothesized to lead to the variable dry versus wet season impacts on daytime NEE. In contrast, nighttime CO2 release (i.e., nighttime respiration) was consistently and significantly greater, possibly as a result of the enhanced decomposition of litter and coarse woody debris generated by the storm, and this effect was most apparent in the wet seasons compared to the dry seasons. The largest pre- and post-storm differences in NEE coincided roughly with the delayed peak in cumulative mortality of stems in 2007-2008. Across the hurricane-impacted region, cumulative tree mortality rates were also closely correlated with declines in peat surface elevation. Mangrove forest-atmosphere interactions are interpreted with respect to the damage and recovery of stand dynamics and soil accretion processes following the hurricane.

Pattern of nutrient availability and plant community assemblage in Everglades Tree Islands, Florida, USA

We address the relative importance of nutrient availability in relation to other physical and biological factors in determining plant community assemblages around Everglades Tree Islands (Everglades National Park, Florida, USA). We carried out a one-time survey of elevation, soil, water level and vegetation structure and composition at 138 plots located along transects in three tree islands in the Park’s major drainage basin. We used an RDA variance partitioning technique to assess the relative importance of nutrient availability (soil N and P) and other factors in explaining herb and tree assemblages of tree island tail and surrounded marshes. The upland areas of the tree islands accumulate P and show low N concentration, producing a strong island-wide gradient in soil N:P ratio. While soil N:P ratio plays a significant role in determining herb layer and tree layer community assemblage in tree island tails, nevertheless part of its variance is shared with hydrology. The total species variance explained by the predictors is very low. We define a strong gradient in nutrient availability (soil N:P ratio) closely related to hydrology. Hydrology and nutrient availability are both factors influencing community assemblages around tree islands, nevertheless both seem to be acting together and in a complex mechanism. Future research should be focused on segregating these two factors in order to determine whether nutrient leaching from tree islands is a factor determining community assemblages and local landscape pattern in the Everglades, and how this process might be affected by water management.