FlexAnalytics: A flexible data analytics framework for big data applications with I/O performance improvement

Increasingly larger scale applications are generating an unprecedented amount of data. However, the increasing gap between computation and I/O capacity on High End Computing machines makes a severe bottleneck for data analysis. Instead of moving data from its source to the output storage, in-situ analytics processes output data while simulations are running. However, in-situ data analysis incurs much more computing resource contentions with simulations. Such contentions severely damage the performance of simulation on HPE. Since different data processing strategies have different impact on performance and cost, there is a consequent need for flexibility in the location of data analytics. In this paper, we explore and analyze several potential data-analytics placement strategies along the I/O path. To find out the best strategy to reduce data movement in given situation, we propose a flexible data analytics (FlexAnalytics) framework in this paper. Based on this framework, a FlexAnalytics prototype system is developed for analytics placement. FlexAnalytics system enhances the scalability and flexibility of current I/O stack on HEC platforms and is useful for data pre-processing, runtime data analysis and visualization, as well as for large-scale data transfer. Two use cases – scientific data compression and remote visualization – have been applied in the study to verify the performance of FlexAnalytics. Experimental results demonstrate that FlexAnalytics framework increases data transition bandwidth and improves the application end-to-end transfer performance.

Management of multi-scale forest resource data over time

During the last decades there has been a global shift in forest management from a focus solely on timber management to ecosystem management that endorses all aspects of forest functions: ecological, economic and social. This has resulted in a shift in paradigm from sustained yield to sustained diversity of values, goods and benefits obtained at the same time, introducing new temporal and spatial scales into forest resource management. The purpose of the present dissertation was to develop methods that would enable spatial and temporal scales to be introduced into the storage, processing, access and utilization of forest resource data. The methods developed are based on a conceptual view of a forest as a hierarchically nested collection of objects that can have a dynamically changing set of attributes. The temporal aspect of the methods consists of lifetime management for the objects and their attributes and of a temporal succession linking the objects together. Development of the forest resource data processing method concentrated on the extensibility and configurability of the data content and model calculations, allowing for a diverse set of processing operations to be executed using the same framework. The contribution of this dissertation to the utilisation of multi-scale forest resource data lies in the development of a reference data generation method to support forest inventory methods in approaching single-tree resolution.

Effect of zinc content on the processing map for hot working of α brass

The hot deformation behavior of α brass with varying zinc contents in the range 3%–30% was characterized using hot compression testing in the temperature range 600–900 °C and strain rate range 0.001–100 s−1. On the basis of the flow stress data, processing maps showing the variation of the efficiency of power dissipation (given by Image where m is the strain rate sensitivity) with temperature and strain rate were obtained. α brass exhibits a domain of dynamic recrystallization (DRX) at temperatures greater than 0.85Tm and at strain rates lower than 1 s−1. The maximum efficiency of power dissipation increases with increasing zinc content and is in the range 33%–53%. The DRX domain shifts to lower strain rates for higher zinc contents and the strain rate for peak efficiency is in the range 0.0001–0.05 s−1. The results indicate that the DRX in α brass is controlled by the rate of interface formation (nucleation) which depends on the diffusion-controlled process of thermal recovery by climb.

Effect of Zirconium on the Processing MAPS For Hot-Working of Alpha-Beta Brass

The effect of zirconium on the hot working characteristics of alpha and alpha-beta brass was studied in the temperature range of 500 to 850-degrees-C and the strain rate range of 0.001 to 100 s-1. On the basis of the flow stress data, processing maps showing the variation of the efficiency of power dissipation (given by [2m/(m+1)] where m is the strain rate sensitivity) with temperature and strain rate were obtained. The addition of zirconium to alpha brass decreased the maximum efficiency of power dissipation from 53 to 39%, increased the strain rate for dynamic recrystallization (DRX) from 0.001 to 0.1 s-1 and improved the hot workability. Alpha-beta brasses with and without zirconium exhibit a domain in the temperature range from 550 to 750-degrees-C and at strain rates lower than 1 s-1 with a maximum efficiency of power dissipation of nearly 50 % occurring in the temperature range of 700 to 750-degrees-C and a strain rate of 0.001 s-1. In the domain, the alpha phase undergoes DRX and controls the hot deformation of the alloy whereas the beta phase deforms superplastically. The addition of zirconium to alpha-beta brass has not affected the processing maps as it gets partitioned to the beta phase and does not alter the constitutive behavior of the alpha phase

Processing maps for hot working of Cu-Ni-Zn alloys. Part I - Alpha-nickel silver

The constitutive behaviour of agr — nickel silver in the temperature range 700–950 °C and strain rate range 0.001–100 s–1 was characterized with the help of a processing map generated on the basis of the principles of the ldquodynamic materials modelrdquo of Prasadet al Using the flow stress data, processing maps showing the variation of the efficiency of power dissipation (given by 2m/(m+1) wherem is the strain-rate sensitivity) with temperature and strain rate were obtained, agr-nickel silver exhibits a single domain at temperatures greater than 750 °C and at strain rates lower than 1s–1, with a maximum efficiency of 38% occurring at about 950 °C and at a strain rate of 0.1 s–1. In the domain the material undergoes dynamic recrystallization (DRX). On the basis of a model, it is shown that the DRX is controlled by the rate of interface formation (nucleation) which depends on the diffusion-controlled process of thermal recovery by climb. At high strain rates (10 and 100s–1) the material undergoes microstructural instabilities, the manifestations of which are in the form of adiabatic shear bands and strain markings.

Processing maps for hot working of Cu-Ni-Zn alloys Part II Alpha-Beta nickel silver

The constitutive behaviour of agr-beta nickel silver in the temperature range 600�850 °C and strainrate range 0.001�100s�1 was characterized with the help of a processing map generated on the principles of the dynamic materials model. On the basis of the flow-stress data, processing maps showing the variation of the efficiency of power dissipation (given by [2m/(m+1)], wherem is the strain-rate sensitivity) with temperature and strain rate were obtained, agr-beta nickel silver exhibits a single domain at temperatures greater than 700 °C and at strain rates lower than 1 s�1 with a maximum efficiency of power dissipation of about 42% occurring at about 850 °C and at 0.1 s�1. In the domain, the agr phase undergoes dynamic recrystallization and controls the deformation of the alloy, while the beta phase deforms superplastically. Optimum conditions for the processing of agr-beta nickel silver are 850 °C and 0.1 s�1. The material undergoes unstable flow at strain rates of 10 and 100 s�1 and in the temperature range 600�750 °C, manifestated in the form of adiabatic shear bands.

Micro-systems for Optical Protein-Chip

Protein-Chip as micro-assays for the determination of protein interaction, the analysis, the identification and the purification of proteins has large potential applications. The Optical Protein-Chip is able to detect the multi-interaction of proteins and multi-bio-activities of molecules directly and simultaneously with no labeling. The chip is a small matrix on solid substrate containing multi-micro-area prepared by microfabrication with photolithography or soft lithography for surface patterning, and processed with surface modification which includes the physical, chemical, and bio-chemical modifications, etc. The ligand immobilization, such as protein immobilization, especially the oriented immobilization with low steric hindrance and high bio-specific binding activity between ligand and receptor is used to form a sensing surface. Each area of the pattern is corresponding to only one bioactivity. The interval between the areas is non-bioactive and optically extinctive. The affinity between proteins is used to realize non-labeling microassays for the determination of protein identification and protein interaction. The sampling of the chip is non-disturbing, performed with imaging ellipsometry and image processing on a database of proteins.

The Intelligent Measuring Sub-System in the Computer Integrated and Flexible Laser Processing System

Based on the computer integrated and flexible laser processing system, develop the intelligent measuring sub-system. A novel model has been built to compensate the deviations of the main frame, a new-developed 3-D laser tracker system is applied to adjust the accuracy of the system. Analyzing the characteristic of all kinds of automobile dies, which is the main processing object of the laser processing system, classify the types of the surface and border needed to be measured and be processed. According to different types of surface and border, develop 2-D adaptive measuring method based on B?zier curve and 3-D adaptive measuring method based on spline curve. During the data processing, a new 3-D probe compensation method has been described in details. Some measuring experiments and laser processing experiments are carried out to testify the methods. All the methods have been applied in the computer integrated and flexible laser processing system invented by the Institute of Mechanics, CAS.

Anisotropy preserving DTI processing

Statistical analysis of diffusion tensor imaging (DTI) data requires a computational framework that is both numerically tractable (to account for the high dimensional nature of the data) and geometric (to account for the nonlinear nature of diffusion tensors). Building upon earlier studies exploiting a Riemannian framework to address these challenges, the present paper proposes a novel metric and an accompanying computational framework for DTI data processing. The proposed approach grounds the signal processing operations in interpolating curves. Well-chosen interpolating curves are shown to provide a computational framework that is at the same time tractable and information relevant for DTI processing. In addition, and in contrast to earlier methods, it provides an interpolation method which preserves anisotropy, a central information carried by diffusion tensor data. © 2013 Springer Science+Business Media New York.

Cellular automata and non-static image processing for embodied robot systems on a massively parallel processor array

Huelse, M, Barr, D R W, Dudek, P: Cellular Automata and non-static image processing for embodied robot systems on a massively parallel processor array. In: Adamatzky, A et al. (eds) AUTOMATA 2008, Theory and Applications of Cellular Automata. Luniver Press, 2008, pp. 504-510. Sponsorship: EPSRC

Information processing without brains--the power of intercellular regulators in plants.

Plants exhibit different developmental strategies than animals; these are characterized by a tight linkage between environmental conditions and development. As plants have neither specialized sensory organs nor a nervous system, intercellular regulators are essential for their development. Recently, major advances have been made in understanding how intercellular regulation is achieved in plants on a molecular level. Plants use a variety of molecules for intercellular regulation: hormones are used as systemic signals that are interpreted at the individual-cell level; receptor peptide-ligand systems regulate local homeostasis; moving transcriptional regulators act in a switch-like manner over small and large distances. Together, these mechanisms coherently coordinate developmental decisions with resource allocation and growth.

Quantifying data quality for clinical trials using electronic data capture.

BACKGROUND: Historically, only partial assessments of data quality have been performed in clinical trials, for which the most common method of measuring database error rates has been to compare the case report form (CRF) to database entries and count discrepancies. Importantly, errors arising from medical record abstraction and transcription are rarely evaluated as part of such quality assessments. Electronic Data Capture (EDC) technology has had a further impact, as paper CRFs typically leveraged for quality measurement are not used in EDC processes. METHODS AND PRINCIPAL FINDINGS: The National Institute on Drug Abuse Treatment Clinical Trials Network has developed, implemented, and evaluated methodology for holistically assessing data quality on EDC trials. We characterize the average source-to-database error rate (14.3 errors per 10,000 fields) for the first year of use of the new evaluation method. This error rate was significantly lower than the average of published error rates for source-to-database audits, and was similar to CRF-to-database error rates reported in the published literature. We attribute this largely to an absence of medical record abstraction on the trials we examined, and to an outpatient setting characterized by less acute patient conditions. CONCLUSIONS: Historically, medical record abstraction is the most significant source of error by an order of magnitude, and should be measured and managed during the course of clinical trials. Source-to-database error rates are highly dependent on the amount of structured data collection in the clinical setting and on the complexity of the medical record, dependencies that should be considered when developing data quality benchmarks.

Regularized iterative and non-iterative procedures for object restoration from experimental data

A regularized algorithm for the recovery of band-limited signals from noisy data is described. The regularization is characterized by a single parameter. Iterative and non-iterative implementations of the algorithm are shown to have useful properties, the former offering the advantage of flexibility and the latter a potential for rapid data processing. Comparative results, using experimental data obtained in laser anemometry studies with a photon correlator, are presented both with and without regularization. © 1983 Taylor & Francis Ltd.

Time of flight mass spectrometry for quantitative data analysis in fast transient studies using a Temporal Analysis of Products (TAP) reactor

A Time of flight (ToF) mass spectrometer suitable in terms of sensitivity, detector response and time resolution, for application in fast transient Temporal Analysis of Products (TAP) kinetic catalyst characterization is reported. Technical difficulties associated with such application as well as the solutions implemented in terms of adaptations of the ToF apparatus are discussed. The performance of the ToF was validated and the full linearity of the specific detector over the full dynamic range was explored in order to ensure its applicability for the TAP application. The reported TAP-ToF setup is the first system that achieves the high level of sensitivity allowing monitoring of the full 0-200 AMU range simultaneously with sub-millisecond time resolution. In this new setup, the high sensitivity allows the use of low intensity pulses ensuring that transport through the reactor occurs in the Knudsen diffusion regime and that the data can, therefore, be fully analysed using the reported theoretical TAP models and data processing.

A review of ADP processing software

Data processing is an essential part of Acoustic Doppler Profiler (ADP) surveys, which have become the standard tool in assessing flow characteristics at tidal power development sites. In most cases, further processing beyond the capabilities of the manufacturer provided software tools is required. These additional tasks are often implemented by every user in mathematical toolboxes like MATLAB, Octave or Python. This requires the transfer of the data from one system to another and thus increases the possibility of errors. The application of dedicated tools for visualisation of flow or geographic data is also often beneficial and a wide range of tools are freely available, though again problems arise from the necessity of transferring the data. Furthermore, almost exclusively PCs are supported directly by the ADP manufacturers, whereas small computing solutions like tablet computers, often running Android or Linux operating systems, seem better suited for online monitoring or data acquisition in field conditions. While many manufacturers offer support for developers, any solution is limited to a single device of a single manufacturer. A common data format for all ADP data would allow development of applications and quicker distribution of new post processing methodologies across the industry.