A design pattern for efficient retrieval of large data sets from remote data sources

Retrieving large amounts of information over wide area networks, including the Internet, is problematic due to issues arising from latency of response, lack of direct memory access to data serving resources, and fault tolerance. This paper describes a design pattern for solving the issues of handling results from queries that return large amounts of data. Typically these queries would be made by a client process across a wide area network (or Internet), with one or more middle-tiers, to a relational database residing on a remote server. The solution involves implementing a combination of data retrieval strategies, including the use of iterators for traversing data sets and providing an appropriate level of abstraction to the client, double-buffering of data subsets, multi-threaded data retrieval, and query slicing. This design has recently been implemented and incorporated into the framework of a commercial software product developed at Oracle Corporation.

The precipitation climate of Central Asia: intercomparison of observational and numerical data sources in a remote semiarid region

In this study, we systematically compare a wide range of observational and numerical precipitation datasets for Central Asia. Data considered include two re-analyses, three datasets based on direct observations, and the output of a regional climate model simulation driven by a global re-analysis. These are validated and intercompared with respect to their ability to represent the Central Asian precipitation climate. In each of the datasets, we consider the mean spatial distribution and the seasonal cycle of precipitation, the amplitude of interannual variability, the representation of individual yearly anomalies, the precipitation sensitivity (i.e. the response to wet and dry conditions), and the temporal homogeneity of precipitation. Additionally, we carried out part of these analyses for datasets available in real time. The mutual agreement between the observations is used as an indication of how far these data can be used for validating precipitation data from other sources. In particular, we show that the observations usually agree qualitatively on anomalies in individual years while it is not always possible to use them for the quantitative validation of the amplitude of interannual variability. The regional climate model is capable of improving the spatial distribution of precipitation. At the same time, it strongly underestimates summer precipitation and its variability, while interannual variations are well represented during the other seasons, in particular in the Central Asian mountains during winter and spring

Data queries over heterogeneous sources

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Iis--integrated Interactome System: A Web-based Platform For The Annotation, Analysis And Visualization Of Protein-metabolite-gene-drug Interactions By Integrating A Variety Of Data Sources And Tools.

High-throughput screening of physical, genetic and chemical-genetic interactions brings important perspectives in the Systems Biology field, as the analysis of these interactions provides new insights into protein/gene function, cellular metabolic variations and the validation of therapeutic targets and drug design. However, such analysis depends on a pipeline connecting different tools that can automatically integrate data from diverse sources and result in a more comprehensive dataset that can be properly interpreted. We describe here the Integrated Interactome System (IIS), an integrative platform with a web-based interface for the annotation, analysis and visualization of the interaction profiles of proteins/genes, metabolites and drugs of interest. IIS works in four connected modules: (i) Submission module, which receives raw data derived from Sanger sequencing (e.g. two-hybrid system); (ii) Search module, which enables the user to search for the processed reads to be assembled into contigs/singlets, or for lists of proteins/genes, metabolites and drugs of interest, and add them to the project; (iii) Annotation module, which assigns annotations from several databases for the contigs/singlets or lists of proteins/genes, generating tables with automatic annotation that can be manually curated; and (iv) Interactome module, which maps the contigs/singlets or the uploaded lists to entries in our integrated database, building networks that gather novel identified interactions, protein and metabolite expression/concentration levels, subcellular localization and computed topological metrics, GO biological processes and KEGG pathways enrichment. This module generates a XGMML file that can be imported into Cytoscape or be visualized directly on the web. We have developed IIS by the integration of diverse databases following the need of appropriate tools for a systematic analysis of physical, genetic and chemical-genetic interactions. IIS was validated with yeast two-hybrid, proteomics and metabolomics datasets, but it is also extendable to other datasets. IIS is freely available online at: http://www.lge.ibi.unicamp.br/lnbio/IIS/.

Keyword search across distributed heterogenous structured data sources

Magdeburg, Univ., Fak. für Informatik, Diss., 2012

All-Ireland Health Data Inventory. Part 1: Metadata for key data sources, version 1.0

The All-Ireland Health Data Inventory. Part 1 is a catalogue of key sources of health data in the Republic and Northern Ireland. It includes relevant datasets from the major information reviews, conducted in the North and South, in the past few years. Information is essential for informed decision making and service provision. This inventory draws together information sources to facilitate such decision making. The inventory is intended as a resource for health professionals, researchers and the general public, providing the first phase of a ‘one-stop’ catalogue of health data. The datasets have been catalogued using an expanding numbering system which will allow for the inclusion of future resources. The Institute of Public Health in Ireland is in the process of expanding the Inventory to include further data sources.

Global burden of dementia in the year 2050: summary of methods and data sources

��The number of people suffering dementia will triple in the next 40 years, according to a new study by the World Health Organization, leading to catastrophic social and financial costs. Dementia, a brain illness that affects memory, behavior and the ability to perform even common tasks, affects mostly older people; Alzheimer's causes many cases. Read the report:Global burden of dementia in the year 2050: summary of methods and data sources

2008 Health Inequalities Profiles: Data sources

Links to data sources and methods as used in the production of erpho's 2008 Health Inequalities Profiles. This year's profiles cover the same indicators as previous profiles. Changes since last year:> A fifth time period: 2005-07> Updated populations > IMD 2007> Standardised against European Standard Population> Added comparator area 'All but most deprived' (80/20)

2009 Cancer in Iowa Report: Data Sources, 2009

Cancer is a reportable disease as stated in the Iowa Administrative Code. Cancer data are collected by the State Health Registry of Iowa, located at The University of Iowa in the College of Public Health’s Department of Epidemiology. The staff includes more than 50 people. Half of them, situated throughout the state, regularly visit hospitals, clinics, and medical laboratories in Iowa and neighboring states to collect cancer data. A follow-up program tracks more than 99 percent of the cancer survivors diagnosed since 1973. This program provides regular updates for follow-up and survival. The Registry maintains the confidentiality of the patients, physicians, and hospitals providing data. In 2009 data will be collected on an estimated 16,000 new cancers among Iowa residents. In situ cases of bladder cancer are included in the estimates for bladder cancer, to be in agreement with the definition of reportable cases of the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute. Since 1973 the Iowa Registry has been funded primarily by the SEER Program of the National Cancer Institute. Iowa represents rural and Midwestern populations and provides data included in many National Cancer Institute publications. Beginning in 1990 between 5 and 10 percent of the Registry’s annual operating budget has been provided by the state of Iowa. Beginning in 2003, the University of Iowa has been providing cost-sharing funds. The Registry also receives funding through grants and contracts with university, state, and national researchers investigating cancer-related topics.

Multichannel Remote Data Acquisition and Analysing Systems For Fisheries / Environmental Investigations

Remote Data acquisition and analysing systems developed for fisheries and related environmental studies have been reported. It consists of three units. The first one namely multichannel remote data acquisition system is installed at the remote place powered by a rechargeable battery. It acquires and stores the 16 channel environmental data on a battery backed up RAM. The second unit called the Field data analyser is used for insitue display and analysis of the data stored in the backed up RAM. The third unit namely Laboratory data analyser is an IBM compatible PC based unit for detailed analysis and interpretation of the data after bringing the RAM unit to the laboratory. The data collected using the system has been analysed and presented in the form of a graph. The system timer operated at negligibly low current, switches on the power to the entire remote operated system at prefixed time interval of 2 hours.Data storage at remote site on low power battery backedupRAM and retrieval and analysis of data using PC are the special i ty of the system. The remote operated system takes about 7 seconds including the 5 second stabilization time to acquire and store data and is very ideal for remote operation on rechargeable bat tery. The system can store 16 channel data scanned at 2 hour interval for 10 days on 2K backed up RAM with memory expansion facility for 8K RAM.