Pacote de sincronização de réplicas de bases de dados

A replicação de base de dados tem como objectivo a cópia de dados entre bases de dados distribuídas numa rede de computadores. A replicação de dados é importante em várias situações, desde a realização de cópias de segurança da informação, ao balanceamento de carga, à distribuição da informação por vários locais, até à integração de sistemas heterogéneos. A replicação possibilita uma diminuição do tráfego de rede, pois os dados ficam disponíveis localmente possibilitando também o seu acesso no caso de indisponibilidade da rede. Esta dissertação baseia-se na realização de um trabalho que consistiu no desenvolvimento de uma aplicação genérica para a replicação de bases de dados a disponibilizar como open source software. A aplicação desenvolvida possibilita a integração de dados entre vários sistemas, com foco na integração de dados heterogéneos, na fragmentação de dados e também na possibilidade de adaptação a várias situações. ABSTRACT: Data replication is a mechanism to synchronize and integrate data between distributed databases over a computer network. Data replication is an important tool in several situations, such as the creation of backup systems, load balancing between various nodes, distribution of information between various locations, integration of heterogeneous systems. Replication enables a reduction in network traffic, because data remains available locally even in the event of a temporary network failure. This thesis is based on the work carried out to develop an application for database replication to be made accessible as open source software. The application that was built allows for data integration between various systems, with particular focus on, amongst others, the integration of heterogeneous data, the fragmentation of data, replication in cascade, data format changes between replicas, master/slave and multi master synchronization.

High performance and energy-efficient instruction cache design and optimisation for ultra-low-power multi-core clusters

High Energy efficiency and high performance are the key regiments for Internet of Things (IoT) end-nodes. Exploiting cluster of multiple programmable processors has recently emerged as a suitable solution to address this challenge. However, one of the main bottlenecks for multi-core architectures is the instruction cache. While private caches fall into data replication and wasting area, fully shared caches lack scalability and form a bottleneck for the operating frequency. Hence we propose a hybrid solution where a larger shared cache (L1.5) is shared by multiple cores connected through a low-latency interconnect to small private caches (L1). However, it is still limited by large capacity miss with a small L1. Thus, we propose a sequential prefetch from L1 to L1.5 to improve the performance with little area overhead. Moreover, to cut the critical path for better timing, we optimized the core instruction fetch stage with non-blocking transfer by adopting a 4 x 32-bit ring buffer FIFO and adding a pipeline for the conditional branch. We present a detailed comparison of different instruction cache architectures' performance and energy efficiency recently proposed for Parallel Ultra-Low-Power clusters. On average, when executing a set of real-life IoT applications, our two-level cache improves the performance by up to 20% and loses 7% energy efficiency with respect to the private cache. Compared to a shared cache system, it improves performance by up to 17% and keeps the same energy efficiency. In the end, up to 20% timing (maximum frequency) improvement and software control enable the two-level instruction cache with prefetch adapt to various battery-powered usage cases to balance high performance and energy efficiency.

Replication vs erasure coding in data centric storage for wireless sensor networks

In-network storage of data in wireless sensor networks contributes to reduce the communications inside the network and to favor data aggregation. In this paper, we consider the use of n out of m codes and data dispersal in combination to in-network storage. In particular, we provide an abstract model of in-network storage to show how n out of m codes can be used, and we discuss how this can be achieved in five cases of study. We also define a model aimed at evaluating the probability of correct data encoding and decoding, we exploit this model and simulations to show how, in the cases of study, the parameters of the n out of m codes and the network should be configured in order to achieve correct data coding and decoding with high probability.

Replication Data Management,needs and solutions: an initial evaluation of conceptual approaches for integrating heterogeneous replication study data

Replication Data Management (RDM) aims at enabling the use of data collections from several iterations of an experiment. However, there are several major challenges to RDM from integrating data models and data from empirical study infrastructures that were not designed to cooperate, e.g., data model variation of local data sources. [Objective] In this paper we analyze RDM needs and evaluate conceptual RDM approaches to support replication researchers. [Method] We adapted the ATAM evaluation process to (a) analyze RDM use cases and needs of empirical replication study research groups and (b) compare three conceptual approaches to address these RDM needs: central data repositories with a fixed data model, heterogeneous local repositories, and an empirical ecosystem. [Results] While the central and local approaches have major issues that are hard to resolve in practice, the empirical ecosystem allows bridging current gaps in RDM from heterogeneous data sources. [Conclusions] The empirical ecosystem approach should be explored in diverse empirical environments.

Data partitioning and replication management in distributed geographic information system (GIS) database

This research presents several components encompassing the scope of the objective of Data Partitioning and Replication Management in Distributed GIS Database. Modern Geographic Information Systems (GIS) databases are often large and complicated. Therefore data partitioning and replication management problems need to be addresses in development of an efficient and scalable solution. ^ Part of the research is to study the patterns of geographical raster data processing and to propose the algorithms to improve availability of such data. These algorithms and approaches are targeting granularity of geographic data objects as well as data partitioning in geographic databases to achieve high data availability and Quality of Service(QoS) considering distributed data delivery and processing. To achieve this goal a dynamic, real-time approach for mosaicking digital images of different temporal and spatial characteristics into tiles is proposed. This dynamic approach reuses digital images upon demand and generates mosaicked tiles only for the required region according to user's requirements such as resolution, temporal range, and target bands to reduce redundancy in storage and to utilize available computing and storage resources more efficiently. ^ Another part of the research pursued methods for efficient acquiring of GIS data from external heterogeneous databases and Web services as well as end-user GIS data delivery enhancements, automation and 3D virtual reality presentation. ^ There are vast numbers of computing, network, and storage resources idling or not fully utilized available on the Internet. Proposed "Crawling Distributed Operating System "(CDOS) approach employs such resources and creates benefits for the hosts that lend their CPU, network, and storage resources to be used in GIS database context. ^ The results of this dissertation demonstrate effective ways to develop a highly scalable GIS database. The approach developed in this dissertation has resulted in creation of TerraFly GIS database that is used by US government, researchers, and general public to facilitate Web access to remotely-sensed imagery and GIS vector information. ^

Effects of data characteristics on the results of reserve selection algorithms

We tested the effects of four data characteristics on the results of reserve selection algorithms. The data characteristics were nestedness of features (land types in this case), rarity of features, size variation of sites (potential reserves) and size of data sets (numbers of sites and features). We manipulated data sets to produce three levels, with replication, of each of these data characteristics while holding the other three characteristics constant. We then used an optimizing algorithm and three heuristic algorithms to select sites to solve several reservation problems. We measured efficiency as the number or total area of selected sites, indicating the relative cost of a reserve system. Higher nestedness increased the efficiency of all algorithms (reduced the total cost of new reserves). Higher rarity reduced the efficiency of all algorithms (increased the total cost of new reserves). More variation in site size increased the efficiency of all algorithms expressed in terms of total area of selected sites. We measured the suboptimality of heuristic algorithms as the percentage increase of their results over optimal (minimum possible) results. Suboptimality is a measure of the reliability of heuristics as indicative costing analyses. Higher rarity reduced the suboptimality of heuristics (increased their reliability) and there is some evidence that more size variation did the same for the total area of selected sites. We discuss the implications of these results for the use of reserve selection algorithms as indicative and real-world planning tools.

Reorganization of terminator DNA upon binding replication terminator protein: Implications for the functional replication fork arrest complex

Termination of DNA replication in Bacillus subtilis involves the polar arrest of replication forks by a specific complex formed between the replication terminator protein (RTP) and DNA terminator sites. While determination of the crystal structure of RTP has facilitated our understanding of how a single RTP dimer interacts with terminator DNA, additional information is required in order to understand the assembly of a functional fork arrest complex, which requires an interaction between two RTP dimers and the terminator site. In this study, we show that the conformation of the major B. subtilis DNA terminator, Terl, becomes considerably distorted upon binding RTP. Binding of the first dimer of RTP to the B site of Terl causes the DNA to become slightly unwound and bent by similar to 40 degrees. Binding of a second dimer of RTP to the A site causes the bend angle to increase to similar to 60 degrees. We have used this new data to construct two plausible models that might explain how the ternary terminator complex can block DNA replication in a polar manner, in the first model, polarity of action is a consequence of the two RTP-DNA half-sites having different conformations. These different conformations result from different RTP-DNA contacts at each half-site (due to the intrinsic asymmetry at the terminator DNA), as well as interactions (direct or indirect) between the RTP dimers on the DNA. In the second model, polar fork arrest activity is a consequence of the different affinities of RTP for the A and B sites of the terminator DNA, modulated significantly by direct or indirect interactions between the RTP dimers.

Replication of bovine papillomavirus type 1 (BPV-1) DNA in Saccharomyces cerevisiae following infection with BPV-1 virions

Saccharomyces cerevisiae protoplasts exposed to bovine papillomavirus type 1 (BPV-1) virions demonstrated uptake of virions on electron microscopy. S. cerevisiae cells looked larger after exposure to BPV-1 virions, and cell wall regeneration was delayed. Southern blot hybridization of Hirt DNA from cells exposed to BPV-1 virions demonstrated BPV-1 DNA, which could be detected over 80 days of culture and at least 13 rounds of division. Two-dimensional gel analysis of Hirt DNA showed replicative intermediates, confirming that the BPV-1 genome was replicating within S. cerevisiae. Nicked circle, linear, and supercoiled BPV-1 DNA species were observed in Hirt DNA preparations from S. cerevisiae cells infected for over 50 days, and restriction digestion showed fragments hybridizing to BPV-1 in accord with the predicted restriction map for circular BPV-1 episomes. These data suggest that BPV-1 can infect S. cerevisiae and that BPV-1 episomes can replicate in the infected S. cerevisiae cells.

Molecular and functional analyses of Kunjin virus infectious cDNA clones demonstrate the essential roles for NS2A in virus assembly and for a nonconservative residue in NS3 in RNA replication

A number of full-length cDNA clones of Kunjin virus (KUN) were previously prepared; it was shown that two of them, pAKUN and FLSDX, differed in specific infectivities of corresponding in vitro transcribed RNAs by similar to100,000-fold (A. A. Khromykh et al., J. Virol. 72:7270-7279, 1998). In this study, we analyzed a possible genetic determinant(s) of the observed differences in infectivity initially by sequencing the entire cDNAs of both clones and comparing them with the published sequence of the parental KUN strain MRM61C. We found six common amino acid residues in both cDNA clones that were different from those in the published MRM61C sequence but were similar to those in the published sequences of other flaviviruses from the same subgroup. pAKUN clone had four additional codon changes, i.e., Ile59 to Asn and Arg175 to Lys in NS2A and Tyr518 to His and Ser557 to Pro in NS3. Three of these substitutions except the previously shown marker mutation, Arg175 to Lys in NS2A, reverted to the wild-type sequence in the virus eventually recovered from pAKUN RNA-transfected BHK cells, demonstrating the functional importance of these residues in viral replication and/or viral assembly. Exchange of corresponding DNA fragments between pAKUN and FLSDX clones and site-directed mutagenesis revealed that the Tyr518-to-His mutation in NS3 was responsible for an similar to5-fold decrease in specific infectivity of transcribed RNA, while the Ile59-to-Asn mutation in NS2A completely blocked virus production. Correction of the Asn59 in pAKUN NS2A to the wild-type lie residue resulted in complete restoration of RNA infectivity. Replication of KUN replicon RNA with an Ile59-to-Asn substitution in NS2A and with a Ser557-to-Pro substitution in NS3 was not affected, while the Tyr518-to-His substitution in NS3 led to severe inhibition of RNA replication. The impaired function of the mutated NS2A in production of infectious virus was complemented in trans by the helper wild-type NS2A produced from the KUN replicon RNA. However, replicon RNA with mutated NS2A could not be packaged in trans by the KUN structural proteins. The data demonstrated essential roles for the KUN nonstructural protein NS2A in virus assembly and for NS3 in RNA replication and identified specific single-amino-acid residues involved in these functions.

Estimating data divergence in cloud computing storage systems

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

TSKY: a dependable middleware solution for data privacy using public storage clouds

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Healing replicas in a software component replication system

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Concise server-wide causality management for eventually consistent data stores

Large scale distributed data stores rely on optimistic replication to scale and remain highly available in the face of net work partitions. Managing data without coordination results in eventually consistent data stores that allow for concurrent data updates. These systems often use anti-entropy mechanisms (like Merkle Trees) to detect and repair divergent data versions across nodes. However, in practice hash-based data structures are too expensive for large amounts of data and create too many false conflicts. Another aspect of eventual consistency is detecting write conflicts. Logical clocks are often used to track data causality, necessary to detect causally concurrent writes on the same key. However, there is a nonnegligible metadata overhead per key, which also keeps growing with time, proportional with the node churn rate. Another challenge is deleting keys while respecting causality: while the values can be deleted, perkey metadata cannot be permanently removed without coordination. Weintroduceanewcausalitymanagementframeworkforeventuallyconsistentdatastores,thatleveragesnodelogicalclocks(BitmappedVersion Vectors) and a new key logical clock (Dotted Causal Container) to provides advantages on multiple fronts: 1) a new efficient and lightweight anti-entropy mechanism; 2) greatly reduced per-key causality metadata size; 3) accurate key deletes without permanent metadata.

Rare genomic structural variants in complex disease: lessons from the replication of associations with obesity.

The limited ability of common variants to account for the genetic contribution to complex disease has prompted searches for rare variants of large effect, to partly explain the 'missing heritability'. Analyses of genome-wide genotyping data have identified genomic structural variants (GSVs) as a source of such rare causal variants. Recent studies have reported multiple GSV loci associated with risk of obesity. We attempted to replicate these associations by similar analysis of two familial-obesity case-control cohorts and a population cohort, and detected GSVs at 11 out of 18 loci, at frequencies similar to those previously reported. Based on their reported frequencies and effect sizes (OR≥25), we had sufficient statistical power to detect the large majority (80%) of genuine associations at these loci. However, only one obesity association was replicated. Deletion of a 220 kb region on chromosome 16p11.2 has a carrier population frequency of 2×10(-4) (95% confidence interval [9.6×10(-5)-3.1×10(-4)]); accounts overall for 0.5% [0.19%-0.82%] of severe childhood obesity cases (P = 3.8×10(-10); odds ratio = 25.0 [9.9-60.6]); and results in a mean body mass index (BMI) increase of 5.8 kg.m(-2) [1.8-10.3] in adults from the general population. We also attempted replication using BMI as a quantitative trait in our population cohort; associations with BMI at or near nominal significance were detected at two further loci near KIF2B and within FOXP2, but these did not survive correction for multiple testing. These findings emphasise several issues of importance when conducting rare GSV association, including the need for careful cohort selection and replication strategy, accurate GSV identification, and appropriate correction for multiple testing and/or control of false discovery rate. Moreover, they highlight the potential difficulty in replicating rare CNV associations across different populations. Nevertheless, we show that such studies are potentially valuable for the identification of variants making an appreciable contribution to complex disease.