Partial replication in distributed software transactional memory

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

A distributed platform for the volunteer execution of workflows on a local area network

Thesis submitted in fulfilment of the requirements for the Degree of Master of Science in Computer Science

Integrated data model and DSL modifications

Companies are increasingly more and more dependent on distributed web-based software systems to support their businesses. This increases the need to maintain and extend software systems with up-to-date new features. Thus, the development process to introduce new features usually needs to be swift and agile, and the supporting software evolution process needs to be safe, fast, and efficient. However, this is usually a difficult and challenging task for a developer due to the lack of support offered by programming environments, frameworks, and database management systems. Changes needed at the code level, database model, and the actual data contained in the database must be planned and developed together and executed in a synchronized way. Even under a careful development discipline, the impact of changing an application data model is hard to predict. The lifetime of an application comprises changes and updates designed and tested using data, which is usually far from the real, production, data. So, coding DDL and DML SQL scripts to update database schema and data, is the usual (and hard) approach taken by developers. Such manual approach is error prone and disconnected from the real data in production, because developers may not know the exact impact of their changes. This work aims to improve the maintenance process in the context of Agile Platform by Outsystems. Our goal is to design and implement new data-model evolution features that ensure a safe support for change and a sound migration process. Our solution includes impact analysis mechanisms targeting the data model and the data itself. This provides, to developers, a safe, simple, and guided evolution process.

Design, implementation, and performance of a distributed and scalable sensor system for critical distance measurements in the CMS detector at LHC

The “CMS Safety Closing Sensors System” (SCSS, or CSS for brevity) is a remote monitoring system design to control safety clearance and tight mechanical movements of parts of the CMS detector, especially during CMS assembly phases. We present the different systems that makes SCSS: its sensor technologies, the readout system, the data acquisition and control software. We also report on calibration and installation details, which determine the resolution and limits of the system. We present as well our experience from the operation of the system and the analysis of the data collected since 2008. Special emphasis is given to study positioning reproducibility during detector assembly and understanding how the magnetic fields influence the detector structure.

Monitoring and analysis of queries in distributed databases

Dissertação de Mestrado em Engenharia Informática

Increasing the scalability and the speedup of a fish school distributed simulator

El present treball fa un anàlisi i desenvolupament sobre les millores en la velocitat i en l’escalabilitat d'un simulador distribuït de grups de peixos. Aquests resultats s’han obtingut fent servir una nova estratègia de comunicació per als processos lògics (LPs) i canvis en l'algoritme de selecció de veïns que s'aplica a cadascun dels peixos en cada pas de simulació. L’idea proposada permet que cada procés lògic anticipi futures necessitats de dades pels seus veïns reduint el temps de comunicació al limitar la quantitat de missatges intercanviats entre els LPs. El nou algoritme de selecció dels veïns es va desenvolupar amb l'objectiu d'evitar treball innecessari permetent la disminució de les instruccions executades en cada pas de simulació i per cadascun del peixos simulats reduint de forma significativa el temps de simulació.

Aplicaciones Single Program Multiple Data (SPMD) en ambientes distribuidos

Un reto al ejecutar las aplicaciones en un cluster es lograr mejorar las prestaciones utilizando los recursos de manera eficiente, y este reto es mayor al utilizar un ambiente distribuido. Teniendo en cuenta este reto, se proponen un conjunto de reglas para realizar el cómputo en cada uno de los nodos, basado en el análisis de cómputo y comunicaciones de las aplicaciones, se analiza un esquema de mapping de celdas y un método para planificar el orden de ejecución, tomando en consideración la ejecución por prioridad, donde las celdas de fronteras tienen una mayor prioridad con respecto a las celdas internas. En la experimentación se muestra el solapamiento del computo interno con las comunicaciones de las celdas fronteras, obteniendo resultados donde el Speedup aumenta y los niveles de eficiencia se mantienen por encima de un 85%, finalmente se obtiene ganancias de los tiempos de ejecución, concluyendo que si se puede diseñar un esquemas de solapamiento que permita que la ejecución de las aplicaciones SPMD en un cluster se hagan de forma eficiente.

Hydrogeophysical data integration at larger scales: Application of Bayesian sequential simulation for the characterization of heterogeneous alluvial aquifers

Knowledge of the spatial distribution of hydraulic conductivity (K) within an aquifer is critical for reliable predictions of solute transport and the development of effective groundwater management and/or remediation strategies. While core analyses and hydraulic logging can provide highly detailed information, such information is inherently localized around boreholes that tend to be sparsely distributed throughout the aquifer volume. Conversely, larger-scale hydraulic experiments like pumping and tracer tests provide relatively low-resolution estimates of K in the investigated subsurface region. As a result, traditional hydrogeological measurement techniques contain a gap in terms of spatial resolution and coverage, and they are often alone inadequate for characterizing heterogeneous aquifers. Geophysical methods have the potential to bridge this gap. The recent increased interest in the application of geophysical methods to hydrogeological problems is clearly evidenced by the formation and rapid growth of the domain of hydrogeophysics over the past decade (e.g., Rubin and Hubbard, 2005).

A Data mining approach to indirect inference

Consider a model with parameter phi, and an auxiliary model with parameter theta. Let phi be a randomly sampled from a given density over the known parameter space. Monte Carlo methods can be used to draw simulated data and compute the corresponding estimate of theta, say theta_tilde. A large set of tuples (phi, theta_tilde) can be generated in this manner. Nonparametric methods may be use to fit the function E(phi|theta_tilde=a), using these tuples. It is proposed to estimate phi using the fitted E(phi|theta_tilde=theta_hat), where theta_hat is the auxiliary estimate, using the real sample data. This is a consistent and asymptotically normally distributed estimator, under certain assumptions. Monte Carlo results for dynamic panel data and vector autoregressions show that this estimator can have very attractive small sample properties. Confidence intervals can be constructed using the quantiles of the phi for which theta_tilde is close to theta_hat. Such confidence intervals are found to have very accurate coverage.

Learning about Distributed Systems

This paper presents the "state of the art" about distributed systems and applications and it's focused on teaching about these systems. It presents different platforms where to run distributed applications and describes some development toolkits whose can be used to develop prototypes, practices and distributed applications. It also presents some existing distributed algorithms useful for class practices, and some tools to help managing distributed environments. Finally, the paper presents some teaching experiences with different approaches on how to teach about distributed systems.

Evaluation of MPI collectives for HPC applications on distributed virtualized environments

Through this study, we will measure how the collective MPI operations behaves in virtual and physical clusters, and its impact on the application performance. As we stated before, we will use as a test case the Weather Research and Forecasting simulations.

Possible solution of some essential zero problems in compositional data analysis

One of the tantalising remaining problems in compositional data analysis lies in how to deal with data sets in which there are components which are essential zeros. By anessential zero we mean a component which is truly zero, not something recorded as zero simply because the experimental design or the measuring instrument has not been sufficiently sensitive to detect a trace of the part. Such essential zeros occur inmany compositional situations, such as household budget patterns, time budgets,palaeontological zonation studies, ecological abundance studies. Devices such as nonzero replacement and amalgamation are almost invariably ad hoc and unsuccessful insuch situations. From consideration of such examples it seems sensible to build up amodel in two stages, the first determining where the zeros will occur and the secondhow the unit available is distributed among the non-zero parts. In this paper we suggest two such models, an independent binomial conditional logistic normal model and a hierarchical dependent binomial conditional logistic normal model. The compositional data in such modelling consist of an incidence matrix and a conditional compositional matrix. Interesting statistical problems arise, such as the question of estimability of parameters, the nature of the computational process for the estimation of both the incidence and compositional parameters caused by the complexity of the subcompositional structure, the formation of meaningful hypotheses, and the devising of suitable testing methodology within a lattice of such essential zero-compositional hypotheses. The methodology is illustrated by application to both simulated and real compositional data

Tree-based message diffusion for managing replicated data in unreliable and resource-constrained peer-to-peer environment

Abstract This thesis proposes a set of adaptive broadcast solutions and an adaptive data replication solution to support the deployment of P2P applications. P2P applications are an emerging type of distributed applications that are running on top of P2P networks. Typical P2P applications are video streaming, file sharing, etc. While interesting because they are fully distributed, P2P applications suffer from several deployment problems, due to the nature of the environment on which they perform. Indeed, defining an application on top of a P2P network often means defining an application where peers contribute resources in exchange for their ability to use the P2P application. For example, in P2P file sharing application, while the user is downloading some file, the P2P application is in parallel serving that file to other users. Such peers could have limited hardware resources, e.g., CPU, bandwidth and memory or the end-user could decide to limit the resources it dedicates to the P2P application a priori. In addition, a P2P network is typically emerged into an unreliable environment, where communication links and processes are subject to message losses and crashes, respectively. To support P2P applications, this thesis proposes a set of services that address some underlying constraints related to the nature of P2P networks. The proposed services include a set of adaptive broadcast solutions and an adaptive data replication solution that can be used as the basis of several P2P applications. Our data replication solution permits to increase availability and to reduce the communication overhead. The broadcast solutions aim, at providing a communication substrate encapsulating one of the key communication paradigms used by P2P applications: broadcast. Our broadcast solutions typically aim at offering reliability and scalability to some upper layer, be it an end-to-end P2P application or another system-level layer, such as a data replication layer. Our contributions are organized in a protocol stack made of three layers. In each layer, we propose a set of adaptive protocols that address specific constraints imposed by the environment. Each protocol is evaluated through a set of simulations. The adaptiveness aspect of our solutions relies on the fact that they take into account the constraints of the underlying system in a proactive manner. To model these constraints, we define an environment approximation algorithm allowing us to obtain an approximated view about the system or part of it. This approximated view includes the topology and the components reliability expressed in probabilistic terms. To adapt to the underlying system constraints, the proposed broadcast solutions route messages through tree overlays permitting to maximize the broadcast reliability. Here, the broadcast reliability is expressed as a function of the selected paths reliability and of the use of available resources. These resources are modeled in terms of quotas of messages translating the receiving and sending capacities at each node. To allow a deployment in a large-scale system, we take into account the available memory at processes by limiting the view they have to maintain about the system. Using this partial view, we propose three scalable broadcast algorithms, which are based on a propagation overlay that tends to the global tree overlay and adapts to some constraints of the underlying system. At a higher level, this thesis also proposes a data replication solution that is adaptive both in terms of replica placement and in terms of request routing. At the routing level, this solution takes the unreliability of the environment into account, in order to maximize reliable delivery of requests. At the replica placement level, the dynamically changing origin and frequency of read/write requests are analyzed, in order to define a set of replica that minimizes communication cost.

@neurIST infrastructure for advanced disease management through integration of heterogeneous data, computing, and complex processing services

The increasing volume of data describing humandisease processes and the growing complexity of understanding, managing, and sharing such data presents a huge challenge for clinicians and medical researchers. This paper presents the@neurIST system, which provides an infrastructure for biomedical research while aiding clinical care, by bringing together heterogeneous data and complex processing and computing services. Although @neurIST targets the investigation and treatment of cerebral aneurysms, the system’s architecture is generic enough that it could be adapted to the treatment of other diseases.Innovations in @neurIST include confining the patient data pertaining to aneurysms inside a single environment that offers cliniciansthe tools to analyze and interpret patient data and make use of knowledge-based guidance in planning their treatment. Medicalresearchers gain access to a critical mass of aneurysm related data due to the system’s ability to federate distributed informationsources. A semantically mediated grid infrastructure ensures that both clinicians and researchers are able to seamlessly access andwork on data that is distributed across multiple sites in a secure way in addition to providing computing resources on demand forperforming computationally intensive simulations for treatment planning and research.

The Exotic Palaeo-Thethys Terrane in Central Iran: New geological data from Anarak, Jandaq and Posht-e-Badam areas

Résumé Le « terrane » d'Anarak-Jandak occupe une position géologique clé au nord-ouest du Microcontinent Centre-East Iranien (CE1M), connecté avec le Bloc du Grand Kavir et la ceinture métamorphique de Sanandaj-Sirjan. Nous discutons ici l'origine de ces différentes unités, reliées jusqu'à présent à des épisodes orogéniques d'âge Précambrien à Paléozoïque inférieur, pour conclure finalement de leur affinité paléotéthysienne. Leur histoire commence par un épisode de rifting d'âge Ordovicien supérieur-Dévonien inférieur, pour se terminer au Trias par la collision des blocs Cimmériens dérivé du Gondwana avec le Bloc du Turan d'affinité asiatique (événement Eocimmérien). La plus importante unité métamorphique affleurant au sud-ouest de la région de Jandak-Anarak-Kaboudan est une épaisse séquence silicoclastique à grains fins contenant des blocs ophiolitiques (marginal-sea-type), et des associations basalte-gabbro à signatures géochimiques de type supra-subduction. Dans la région de Nakhlak, nous avons daté ces gabbros par la méthode U-Pb à 387f0.11 Ma ; les roches métamorphiques pélitiques ont donné des âges de refroidissement Ar-Ar pour la muscovite de 320 à 333 Ma. Ce complexe d'accrétion "varisque" a été métamorphisé dans le faciès schiste vert-amphibolite au cours de l'accrétion de la ceinture granitique d'Airekan, d'âge Cambrien inférieur (549±15 Ma par la méthode U/Pb), qui affleure aujourd'hui à l'extrémité nord-ouest du terrane d'Anarak-Jandak . La subduction vers le nord de l'océan Paléotéthys depuis le Paléazoïque supérieur jusqu'au Trias, a permis l'accumulation de grandes quantités de matériel océanique dans la zone de subduction. Par exemple, une succession de guyots (Anarak, Kaboudan, et Meraji Seamounts) et de hauts sous-marins, entrés en collision oblique avec le prisme d'accrétion, est à l'origine d'un léger métamorphisme de type HP qui affecte ces séries {âges Ar-Ar de 280 à 230 Ma). De plus, le magmatisme bimodal de Chah Gorbeh est caractérisé d'une part par des roches de type trondjémite-gabbros (262 Ma), d'autre part par des laves en coussin de type basaltes alcalins-rhyolites; ces roches magmatiques ont recoupé l'ophiolite d'Anarak lors de la mise en place de cette dernière dans la fosse interne de subduction. Quant au prisme d'accrétion de Doshakh, d'âge essentiellement Permien supérieur, i1 a été accrété le long de la marge continentale et métamorphisé dans le faciès schiste vert. La fermeture de la Paléotéthys s'enregistre finalement par la sédimentation dans le bassin d'avant pays du flysch de Bayazeh, d'âge probable Triasique. Le matériel issu de l'arc magmatique de la Paléotéthys est très bien préservé dans les dépôts infra-arc Dévonien supérieur-Carbonifère de Godar-e-Siah, ainsi que dans la succession d'avant-arc de Nakhlak. Pendant l'intervalle Paléozoïque supérieur-Trias, la région de Jandak a été soumise à un régime extensif de type bassin d'arrière-arc, dont un témoin pourrait être la ceinture ophiolitique d'Arusan, elle-même comparable aux écailles ophiolitiques d'Aghdarband au nord-est de l'Iran. Cet ensemble métamorphique est recoupé par des granites d'arc à collisionnel datés à 215±15 Ma. Dans la région de Yazd, témoin de la marge passive Cimmérienne, la sédimentation syn-rift Silurienne à Dévonienne inférieure a été interrompue pendant l'intervalle Trias moyen-Trias supérieur; il en a été de même pour les dépôts de plate-forme Paléozoïque supérieur. L'érosion, qui dans ce dernier cas a atteint le Permien, pourrait être liée au bombement flexural de la marge passive. La collision finale n'a pas induit de déformations trop importantes, et se caractérise par la mise en place de nappes sur la marge passive. Cet événement est scellé par des dépôts molassique du Lias. D'un point de vue régional, la zone s'étendant actuellement de la Mer Noire au Pamir a été soumise à six épisodes d'extension-compression du Jurassique inférieur (début du l'ouverture en position arrière-arc de la Néotéthys) à l'Eocène moyen. Par exemple, le terrane d'AnarakJandak, probablement situé entre le Kopeh Dagh et la plate-forme nord Afghane, s'est complètement détaché de sa patrie d'origine au début du Crétacé supérieur. Des preuves de cet événement se retrouvent dans les séries de plate-forme de Khur (préservation de séries syn-rift puis de marge passive). Les ophiolites de Nain et de Sabzevar sont de plus interprétée comme un témoin de l'existence de ce bassin d'arrière-arc. Dans l'intervalle Eocène-Oligocène, l'indentation par la plaque indienne de l'Eurasie a été contemporaine de la rotation horaire de fragments de l'ancien microcontinent Iranien et de la formation du CEIM. Cette rotation est responsable du transport du terrane d'Anarak-Jandak vers sa position actuelle en Iran Central, et de la dislocation de Terranes de moindre importance, comme le bloc de Posht-e Badam. Depuis le Miocène supérieur, et à la suite de la collision entre l'Arabie et l'Iran, le ternane d'Anarak-Jandak a subi des déformations liées à l'activité d'une zone de cisaillement dextre parallèle à la suture du Zagros, à l'arrière de l'arc magmatique d'Uromieh-Dokhtar. Résumé large public Le Microcontinent Centre-Est Iranien occupe une position géologique clé au centre de l'Iran. Les différentes unités qui le composent, reliées jusqu'à présent à des épisodes orogéniques d'âge Précambrien à Paléozoïque inférieur, sont maintenant rajeunies et liés à la fermeture de l'océean Paléotéthys. Leur histoire commence par un épisode de rifting d'âge Ordovicien supérieur à Dévonien inférieur, pour se terminer au Trias par la collision des- blocs Cimmériens, dérivés du Gondwana, avec le Bloc du Turan d'affinité asiatique. Dans la marge active asiatique de la Paléotéthys, nous avons daté les restes d'un océan marginal à 387±0.11 Ma. Ce complexe d'accrétion a été métamorphisé au cours de la réaccrétion de la ceinture granitique d'Airekan, d'âge Cambrien inférieur (549±15 Ma), qui affleure aujourd'hui à l'extrémité nord-ouest du « terrane » d'Anarak-Jandak correspondant à la plus grande partie de la région étudiée. Le matériel issu de l'arc magmatique de la Paléotéthys est très bien préservé et daté du Dévonien supérieur-Carbonifère. Pendant l'intervalle Paléozoïque supérieur-Trias, la région a été soumise à un régime extensif de type bassin d'arrière-arc, dont un témoin pourrait être la ceinture ophiolitique d'Arusan, comparable aux écailles ophiolitiques d'Aghdarband au nord-est de l'Iran. Cet ensemble métamorphique est recoupé par des granites datés à 215±15 Ma. La subduction vers le nord de l'océan Paléotéthys depuis le Paléozoïque supérieur jusqu'au Trias, a permis l'accumulation de grandes quantités de matériel océanique dans la zone de subduction. Par exemple, une succession de volcans sous-marins, entrés en collision avec le prisme d'accrétion, est à l'origine d'un léger métamorphisme de type HP qui affecte ces séries (280 à 230 Ma). Quant au prisme d'accrétion de Doshakh, d'âge essentiellement Permien supérieur, il a été mis en place le long de la marge continentale et métamorphisé dans le faciès schiste vert. La fermeture de la Paléotéthys s'enregistre finalement par la sédimentation dans le bassin d'avant pays du flysch de Bayazeh, d'âge Triasique. Dans la région de Yazd, on trouve les témoins de la marge passive Cimmérienne, la sédimentation syn-rift Silurienne à Dévonienne inférieure a été interrompue pendant l'intervalle Trias moyen-Trias supérieur, marqué par la flexuration de la marge passive lorsqu'elle rentra en collision avec la marge active asiatique. Cet événement est scellé par des dépôts molassique à charbon du Lias. Le «terrane» d'Anarak-Jandak, probablement situé à l'origine entre le Kopeh Dagh et la plate-forme nord Afghane, s'est complètement détaché de cette région au début du Crétacé supérieur lors de l'ouverture d'un bassin d'arrière-arc, engendré, cette fois, par la subduction de l'océan Néotéthys situé au sud des blocs cimmériens. Des preuves de cet événement se retrouvent dans les séries syn-rift, puis de marge passive de Khour. Les ophiolites de Nain et de Sabzevar sont interprétées comme un témoin de l'existence de ce bassin d'arrière-arc. Dans l'intervalle Eocène-Oligocène, l'indentation de l'Eurasie par la plaque indienne a été contemporaine de la rotation horaire de fragments de l'ancien microcontinent centre-Iranien. Cette rotation de près de 90° est responsable du transport du « terrane » d'Anarak-Jandak vers sa position actuelle. Abstract The Anarak-Jandaq terrane occupies a strategic geological situation at the north-western part of the Central-East Iranian Microcontinent (CEIM) and in connection with the Great Kavir Block and Sanandaj-Sirjan metamorphic belt. Our recent findings redefine the origin of these mentioned areas so far attributed to the Precambrian-Early Palaeozoic orogenic episodes, to be now directly related to the tectonic evolution of the Palaeo-Tethys Ocean, commenced by Late Ordovician-Early Devonian rifting events and terminated in the Triassic by the Eocimmerian tectonic event due to the collision of the Cimmerian blocks with the Asiatic Turan block. The most distributed metamorphic unit that is exposed from the south-west of Jandaq to the Anarak and Kaboudan areas is a thick and fine grain siliciclastic sequence accompanied by marginal-sea-basin ophiolitic blocks including basalt-gabbro association with supra-subduction-geochemical signature. These gabbros in the Nakhlak area were dated by U/Pb method at 387.6 ± 0.11 Ma and the metamorphic pelitic rocks yielded a range of 320 to 333 Ma muscovite-cooling ages based on 40Ar/39 Ar method. This "Variscan" accretionary complex was metamorphosed in greenschist-amphibolite facies during accretion to the Lower Cambrian Airekan granitic belt (549 ± 15 Ma by U/Pb method) that crops out at the northwestern edge of the Anarak-Jandaq terrane. Continued northward subduction of the Palaeo-Tethys Ocean during the entire Late Palaeozoic-Middle Triassic brought huge amount of oceanic material to the subduction zone. One chain of Carboniferous-Triassic oceanic rises and seamounts (the Anarak, Kaboudan, and Meraji Seamounts) obliquely collided with the accretionary wedge and created a mild HP metamorphic event (280-230 Ma based on 40Ar/39Ar results). Bimodal magmatism of the Chah Gorbeh area is characterized by a 262 Ma trondjemite-gabbro as well as pillow alkalibasalts-rhyolites which intruded the Anarak ophiolite when it was being emplaced within the inner-wall trench. The mainly Late Permian-Triassic Doshakh wedge was accreted along the continent and metamorphosed under lower greenschist facies and the probable Triassic Bayazeh flysch filled the foreland basin during the final closure. The Palaeo-Tethys magmatic arc products have been well preserved in the Late Devonian-Carboniferous Godar-e-Siah intra-arc deposits and the Triassic Nakhlak fore-arc succession. During the Late Palaeozoic-Triassic times, the Jandaq area has been affected by back-arc extension and probably the Arusan ophiolitic belt is the remnant of this narrow basin comparable to the Aqdarband ophiolitic remnant in north-east Iran. This metamorphic belt was intruded by 215 ± 15 Ma arc to collisional granites. In the passive margin of the Cimmerian block, on the Yazd region, the Silurian-Early Devonian syn-rift succession as well as the nearly continuous Upper Palaeozoic platform-type deposition was interrupted during the Middle to Late Triassic time, local erosion down to Devonian levels may be related to flexural bulge erosion. The collision event was not so strong to generate intensive deformation but was accompanied by some nappe thrusting onto the passive margin. It is finally unconformably covered by Liassic continental molassic deposits. Related to the onset of Neo-Tethyan back-arc opening in Early Jurassic to Mid-Eocene times, six periods of extensional-compressional events have differently influenced an elongated area, extending from the West Black Sea to Pamir. The Anarak-Jandaq terrane which was situated somewhere in this affected area, probably between the Kopeh Dagh and North Afghan platform, was completely detached from its source at the beginning of the Late Cretaceous