On Distributed Storage Codes

Distributed storage systems are studied. The interest in such system has become relatively wide due to the increasing amount of information needed to be stored in data centers or different kinds of cloud systems. There are many kinds of solutions for storing the information into distributed devices regarding the needs of the system designer. This thesis studies the questions of designing such storage systems and also fundamental limits of such systems. Namely, the subjects of interest of this thesis include heterogeneous distributed storage systems, distributed storage systems with the exact repair property, and locally repairable codes. For distributed storage systems with either functional or exact repair, capacity results are proved. In the case of locally repairable codes, the minimum distance is studied. Constructions for exact-repairing codes between minimum bandwidth regeneration (MBR) and minimum storage regeneration (MSR) points are given. These codes exceed the time-sharing line of the extremal points in many cases. Other properties of exact-regenerating codes are also studied. For the heterogeneous setup, the main result is that the capacity of such systems is always smaller than or equal to the capacity of a homogeneous system with symmetric repair with average node size and average repair bandwidth. A randomized construction for a locally repairable code with good minimum distance is given. It is shown that a random linear code of certain natural type has a good minimum distance with high probability. Other properties of locally repairable codes are also studied.

Demand response — a different form of distributed storage?

Reduced flexibility of low carbon generation could pose new challenges for future energy systems. Both demand response and distributed storage may have a role to play in supporting future system balancing. This paper reviews how these technically different, but functionally similar approaches compare and compete with one another. Household survey data is used to test the effectiveness of price signals to deliver demand responses for appliances with a high degree of agency. The underlying unit of storage for different demand response options is discussed, with particular focus on the ability to enhance demand side flexibility in the residential sector. We conclude that a broad range of options, with different modes of storage, may need to be considered, if residential demand flexibility is to be maximised.

Reliable and persistent storage for CoDeS a distributed collaborative system

This research is aimed to find a solution for a distributed storage system adapted for CoDeS. By studying how DSSs work and how they are implemented, we can conclude how we can implement a DSS compatible with CoDeS requirements.

Enabling dCache to use preexistent mass storage systems

A mesura que la investigació depèn cada vegada més dels computadors, l'emmagatzematge de dades comença a convertir-se en un recurs escàs per als projectes, i suposa una gran part del cost total. Alguns projectes intenten resoldre aquest problema emprant emmagatzament distribuït. És doncs necessari que alguns centres proveeixin de grans quantitats d'emmagatzematge massiu de baix cost basat en cintes magnètiques. L'inconvenient d'aquesta solució és que el rendiment disminueix, particularment a l'hora de tractar-se de grans quantitats d'arxius petits. El nostre objectiu és crear un híbrid entre un sistema d'alt cost i rendiment basat en discs, i un de baix cost i rendiment basat en cintes. Per això, unirem dCache, un sistema d'emmagatzematge distribuït, amb Castor, un sistema d'emmagatzematge jeràrquic, creant sistemes de fitxers virtuals que contindran grans quantitats d'arxius petits per millorar el rendiment global del sistema.

An online data access prediction and optimization approach for distributed systems

Current scientific applications have been producing large amounts of data. The processing, handling and analysis of such data require large-scale computing infrastructures such as clusters and grids. In this area, studies aim at improving the performance of data-intensive applications by optimizing data accesses. In order to achieve this goal, distributed storage systems have been considering techniques of data replication, migration, distribution, and access parallelism. However, the main drawback of those studies is that they do not take into account application behavior to perform data access optimization. This limitation motivated this paper which applies strategies to support the online prediction of application behavior in order to optimize data access operations on distributed systems, without requiring any information on past executions. In order to accomplish such a goal, this approach organizes application behaviors as time series and, then, analyzes and classifies those series according to their properties. By knowing properties, the approach selects modeling techniques to represent series and perform predictions, which are, later on, used to optimize data access operations. This new approach was implemented and evaluated using the OptorSim simulator, sponsored by the LHC-CERN project and widely employed by the scientific community. Experiments confirm this new approach reduces application execution time in about 50 percent, specially when handling large amounts of data.

Creating proactive interference in immediate recall: Building a DOG from a DART, a MOP, and a FIG

Phonemic codes are accorded a privileged role in most current models of immediate serial recall, although their effects are apparent in short-term proactive interference (PI) effects as well. The present research looks at how assumptions concerning distributed representation and distributed storage involving both semantic and phonemic codes might be operationalized to produce PI in a short-term cued recall task. The four experiments reported here attempted to generate the phonemic characteristics of a nonrhyming, interfering foil from unrelated filler items in the same list. PI was observed when a rhyme of the foil was studied or when the three phonemes of the foil were distributed across three studied filler items. The results suggest that items in short-term memory are stored in terms of feature bundles and that all items are simultaneously available at retrieval.

Is the electric vehicle a solution for the wind power integration in the portuguese power system?

The integration of Plug-in electric vehicles in the transportation sector has a great potential to reduce oil dependency, the GHG emissions and to contribute for the integration of renewable sources into the electricity generation mix. Portugal has a high share of wind energy, and curtailment may occur, especially during the off-peak hours with high levels of hydro generation. In this context, the electric vehicles, seen as a distributed storage system, can help to reduce the potential wind curtailments and, therefore, increase the integration of wind power into the power system. In order to assess the energy and environmental benefits of this integration, a methodology based on a unit commitment and economic dispatch is adapted and implemented. From this methodology, the thermal generation costs, the CO2 emissions and the potential wind generation curtailment are computed. Simulation results show that a 10% penetration of electric vehicles in the Portuguese fleet would increase electrical load by 3% and reduce wind curtailment by only 26%. This results from the fact that the additional generation required to supply the electric vehicles is mostly thermal. The computed CO2 emissions of the EV are 92 g CO2/kWh which become closer to those of some new ICE engines.

Hadoop : processament distribuït de gran volum de dades en el núvol d'Apache

Avui en dia es genera un volum increïble de dades de diferents tipus i que provenen de multitud d'orígens. Els sistemes d'emmagatzematge i processament distribuït són els elements tecnològics que fan possible capturar aquest allau de dades i permeten donar-ne un valor a través d'anàlisis diversos. Hadoop, que integra un sistema d'emmagatzematge i processament distribuïts, s'ha convertit en l'estàndard de-facto per a aplicacions que necessiten una gran capacitat d'emmagatzematge, inclús de l'ordre de desenes de PBs. En aquest treball farem un estudi de Hadoop, analitzarem l'eficiència del seu sistema de durabilitat i en proposarem una alternativa.

Dynamic services in mobile ad hoc networks

The increasing diffusion of wireless-enabled portable devices is pushing toward the design of novel service scenarios, promoting temporary and opportunistic interactions in infrastructure-less environments. Mobile Ad Hoc Networks (MANET) are the general model of these higly dynamic networks that can be specialized, depending on application cases, in more specific and refined models such as Vehicular Ad Hoc Networks and Wireless Sensor Networks. Two interesting deployment cases are of increasing relevance: resource diffusion among users equipped with portable devices, such as laptops, smart phones or PDAs in crowded areas (termed dense MANET) and dissemination/indexing of monitoring information collected in Vehicular Sensor Networks. The extreme dynamicity of these scenarios calls for novel distributed protocols and services facilitating application development. To this aim we have designed middleware solutions supporting these challenging tasks. REDMAN manages, retrieves, and disseminates replicas of software resources in dense MANET; it implements novel lightweight protocols to maintain a desired replication degree despite participants mobility, and efficiently perform resource retrieval. REDMAN exploits the high-density assumption to achieve scalability and limited network overhead. Sensed data gathering and distributed indexing in Vehicular Networks raise similar issues: we propose a specific middleware support, called MobEyes, exploiting node mobility to opportunistically diffuse data summaries among neighbor vehicles. MobEyes creates a low-cost opportunistic distributed index to query the distributed storage and to determine the location of needed information. Extensive validation and testing of REDMAN and MobEyes prove the effectiveness of our original solutions in limiting communication overhead while maintaining the required accuracy of replication degree and indexing completeness, and demonstrates the feasibility of the middleware approach.

Adapting micro-economy of energy corporations to macroeconomy policies aiming at a sustainable economy

Esta Tesis surgió ante la intensidad y verosimilitud de varias señales o “warnings” asociadas a políticas dirigidas a reducir el peso del petróleo en el sector energético, tanto por razones económicas, como geopolíticas, como ambientales. Como tal Tesis se consolidó al ir incorporando elementos novedosos pero esenciales en el mundo petrolífero, particularmente las “tecnologías habilitantes”, tanto de incidencia directa, como el “fracking” como indirecta, del cual es un gran ejemplo el Vehículo Eléctrico (puro). La Tesis se definió y estructuró para elaborar una serie de indagaciones y disquisiciones, que comportaran un conjunto de conclusiones que fueran útiles para las corporaciones energéticas. También para la comprensión de la propia evolución del sector y de sus prestaciones técnicas y económicas, de cara a dar el servicio que los usuarios finales piden. Dentro de las tareas analíticas y reflexivas de la Tesis, se acuñaron ciertos términos conceptuales para explicar más certeramente la realidad del sector, y tal es el caso del “Investment burden”, que pondera la inversión específica (€/W) requerida por una instalación, con la duración del período de construcción y los riesgos tanto tangibles como regulatorios. Junto a ello la Tesis propone una herramienta de estudio y prognosis, denominada “Market integrated energy efficiency”, especialmente aplicable a dicotomías. Tal es el caso del coche térmico, versus coche eléctrico. El objetivo es optimizar una determinada actividad energética, o la productividad total del sector. Esta Tesis propone varias innovaciones, que se pueden agrupar en dos niveles: el primero dentro del campo de la Energía, y el segundo dentro del campo de las corporaciones, y de manera especial de las corporaciones del sector hidrocarburos. A nivel corporativo, la adaptación a la nueva realidad será función directa de la capacidad de cada corporación para desarrollar y/o comprar las tecnologías que permitan mantener o aumentar cuota de mercado. Las conclusiones de la Tesis apuntan a tres opciones principalmente para un replanteamiento corporativo: - Diversificación energética - Desplazamiento geográfico - Beneficiándose de posibles nuevos nichos tecnológicos, como son: • En upstream: Recuperación estimulada de petróleo mediante uso de energías renovables • En downstream: Aditivos orientados a reducir emisiones • En gestión del cambio: Almacenamiento energético con fines operativos Algunas políticas energéticas siguen la tendencia de crecimiento cero de algunos países de la OCDE. No obstante, la realidad mundial es muy diferente a la de esos países. Por ejemplo, según diversas estimaciones (basadas en bancos de datos solventes, referenciados en la Tesis) el número de vehículos aumentará desde aproximadamente mil millones en la actualidad hasta el doble en 2035; mientras que la producción de petróleo sólo aumentará de 95 a 145 millones de barriles al día. Un aumento del 50% frente a un aumento del 100%. Esto generará un curioso desajuste, que se empezará a sentir en unos pocos años. Las empresas y corporaciones del sector hidrocarburos pueden perder el monopolio que atesoran actualmente en el sector transporte frente a todas las demás fuentes energéticas. Esa pérdida puede quedar compensada por una mejor gestión de todas sus capacidades y una participación más integrada en el mundo de la energía, buscando sinergias donde hasta ahora no había sino distanciamiento. Los productos petrolíferos pueden alimentar cualquier tipo de maquina térmica, como las turbinas Brayton, o alimentar reformadores para la producción masiva de H2 para su posterior uso en pilas combustible. El almacenamiento de productos derivados del petróleo no es ningún reto ni plantea problema alguno; y sin embargo este almacenamiento es la llave para resolver muchos problemas. Es posible que el comercio de petróleo se haga menos volátil debido a los efectos asociados al almacenamiento; pero lo que es seguro es que la eficiencia energética de los usos de ese petróleo será más elevada. La Tesis partía de ciertas amenazas sobre el futuro del petróleo, pero tras el análisis realizado se puede vislumbrar un futuro prometedor en la fusión de políticas medioambientales coercitivas y las nuevas tecnologías emergentes del actual portafolio de oportunidades técnicas. ABSTRACT This Thesis rises from the force and the credibility of a number of warning signs linked to policies aimed at reducing the role of petroleum in the energy industry due to economical, geopolitical and environmental drives. As such Thesis, it grew up based on aggregating new but essentials elements into the petroleum sector. This is the case of “enabling technologies” that have a direct impact on the petroleum industry (such as fracking), or an indirect but deep impact (such as the full electrical vehicle). The Thesis was defined and structured in such a way that could convey useful conclusions for energy corporations through a series of inquiries and treatises. In addition to this, the Thesis also aims at understating la evolution of the energy industry and its capabilities both technical and economical, towards delivering the services required by end users. Within the analytical task performed in the Thesis, new terms were coined. They depict concepts that aid at explaining the facts of the energy industry. This is the case for “Investment burden”, it weights the specific capital investment (€/W) required to build a facility with the time that takes to build it, as well as other tangible risks as those posed by regulation. In addition to this, the Thesis puts forward an application designed for reviewing and predicting: the so called “Market integrated energy efficiency”, especially well-suited for dichotomies, very appealing for the case of the thermal car versus the electric car. The aim is to optimize energy related activity; or even the overall productivity of the system. The innovations proposed in this Thesis can be classified in two tiers. Tier one, within the energy sector; and tier two, related to Energy Corporation in general, but with oil and gas corporations at heart. From a corporate level, the adaptation to new energy era will be linked with the corporation capability to develop or acquire those technologies that will yield to retaining or enhancing market share. The Thesis highlights three options for corporate evolution: - diversification within Energy - geographic displacement - profiting new technologies relevant to important niches of work for the future, as: o Upstream: enhanced oil recovery using renewable energy sources (for upstream companies in the petroleum business) o Downstream: additives for reducing combustion emissions o Management of Change: operational energy storage Some energy policies tend to follow the zero-growth of some OECD countries, but the real thing could be very different. For instance, and according to estimates the number of vehicles in use will grow from 1 billion to more than double this figure 2035; but oil production will only grow from 95 million barrel/day to 145 (a 50% rise of versus an intensification of over a 100%). Hydrocarbon Corporation can lose the monopoly they currently hold over the supply of energy to transportation. This lose can be mitigated through an enhanced used of their capabilities and a higher degree of integration in the world of energy, exploring for synergies in those places were gaps were present. Petroleum products can be used to feed any type of thermal machine, as Brayton turbines, or steam reformers to produce H2 to be exploited in fuel cells. Storing petroleum products does not present any problem, but very many problems can be solved with them. Petroleum trading will likely be less volatile because of the smoothing effects of distributed storage, and indeed the efficiency in petroleum consumption will be much higher. The Thesis kicked off with a menace on the future of petroleum. However, at the end of the analysis, a bright future can be foreseen in the merging between highly demanding environmental policies and the relevant technologies of the currently emerging technical portfolio.

Coloring random graphs and maximizing local diversity

We study a variation of the graph coloring problem on random graphs of finite average connectivity. Given the number of colors, we aim to maximize the number of different colors at neighboring vertices (i.e. one edge distance) of any vertex. Two efficient algorithms, belief propagation and Walksat are adapted to carry out this task. We present experimental results based on two types of random graphs for different system sizes and identify the critical value of the connectivity for the algorithms to find a perfect solution. The problem and the suggested algorithms have practical relevance since various applications, such as distributed storage, can be mapped onto this problem.

Decentralized systems for the protection and portability of personal data

The General Data Protection Regulation (GDPR) has been designed to help promote a view in favor of the interests of individuals instead of large corporations. However, there is the need of more dedicated technologies that can help companies comply with GDPR while enabling people to exercise their rights. We argue that such a dedicated solution must address two main issues: the need for more transparency towards individuals regarding the management of their personal information and their often hindered ability to access and make interoperable personal data in a way that the exercise of one's rights would result in straightforward. We aim to provide a system that helps to push personal data management towards the individual's control, i.e., a personal information management system (PIMS). By using distributed storage and decentralized computing networks to control online services, users' personal information could be shifted towards those directly concerned, i.e., the data subjects. The use of Distributed Ledger Technologies (DLTs) and Decentralized File Storage (DFS) as an implementation of decentralized systems is of paramount importance in this case. The structure of this dissertation follows an incremental approach to describing a set of decentralized systems and models that revolves around personal data and their subjects. Each chapter of this dissertation builds up the previous one and discusses the technical implementation of a system and its relation with the corresponding regulations. We refer to the EU regulatory framework, including GDPR, eIDAS, and Data Governance Act, to build our final system architecture's functional and non-functional drivers. In our PIMS design, personal data is kept in a Personal Data Space (PDS) consisting of encrypted personal data referring to the subject stored in a DFS. On top of that, a network of authorization servers acts as a data intermediary to provide access to potential data recipients through smart contracts.

Calibration of structure in a distributed forecasting model for a semiarid flash flood: dynamic surface storage and channel roughness

Flash floods pose a significant danger for life and property. Unfortunately, in arid and semiarid environment the runoff generation shows a complex non-linear behavior with a strong spatial and temporal non-uniformity. As a result, the predictions made by physically-based simulations in semiarid areas are subject to great uncertainty, and a failure in the predictive behavior of existing models is common. Thus better descriptions of physical processes at the watershed scale need to be incorporated into the hydrological model structures. For example, terrain relief has been systematically considered static in flood modelling at the watershed scale. Here, we show that the integrated effect of small distributed relief variations originated through concurrent hydrological processes within a storm event was significant on the watershed scale hydrograph. We model these observations by introducing dynamic formulations of two relief-related parameters at diverse scales: maximum depression storage, and roughness coefficient in channels. In the final (a posteriori) model structure these parameters are allowed to be both time-constant or time-varying. The case under study is a convective storm in a semiarid Mediterranean watershed with ephemeral channels and high agricultural pressures (the Rambla del Albujón watershed; 556 km 2 ), which showed a complex multi-peak response. First, to obtain quasi-sensible simulations in the (a priori) model with time-constant relief-related parameters, a spatially distributed parameterization was strictly required. Second, a generalized likelihood uncertainty estimation (GLUE) inference applied to the improved model structure, and conditioned to observed nested hydrographs, showed that accounting for dynamic relief-related parameters led to improved simulations. The discussion is finally broadened by considering the use of the calibrated model both to analyze the sensitivity of the watershed to storm motion and to attempt the flood forecasting of a stratiform event with highly different behavior.

Using high resolution tracer data to constrain water storage, flux and age estimates in a spatially distributed rainfall-runoff model

Acknowledgements The authors would like to thank Jonathan Dick, Josie Geris, Jason Lessels, and Claire Tunaley for data collection and Audrey Innes for lab sample preparation. We also thank Christian Birkel for discussions about the model structure and comments on an earlier draft of the paper. Climatic data were provided by Iain Malcolm and Marine Scotland Fisheries at the Freshwater Lab, Pitlochry. Additional precipitation data were provided by the UK Meteorological Office and the British Atmospheric Data Centre (BADC).We thank the European Research Council ERC (project GA 335910 VEWA) for funding the VeWa project.