BUILDING EFFICIENT AND COST-EFFECTIVE CLOUD-BASED BIG DATA MANAGEMENT SYSTEMS

In today’s big data world, data is being produced in massive volumes, at great velocity and from a variety of different sources such as mobile devices, sensors, a plethora of small devices hooked to the internet (Internet of Things), social networks, communication networks and many others. Interactive querying and large-scale analytics are being increasingly used to derive value out of this big data. A large portion of this data is being stored and processed in the Cloud due the several advantages provided by the Cloud such as scalability, elasticity, availability, low cost of ownership and the overall economies of scale. There is thus, a growing need for large-scale cloud-based data management systems that can support real-time ingest, storage and processing of large volumes of heterogeneous data. However, in the pay-as-you-go Cloud environment, the cost of analytics can grow linearly with the time and resources required. Reducing the cost of data analytics in the Cloud thus remains a primary challenge. In my dissertation research, I have focused on building efficient and cost-effective cloud-based data management systems for different application domains that are predominant in cloud computing environments. In the first part of my dissertation, I address the problem of reducing the cost of transactional workloads on relational databases to support database-as-a-service in the Cloud. The primary challenges in supporting such workloads include choosing how to partition the data across a large number of machines, minimizing the number of distributed transactions, providing high data availability, and tolerating failures gracefully. I have designed, built and evaluated SWORD, an end-to-end scalable online transaction processing system, that utilizes workload-aware data placement and replication to minimize the number of distributed transactions that incorporates a suite of novel techniques to significantly reduce the overheads incurred both during the initial placement of data, and during query execution at runtime. In the second part of my dissertation, I focus on sampling-based progressive analytics as a means to reduce the cost of data analytics in the relational domain. Sampling has been traditionally used by data scientists to get progressive answers to complex analytical tasks over large volumes of data. Typically, this involves manually extracting samples of increasing data size (progressive samples) for exploratory querying. This provides the data scientists with user control, repeatable semantics, and result provenance. However, such solutions result in tedious workflows that preclude the reuse of work across samples. On the other hand, existing approximate query processing systems report early results, but do not offer the above benefits for complex ad-hoc queries. I propose a new progressive data-parallel computation framework, NOW!, that provides support for progressive analytics over big data. In particular, NOW! enables progressive relational (SQL) query support in the Cloud using unique progress semantics that allow efficient and deterministic query processing over samples providing meaningful early results and provenance to data scientists. NOW! enables the provision of early results using significantly fewer resources thereby enabling a substantial reduction in the cost incurred during such analytics. Finally, I propose NSCALE, a system for efficient and cost-effective complex analytics on large-scale graph-structured data in the Cloud. The system is based on the key observation that a wide range of complex analysis tasks over graph data require processing and reasoning about a large number of multi-hop neighborhoods or subgraphs in the graph; examples include ego network analysis, motif counting in biological networks, finding social circles in social networks, personalized recommendations, link prediction, etc. These tasks are not well served by existing vertex-centric graph processing frameworks whose computation and execution models limit the user program to directly access the state of a single vertex, resulting in high execution overheads. Further, the lack of support for extracting the relevant portions of the graph that are of interest to an analysis task and loading it onto distributed memory leads to poor scalability. NSCALE allows users to write programs at the level of neighborhoods or subgraphs rather than at the level of vertices, and to declaratively specify the subgraphs of interest. It enables the efficient distributed execution of these neighborhood-centric complex analysis tasks over largescale graphs, while minimizing resource consumption and communication cost, thereby substantially reducing the overall cost of graph data analytics in the Cloud. The results of our extensive experimental evaluation of these prototypes with several real-world data sets and applications validate the effectiveness of our techniques which provide orders-of-magnitude reductions in the overheads of distributed data querying and analysis in the Cloud.

The FOOT experiment: Trigger and Data Acquisition (TDAQ) development and data analysis

Hadrontherapy employs high-energy beams of charged particles (protons and heavier ions) to treat deep-seated tumours: these particles have a favourable depth-dose distribution in tissue characterized by a low dose in the entrance channel and a sharp maximum (Bragg peak) near the end of their path. In these treatments nuclear interactions have to be considered: beam particles can fragment in the human body releasing a non-zero dose beyond the Bragg peak while fragments of human body nuclei can modify the dose released in healthy tissues. These effects are still in question given the lack of interesting cross sections data. Also space radioprotection can profit by fragmentation cross section measurements: the interest in long-term manned space missions beyond Low Earth Orbit is growing in these years but it has to cope with major health risks due to space radiation. To this end, risk models are under study: however, huge gaps in fragmentation cross sections data are currently present preventing an accurate benchmark of deterministic and Monte Carlo codes. To fill these gaps in data, the FOOT (FragmentatiOn Of Target) experiment was proposed. It is composed by two independent and complementary setups, an Emulsion Cloud Chamber and an electronic setup composed by several subdetectors providing redundant measurements of kinematic properties of fragments produced in nuclear interactions between a beam and a target. FOOT aims to measure double differential cross sections both in angle and kinetic energy which is the most complete information to address existing questions. In this Ph.D. thesis, the development of the Trigger and Data Acquisition system for the FOOT electronic setup and a first analysis of 400 MeV/u 16O beam on Carbon target data acquired in July 2021 at GSI (Darmstadt, Germany) are presented. When possible, a comparison with other available measurements is also reported.

Big data analysis systems in IoE environments for managing privacy and data protection: pseudonymity, de-anonymization and the right to be forgotten

The thesis represents the conclusive outcome of the European Joint Doctorate programmein Law, Science & Technology funded by the European Commission with the instrument Marie Skłodowska-Curie Innovative Training Networks actions inside of the H2020, grantagreement n. 814177. The tension between data protection and privacy from one side, and the need of granting further uses of processed personal datails is investigated, drawing the lines of the technological development of the de-anonymization/re-identification risk with an explorative survey. After acknowledging its span, it is questioned whether a certain degree of anonymity can still be granted focusing on a double perspective: an objective and a subjective perspective. The objective perspective focuses on the data processing models per se, while the subjective perspective investigates whether the distribution of roles and responsibilities among stakeholders can ensure data anonymity.

Construction and test of a cosmic ray telescope based on CMS Drift Tube chambers and data acquisition prototypes of the Phase-2 upgrade

In this thesis work, a cosmic-ray telescope was set up in the INFN laboratories in Bologna using smaller size replicas of CMS Drift Tubes chambers, called MiniDTs, to test and develop new electronics for the CMS Phase-2 upgrade. The MiniDTs were assembled in INFN National Laboratory in Legnaro, Italy. Scintillator tiles complete the telescope, providing a signal independent of the MiniDTs for offline analysis. The telescope readout is a test system for the CMS Phase-2 upgrade data acquisition design. The readout is based on the early prototype of a radiation-hard FPGA-based board developed for the High Luminosity LHC CMS upgrade, called On Board electronics for Drift Tubes. Once the set-up was operational, we developed an online monitor to display in real-time the most important observables to check the quality of the data acquisition. We performed an offline analysis of the collected data using a custom version of CMS software tools, which allowed us to estimate the time pedestal and drift velocity in each chamber, evaluate the efficiency of the different DT cells, and measure the space and time resolution of the telescope system.

Desenvolvimento de uma metodologia baseada em redes neuronais para o cálculo da energia produzida por aerogeradores

O uso da energia eólica para a produção de eletricidade apresenta na última década um crescimento apreciável. Monitorizar o desempenho dos aerogeradores torna-se um processo incontornável, quer por motivos financeiros, quer por questões operacionais. Os investimentos despendidos na construção de parques eólicos são muito consideráveis, pelo que é essencial a análise constante dos aspetos preponderantes no retorno do investimento. A maximização da energia produzida por cada aerogerador é o objetivo principal da monitorização dos parques eólicos. Os sistemas Supervisory Control and Data Acquisition (SCADAs) instalados nos parques eólicos permitem uma supervisão em tempo real relativamente ao estado e funcionamento dos aerogeradores, adquirindo uma elevada importância na avaliação dos rendimentos energéticos e anomalias de funcionamento, garantido desta forma melhorias de produtividade. O objetivo deste trabalho é estimar a energia produzida pelos aerogeradores quando ocorrem falhas de comunicação com o seu contador interno ou avaria do mesmo. A ocorrência destas situações não permite a monitorização da energia produzida durante esse período. Foram analisados dados operacionais dos aerogeradores relativos a um parque eólico localizado na zona Norte de Portugal, sendo usados os dados recolhidos pelo sistema SCADA sobre a forma de médias de 10 min referentes ao período de janeiro de 2011 a agosto 2011. O desempenho da rede neuronal depende da qualidade e quantidade do conjunto de dados usados para o treino da rede. Os dados usados devem representar de forma fiel o estado que se pretende para o equipamento. Para a obtenção do objetivo proposto foi fundamental a identificação das grandezas disponíveis a utilizar no método de cálculo da energia produzida. Os resultados obtidos com aplicação das redes neuronais no método de cálculo da energia produzida por aerogeradores demonstram que independentemente do período de indisponibilidade da informação referente à energia produzida é possível estimar o valor da mesma.

Análise e implementação de filtros e controladores num sistema de processamento digital de sinal

Dissertação para obtenção do grau de Mestre em Engenharia Electrotécnica Ramo de Automação e Electrónica Industrial

A Communication and Resources Management Scheme to Support the Smart Grid Integration of Multiplayers Access to Resources Information

The increasing and intensive integration of distributed energy resources into distribution systems requires adequate methodologies to ensure a secure operation according to the smart grid paradigm. In this context, SCADA (Supervisory Control and Data Acquisition) systems are an essential infrastructure. This paper presents a conceptual design of a communication and resources management scheme based on an intelligent SCADA with a decentralized, flexible, and intelligent approach, adaptive to the context (context awareness). The methodology is used to support the energy resource management considering all the involved costs, power flows, and electricity prices leading to the network reconfiguration. The methodology also addresses the definition of the information access permissions of each player to each resource. The paper includes a 33-bus network used in a case study that considers an intensive use of distributed energy resources in five distinct implemented operation contexts.

Sistema d'adquisició de temperatures de 2 canals via R.F.

Poder mesurar i enregistrar diferents tipus de magnituds com pressió, força, temperatura etc. s’ha convertit en una necessitat per moltes aplicacions actuals. Aquestes magnituds poden tenir procedències molt diverses, tals com l’entorn, o poden ser generades per sistemes mecànics, elèctrics, etc. Per tal de poder adquirir aquestes magnituds, s’utilitzen els sistemes d’adquisició de dades. Aquests sistemes, prenen mostres analògiques del món real, i les transformen en dades digitals que poden ser manipulades per un sistema electrònic. Pràcticament qualsevol magnitud es pot mesurar utilitzant el sensor adient. Una magnitud molt utilitzada en sistemes d’adquisició de dades, és la temperatura. Els sistemes d’adquisició de temperatures estan molt generalitzats, i podem trobar-los com a sistemes, on l’objectiu és mostrar les dades adquirides, o podem trobar-los formant part de sistemes de control, aportant uns inputs necessaris per el seu correcte funcionament, garantir-ne l’estabilitat, seguretat etc. Aquest projecte, promogut per l’empresa Elausa, s’encarregarà d’adquirir, el senyal d’entrada de 2 Termoparells. Aquests mesuraran temperatures de circuits electrònics, que es trobaran dintre la càmera climàtica de Elausa, sotmesos a diferents condicions de temperatura, per tal de rebre l’homologació del circuit. El sistema haurà de poder mostrar les dades adquirides en temps real, i emmagatzemar-les en un PC que estarà ubicat en una oficina, situada a uns 30 m de distància de la sala on es farà el test. El sistema constarà d’un circuit electrònic que adquirirà, i condicionarà el senyal de sortida dels termoparells, per adaptar-lo a la tensió d’entrada d’un convertidor analògic digital, del microcontrolador integrat en aquesta placa. Seguidament aquesta informació, s’enviarà a través d’un mòdul transmissor de radiofreqüència, cap al PC on es visualitzaran les dades adquirides. Els objectius plantejats són els següents: - Dissenyar el circuit electrònic d’adquisició i condicionament del senyal. - Dissenyar, fabricar i muntar el circuit imprès de la placa d’adquisició. - Realitzar el programa de control del microcontrolador. - Realitzar el programa per presentar i desar les dades en un PC. - El sistema ha d’adquirir 2 temperatures, a través de Termoparells amb un rang d’entrada de -40ºC a +240ºC - S’ha de transmetre les dades via R.F. Els resultats del projecte han estat satisfactoris i s’han complert els objectius plantejats.

Wireless Sensor Networks for Infrastructure Monitoring, TR-611, 2010

A good system of preventive bridge maintenance enhances the ability of engineers to manage and monitor bridge conditions, and take proper action at the right time. Traditionally infrastructure inspection is performed via infrequent periodical visual inspection in the field. Wireless sensor technology provides an alternative cost-effective approach for constant monitoring of infrastructures. Scientific data-acquisition systems make reliable structural measurements, even in inaccessible and harsh environments by using wireless sensors. With advances in sensor technology and availability of low cost integrated circuits, a wireless monitoring sensor network has been considered to be the new generation technology for structural health monitoring. The main goal of this project was to implement a wireless sensor network for monitoring the behavior and integrity of highway bridges. At the core of the system is a low-cost, low power wireless strain sensor node whose hardware design is optimized for structural monitoring applications. The key components of the systems are the control unit, sensors, software and communication capability. The extensive information developed for each of these areas has been used to design the system. The performance and reliability of the proposed wireless monitoring system is validated on a 34 feet span composite beam in slab bridge in Black Hawk County, Iowa. The micro strain data is successfully extracted from output-only response collected by the wireless monitoring system. The energy efficiency of the system was investigated to estimate the battery lifetime of the wireless sensor nodes. This report also documents system design, the method used for data acquisition, and system validation and field testing. Recommendations on further implementation of wireless sensor networks for long term monitoring are provided.

Sistema d'adquisició de temperatures de 2 canals via R.F.

Poder mesurar i enregistrar diferents tipus de magnituds com pressió, força, temperatura etc. s’ha convertit en una necessitat per moltes aplicacions actuals. Aquestes magnituds poden tenir procedències molt diverses, tals com l’entorn, o poden ser generades per sistemes mecànics, elèctrics, etc. Per tal de poder adquirir aquestes magnituds, s’utilitzen els sistemes d’adquisició de dades. Aquests sistemes, prenen mostres analògiques del món real, i les transformen en dades digitals que poden ser manipulades per un sistema electrònic. Pràcticament qualsevol magnitud es pot mesurar utilitzant el sensor adient. Una magnitud molt utilitzada en sistemes d’adquisició de dades, és la temperatura. Els sistemes d’adquisició de temperatures estan molt generalitzats, i podem trobar-los com a sistemes, on l’objectiu és mostrar les dades adquirides, o podem trobar-los formant part de sistemes de control, aportant uns inputs necessaris per el seu correcte funcionament, garantir-ne l’estabilitat, seguretat etc. Aquest projecte, promogut per l’empresa Elausa, s’encarregarà d’adquirir, el senyal d’entrada de 2 Termoparells. Aquests mesuraran temperatures de circuits electrònics, que es trobaran dintre la càmera climàtica de Elausa, sotmesos a diferents condicions de temperatura, per tal de rebre l’homologació del circuit. El sistema haurà de poder mostrar les dades adquirides en temps real, i emmagatzemar-les en un PC que estarà ubicat en una oficina, situada a uns 30 m de distància de la sala on es farà el test. El sistema constarà d’un circuit electrònic que adquirirà, i condicionarà el senyal de sortida dels termoparells, per adaptar-lo a la tensió d’entrada d’un convertidor analògic digital, del microcontrolador integrat en aquesta placa. Seguidament aquesta informació, s’enviarà a través d’un mòdul transmissor de radiofreqüència, cap al PC on es visualitzaran les dades adquirides. Els objectius plantejats són els següents: - Dissenyar el circuit electrònic d’adquisició i condicionament del senyal. - Dissenyar, fabricar i muntar el circuit imprès de la placa d’adquisició. - Realitzar el programa de control del microcontrolador. - Realitzar el programa per presentar i desar les dades en un PC. - El sistema ha d’adquirir 2 temperatures, a través de Termoparells amb un rang d’entrada de -40ºC a +240ºC - S’ha de transmetre les dades via R.F. Els resultats del projecte han estat satisfactoris i s’han complert els objectius plantejats.

Sähköverkkojen käytöntuki- ja käytönvalvontajärjestelmien käyttöönotto

Käytöntuki- ja käytönvalvontajärjestelmät, eli käytönhallintajärjestelmät ovat osana sähköverkkoyhtiöiden arkea ja ilman niiden tuomaa apua sähkön jakelun valvonta ja hallinta olisi hyvin haastava tehtävä. Lisäksi käytönhallintajärjestelmät moitteeton toi-miminen tarjoaa hyvän alustan kehittää tulevaisuuden tärkeitä hankkeita kuten Smart Grid konseptia. Tässä työssä tarkastellaan käytönhallintajärjestelmien käyttöönottoa, kuinka se voidaan toteuttaa ja mitä asioita pitää ottaa huomioon käyttöönoton eri vaiheissa. Käytönhallintajärjestelmien käyttöönotto suoritettiin Lappeenrannan teknillisen yliopiston sähkömarkkinalaboratoriossa sijaitsevalla verkkosimulaattorille, johon asennettiin MicroSCADA SYS 600 9.1 käytönvalvontajärjestelmä sekä DMS 600 4.2 käytöntukijärjestelmä.

Computerized analysis of isometric tension studies provides important additional information about vasomotor activity

Concentration-response curves of isometric tension studies on isolated blood vessels are obtained traditionally. Although parameters such as Imax, EC50 and pA2 may be readily calculated, this method does not provide information on the temporal profile of the responses or the actual nature of the reaction curves. Computerized data acquisition systems can be used to obtain average data that represent a new source of otherwise inaccessible information, since early and late responses may be observed separately in detail

Critical infrastructure protection against cyber threats

A postgraduate seminar series with a title Critical Infrastructure Protection against Cyber Threats held at the Department of Military Technology of the National Defence University in the fall of 2013 and 2014. This book is a collection of some of talks that were presented in the seminar. The papers address origin of critical infrastructure protection, wargaming cyberwar in critical infrastructure defence, cyber-target categorization, supervisory control and data acquisition systems vulnerabilities, electric power as critical infrastructure, improving situational awareness of critical infrastructure and trust based situation awareness in high security cloud environment. This set of papers tries to give some insight to current issues of the network-centric critical infrastructure protection. The seminar has always made a publication of the papers but this has been an internal publication of the Finnish Defence Forces and has not hindered publication of the papers in international conferences. Publication of these papers in peer reviewed conferences has indeed been always the goal of the seminar, since it teaches writing conference level papers. We still hope that an internal publication in the department series is useful to the Finnish Defence Forces by offering an easy access to these papers.

The upgraded DO detector

The DO experiment enjoyed a very successful data-collection run at the Fermilab Tevatron collider between 1992 and 1996. Since then, the detector has been upgraded to take advantage of improvements to the Tevatron and to enhance its physics capabilities. We describe the new elements of the detector, including the silicon microstrip tracker, central fiber tracker, solenoidal magnet, preshower detectors, forward muon detector, and forward proton detector. The uranium/liquid -argon calorimeters and central muon detector, remaining from Run 1, are discussed briefly. We also present the associated electronics, triggering, and data acquisition systems, along with the design and implementation of software specific to DO. (c) 2006 Elsevier B.V. All rights reserved.

Laboratório remoto de qualidade da energia elétrica em ambiente virtual

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)