Implementazione di una infrastruttura basata su Kafka e Storm per il Mobile Cloud computing

Lo scopo dell'elaborato di tesi è l'analisi, progettazione e sviluppo di un prototipo di una infrastruttura cloud in grado di gestire un grande flusso di eventi generati da dispositivi mobili. Questi utilizzano informazioni come la posizione assunta e il valore dei sensori locali di cui possono essere equipaggiati al fine di realizzare il proprio funzionamento. Le informazioni così ottenute vengono trasmesse in modo da ottenere una rete di device in grado di acquisire autonomamente informazioni sull'ambiente ed auto-organizzarsi. La costruzione di tale struttura si colloca in un più ampio ambito di ricerca che punta a integrare metodi per la comunicazione ravvicinata con il cloud al fine di permettere la comunicazione tra dispositivi vicini in qualsiasi situazione che si potrebbe presentare in una situazione reale. A definire le specifiche della infrastruttura e quindi a impersonare il ruolo di committente è stato il relatore, Prof. Mirko Viroli, mentre lo sviluppo è stato portato avanti da me e dal correlatore, Ing. Pietro Brunetti. Visti gli studi precedenti riguardanti il cloud computing nell'area dei sistemi complessi distribuiti, Brunetti ha dato il maggiore contributo nella fase di analisi del problema e di progettazione mentre la parte riguardante la effettiva gestione degli eventi, le computazioni in cloud e lo storage dei dati è stata maggiormente affrontata da me. In particolare mi sono occupato dello studio e della implementazione del backend computazionale, basato sulla tecnologia Apache Storm, della componente di storage dei dati, basata su Neo4j, e della costruzione di un pannello di visualizzazione basato su AJAX e Linkurious. A questo va aggiunto lo studio su Apache Kafka, utilizzato come tecnologia per realizzare la comunicazione asincrona ad alte performance tra le componenti. Si è reso necessario costruire un simulatore al fine di condurre i test per verificare il funzionamento della infrastruttura prototipale e per saggiarne l'effettiva scalabilità, considerato il potenziale numero di dispositivi da sostenere che può andare dalle decine alle migliaia. La sfida più importante riguarda la gestione della vicinanza tra dispositivi e la possibilità di scalare la computazione su più macchine. Per questo motivo è stato necessario far uso di tecnologie per l'esecuzione delle operazioni di memorizzazione, calcolo e trasmissione dei dati in grado di essere eseguite su un cluster e garantire una accettabile fault-tolerancy. Da questo punto di vista i lavori che hanno portato alla costruzione della infrastruttura sono risultati essere un'ottima occasione per prendere familiarità con tecnologie prima sconosciute. Quasi tutte le tecnologie utilizzate fanno parte dell'ecosistema Apache e, come esposto all'interno della tesi, stanno ricevendo una grande attenzione da importanti realtà proprio in questo periodo, specialmente Apache Storm e Kafka. Il software prodotto per la costruzione della infrastruttura è completamente sviluppato in Java a cui si aggiunge la componente web di visualizzazione sviluppata in Javascript.

Online Stream Processing di Big Data su Apache Storm per Applicazioni di Instant Coupon

Big data è il termine usato per descrivere una raccolta di dati così estesa in termini di volume,velocità e varietà da richiedere tecnologie e metodi analitici specifici per l'estrazione di valori significativi. Molti sistemi sono sempre più costituiti e caratterizzati da enormi moli di dati da gestire,originati da sorgenti altamente eterogenee e con formati altamente differenziati,oltre a qualità dei dati estremamente eterogenei. Un altro requisito in questi sistemi potrebbe essere il fattore temporale: sempre più sistemi hanno bisogno di ricevere dati significativi dai Big Data il prima possibile,e sempre più spesso l’input da gestire è rappresentato da uno stream di informazioni continuo. In questo campo si inseriscono delle soluzioni specifiche per questi casi chiamati Online Stream Processing. L’obiettivo di questa tesi è di proporre un prototipo funzionante che elabori dati di Instant Coupon provenienti da diverse fonti con diversi formati e protocolli di informazioni e trasmissione e che memorizzi i dati elaborati in maniera efficiente per avere delle risposte in tempo reale. Le fonti di informazione possono essere di due tipologie: XMPP e Eddystone. Il sistema una volta ricevute le informazioni in ingresso, estrapola ed elabora codeste fino ad avere dati significativi che possono essere utilizzati da terze parti. Lo storage di questi dati è fatto su Apache Cassandra. Il problema più grosso che si è dovuto risolvere riguarda il fatto che Apache Storm non prevede il ribilanciamento delle risorse in maniera automatica, in questo caso specifico però la distribuzione dei clienti durante la giornata è molto varia e ricca di picchi. Il sistema interno di ribilanciamento sfrutta tecnologie innovative come le metriche e sulla base del throughput e della latenza esecutiva decide se aumentare/diminuire il numero di risorse o semplicemente non fare niente se le statistiche sono all’interno dei valori di soglia voluti.

Geophysical Analysis of Seasonal Montandon Gravel Ridge Water Table Fluctuation and Moisture Gradient Variation due to Storm Events

The purpose of this research project is to continue exploring the Montandon Long-Term Hydrologic Research Site(LTHR) by using multiple geophysical methods to obtain more accurate and precise information regarding subsurface hydrologic properties of a local gravel ridge,which are important to both the health of surrounding ecosystems and local agriculture. Through using non-invasive geophysical methods such as seismic refraction, Direct Current resistivity and ground penetrating radar (GPR) instead of invasive methods such as boreholedrilling which displace sediment and may alter water flow, data collection is less likely to bias the data itself. In addition to imaging the gravel ridge subsurface, another important researchpurpose is to observe how both water table elevation and the moisture gradient (moisture content of the unsaturated zone) change over a seasonal time period and directly after storm events. The combination of three types of data collection allows the strengths of each method combine together and provide a relatively strongly supported conclusions compared to previous research. Precipitation and geophysical data suggest that an overall increase in precipitation during the summer months causes a sharp decrease in subsurface resistivity within the unsaturated zone. GPR velocity data indicate significant immediate increase in moisture content within the shallow vadose zone (< 1m), suggesting that rain water was infiltrating into the shallow subsurface. Furthermore, the combination of resistivity and GPR results suggest that the decreased resistivity within the shallow layers is due to increased ion content within groundwater. This is unexpected as rainwater is assumed to have a DC resistivity value of 3.33*105 ohm-m. These results may suggest that ions within the sediment must beincorporated into the infiltrating water.

Future storm surge impacts on insurable losses for the North Sea region

The influence of climate change on storm surges including increased mean sea level change and the associated insurable losses are assessed for the North Sea basin. In doing so, the newly developed approach couples a dynamical storm surge model with a loss model. The key element of the approach is the generation of a probabilistic storm surge event set. Together with parametrizations of the inland propagation and the coastal protection failure probability this enables the estimation of annual expected losses. The sensitivity to the parametrizations is rather weak except when the assumption of high level of increased mean sea level change is made. Applying this approach to future scenarios shows a substantial increase of insurable losses with respect to the present day. Superimposing different mean sea level changes shows a nonlinear behavior at the country level, as the future storm surge changes are higher for Germany and Denmark. Thus, the study exhibits the necessity to assess the socio-economic impacts of coastal floods by combining the expected sea level rise with storm surge projections.

The Clearing Storm: A Re-Assessment of the Climate for Collaboration between Academic and Student Affairs

There is increasing recognition among those in higher education that it is no longer adequate to train students in a specific field or industry. Instead, the push is more towards producing well-rounded students. In order to do so, all of a university’s resources must come together and the climate on campus must be one that supportscollaboration. This report is a re-examination of the climate for collaboration on the campus of a private liberal arts university in the Mid-Atlantic region of the United States. It is a follow up to a similar investigation conducted on the same campus by Victor Arcelus(2008) five years earlier. In the interim, the university had re-configured its organizational structure, combining separate academic and student affairs divisions into a single unit overseen by the Provost. Additionally, the university had experienced turnover in several key leadership positions, including those of the President and the chief academic and student affairs officers. The purpose of this investigation, therefore, was to gauge the immediate impact of these changes on conditions for collaboration, which when present, advance student learning and development. Through interviews with six men and women, information was collected on the perceived climate for collaboration between academic and student affairs personnel.Analysis of the interview transcripts revealed that, depending on the position of the interviewee within the university, conditions on campus were seen as either improved or largely unchanged as a result of the transition in leadership and the structural merger of the two divisions.

Reanalysis of the Hamburg Storm Surge of 1962

In February 1962, Hamburg experienced its most catastrophic storm surge event of the 20th century. This paper analyses the event using the Twentieth Century Reanalysis (20CR) dataset. Responsible for the major flood was a strong low pressure system centred over Scandinavia that was associated with strong north-westerly winds towards the German North Sea coast – the ideal storm surge situation for the Elbe estuary. A comparison of the 20CR dataset with observational data proves the applicability of the reanalysis data for this extreme event.

The Storm Surge Event of the Netherlands in 1953

A disastrous storm surge hit the coast of the Netherlands on 31 January and 1 February 1953. We examine the meteorological situation during this event using the Twentieth Century Reanalysis (20CR) data set. We find a strong pressure gradient between Ireland and northern Germany accompanied by strong north-westerly winds over the North Sea. Storm driven sea level rise combined with spring tide contributed to this extreme event. The state of the atmosphere in 20CR during this extreme event is in good agreement with historical observational data

The 1872 Baltic Sea storm surge

On 13 November 1872, the Baltic Sea coast from Denmark to Pomerania was devastated by an extreme storm surge caused by high winds. This is still the strongest surge on record, and understanding its development can contribute to improved risk assessment and protection. In this paper we trace this event in sea-level pressure and wind data from the “Twentieth Century Reanalysis” (20CR) and compare the results with other observation-based data sources. The analysis shows that, in the ensemble mean of 20CR, the general development is qualitatively well depicted, but with much reduced strength compared to other data sets. The same is true when selecting the ensemble member with maximum wind speeds.

Seismic stratigraphy of the Blue Hole (Lighthouse Reef, Belize), a late Holocene climate and storm archive

Five seismic units may be identified in the similar to 8 m thick Holocene sediment package at the bottom of the Blue Hole, a 120 m deep sinkhole located in the atoll lagoon of Lighthouse Reef, Belize. These units may be correlated with the succession of an existing 5.85-m-long sediment core that reaches back to 1385 kyrs BP. The identification of seismic units is based on the fact that uniform, fine-grained background sediments show weak reflections while alternating background and coarser-grained event (storm) beds exhibit strong reflections in the seismic profiles. The main source of sediments is the marginal atoll reef and adjacent lagoon area to the east and north. Northeasterly winds and storms transport sediment into the Blue Hole, as seen in the eastward increase in sediment thickness, i.e., the eastward shallowing of the Blue Hole. Previous assumptions of much thicker Holocene sediment packages in the Blue Hole could not be confirmed. So far, close to 6-m-long cores were retrieved from the Blue Hole but the base of the sedimentary succession remains to be recovered. The nature of the basal sediments is unknown but mid-Holocene and possibly older, Pleistocene sinkhole deposits can be expected. The number of event beds identified in the Blue Hole (n = 37) during a 1.385 kyr-long period and the number of cyclones listed in historical databases suggest that only strong hurricanes (categories 4 and 5) left event beds in the Blue Hole sedimentary succession. Storm beds are numerous during 13-0.9 kyrs BP and 0.8-0.5 kyrs BP.

The Use of Ships' Protests for Reconstruction of Synoptic-Scale Weather and Tropical Storm Identification in the Late Eighteenth Century

Ships’ protests have been used for centuries as legal documents to record and detail damages and indemnify Captains from fault. We use them in this article, along with data extracted through forensic synoptic analysis (McNally, 1994, 2004) to identify a tropical or subtropical system in the North Atlantic Ocean in 1785. They are shown to be viable sources of meteorological information. By comparing a damaging storm in New England in 1996, which included an offshore tropical system, with one reconstructed in 1785, we demonstrate that the tropical system identified in a ship’s protest played a significant role in the 1785 storm. With both forensic reconstruction and anecdotal evidence, we are able to assess that these storms are remarkably identical. The recurrence rate calculated in previous studies of the 1996 storm is 400–500 years. We suggest that reconstruction of additional years in the 1700s would provide the basis for a reanalysis of recurrence rates, with implications for future insurance and reinsurance rates. The application of the methodology to this new data source can also be used for extension of the hurricane database in the North Atlantic basin, and elsewhere, much further back into history than is currently available.