Implementazione di una infrastruttura basata su Kafka e Storm per il Mobile Cloud computing

Lo scopo dell'elaborato di tesi è l'analisi, progettazione e sviluppo di un prototipo di una infrastruttura cloud in grado di gestire un grande flusso di eventi generati da dispositivi mobili. Questi utilizzano informazioni come la posizione assunta e il valore dei sensori locali di cui possono essere equipaggiati al fine di realizzare il proprio funzionamento. Le informazioni così ottenute vengono trasmesse in modo da ottenere una rete di device in grado di acquisire autonomamente informazioni sull'ambiente ed auto-organizzarsi. La costruzione di tale struttura si colloca in un più ampio ambito di ricerca che punta a integrare metodi per la comunicazione ravvicinata con il cloud al fine di permettere la comunicazione tra dispositivi vicini in qualsiasi situazione che si potrebbe presentare in una situazione reale. A definire le specifiche della infrastruttura e quindi a impersonare il ruolo di committente è stato il relatore, Prof. Mirko Viroli, mentre lo sviluppo è stato portato avanti da me e dal correlatore, Ing. Pietro Brunetti. Visti gli studi precedenti riguardanti il cloud computing nell'area dei sistemi complessi distribuiti, Brunetti ha dato il maggiore contributo nella fase di analisi del problema e di progettazione mentre la parte riguardante la effettiva gestione degli eventi, le computazioni in cloud e lo storage dei dati è stata maggiormente affrontata da me. In particolare mi sono occupato dello studio e della implementazione del backend computazionale, basato sulla tecnologia Apache Storm, della componente di storage dei dati, basata su Neo4j, e della costruzione di un pannello di visualizzazione basato su AJAX e Linkurious. A questo va aggiunto lo studio su Apache Kafka, utilizzato come tecnologia per realizzare la comunicazione asincrona ad alte performance tra le componenti. Si è reso necessario costruire un simulatore al fine di condurre i test per verificare il funzionamento della infrastruttura prototipale e per saggiarne l'effettiva scalabilità, considerato il potenziale numero di dispositivi da sostenere che può andare dalle decine alle migliaia. La sfida più importante riguarda la gestione della vicinanza tra dispositivi e la possibilità di scalare la computazione su più macchine. Per questo motivo è stato necessario far uso di tecnologie per l'esecuzione delle operazioni di memorizzazione, calcolo e trasmissione dei dati in grado di essere eseguite su un cluster e garantire una accettabile fault-tolerancy. Da questo punto di vista i lavori che hanno portato alla costruzione della infrastruttura sono risultati essere un'ottima occasione per prendere familiarità con tecnologie prima sconosciute. Quasi tutte le tecnologie utilizzate fanno parte dell'ecosistema Apache e, come esposto all'interno della tesi, stanno ricevendo una grande attenzione da importanti realtà proprio in questo periodo, specialmente Apache Storm e Kafka. Il software prodotto per la costruzione della infrastruttura è completamente sviluppato in Java a cui si aggiunge la componente web di visualizzazione sviluppata in Javascript.

Virtualizzazione di funzioni di rete su piattaforme per cloud computing

Questo documento affronta le novità ed i vantaggi introdotti nel mondo delle reti di telecomunicazioni dai paradigmi di Software Defined Networking e Network Functions Virtualization, affrontandone prima gli aspetti teorici, per poi applicarne i concetti nella pratica, tramite casi di studio gradualmente più complessi. Tali innovazioni rappresentano un'evoluzione dell'architettura delle reti predisposte alla presenza di più utenti connessi alle risorse da esse offerte, trovando quindi applicazione soprattutto nell'emergente ambiente di Cloud Computing e realizzando in questo modo reti altamente dinamiche e programmabili, tramite la virtualizzazione dei servizi di rete richiesti per l'ottimizzazione dell'utilizzo di risorse. Motivo di tale lavoro è la ricerca di soluzioni ai problemi di staticità e dipendenza, dai fornitori dei nodi intermedi, della rete Internet, i maggiori ostacoli per lo sviluppo delle architetture Cloud. L'obiettivo principale dello studio presentato in questo documento è quello di valutare l'effettiva convenienza dell'applicazione di tali paradigmi nella creazione di reti, controllando in questo modo che le promesse di aumento di autonomia e dinamismo vengano rispettate. Tale scopo viene perseguito attraverso l'implementazione di entrambi i paradigmi SDN e NFV nelle sperimentazioni effettuate sulle reti di livello L2 ed L3 del modello OSI. Il risultato ottenuto da tali casi di studio è infine un'interessante conferma dei vantaggi presentati durante lo studio teorico delle innovazioni in analisi, rendendo esse una possibile soluzione futura alle problematiche attuali delle reti.

Elasticity in Cloud Computing: Tecnologia e applicazione aziendale presso Onit

Il Cloud Computing permette di utilizzare al meglio le risorse distribuite allo scopo di risolvere problemi di computazione su larga scala, e viene distribuito dai provider all'utente finale sotto forma di servizio. Presentati i diversi modelli di distribuzione dei servizi Cloud, si discutono le varie tipologie di servizi offerti. Efficaci meccanismi di elasticità e scalabilità hanno permesso al Cloud Computing di superare lo scoglio iniziale di utilizzo medio dei server al 10%. L'elasticità (rapid elasticity) è l’abilità di acquisire e rilasciare le risorse di un'infrastruttura Cloud su richiesta, l’abilità di un'applicazione di cambiare le sue dimensione durante il suo tempo di esecuzione; la scalabilità è un prerequisito per ottenere una buona elasticità per il sistema, ed è l'abilità che ha un layer di sostenere carichi di lavoro variabili continuando ad adempiere agli obblighi imposti dallo SLA allocando o disallocando risorse computazionali. Le diverse modalità di scaling e il loro utilizzo determinano la scalabilità e di conseguenza l'elasticità del sistema, e sfruttano la virtualizzazione per poter funzionare. Ciò ha portato notevoli benefici perchè aumenta l'utilizzo dei server, migliora l'efficienza del sistema, e dona flessibilità in caso di errori massimizzando il tempo di funzionamento. Sono stati introdotti due esempi di sistemi elastici basati ovviamente sulla virtualizzazione come Amazon Web Services e Microsoft Azure, che dominano il mercato del Cloud Computing e fanno uso dei più efficenti meccanismi d'elasticità. Il cuore di questo elaborato è l'analisi dell'ampliamento dell'adozione del Cloud Computing in azienda Onit Group srl. L'obiettivo è trattare i punti fondamentali per il Cloud Computing, analizzarli e immagazzinare tutte queste conoscenze per analizzare lo stato attuale del Cloud nell'azienda focalizzando l'attenzione sui vantaggi e sugli svantaggi che un sostanziale ampliamento dell'adozione ai sistemi Cloud poteva apportare.

Cloud Computing in the EU Policy Sphere Interoperability, Vertical Integration and the Internal Market

Cloud computing is a new development that is based on the premise that data and applications are stored centrally and can be accessed through the Internet. Thisarticle sets up a broad analysis of how the emergence of clouds relates to European competition law, network regulation and electronic commerce regulation, which we relate to challenges for the further development of cloud services in Europe: interoperability and data portability between clouds; issues relating to vertical integration between clouds and Internet Service Providers; and potential problems for clouds to operate on the European Internal Market. We find that these issues are not adequately addressed across the legal frameworks that we analyse, and argue for further research into how to better facilitate innovative convergent services such as cloud computing through European policy – especially in light of the ambitious digital agenda that the European Commission has set out.

Global Standards: Recent Developments betweend the Poles of Privacy and Cloud Computing

The development of the Internet has made it possible to transfer data ‘around the globe at the click of a mouse’. Especially fresh business models such as cloud computing, the newest driver to illustrate the speed and breadth of the online environment, allow this data to be processed across national borders on a routine basis. A number of factors cause the Internet to blur the lines between public and private space: Firstly, globalization and the outsourcing of economic actors entrain an ever-growing exchange of personal data. Secondly, the security pressure in the name of the legitimate fight against terrorism opens the access to a significant amount of data for an increasing number of public authorities.And finally,the tools of the digital society accompany everyone at each stage of life by leaving permanent individual and borderless traces in both space and time. Therefore, calls from both the public and private sectors for an international legal framework for privacy and data protection have become louder. Companies such as Google and Facebook have also come under continuous pressure from governments and citizens to reform the use of data. Thus, Google was not alone in calling for the creation of ‘global privacystandards’. Efforts are underway to review established privacy foundation documents. There are similar efforts to look at standards in global approaches to privacy and data protection. The last remarkable steps were the Montreux Declaration, in which the privacycommissioners appealed to the United Nations ‘to prepare a binding legal instrument which clearly sets out in detail the rights to data protection and privacy as enforceable human rights’. This appeal was repeated in 2008 at the 30thinternational conference held in Strasbourg, at the 31stconference 2009 in Madrid and in 2010 at the 32ndconference in Jerusalem. In a globalized world, free data flow has become an everyday need. Thus, the aim of global harmonization should be that it doesn’t make any difference for data users or data subjects whether data processing takes place in one or in several countries. Concern has been expressed that data users might seek to avoid privacy controls by moving their operations to countries which have lower standards in their privacy laws or no such laws at all. To control that risk, some countries have implemented special controls into their domestic law. Again, such controls may interfere with the need for free international data flow. A formula has to be found to make sure that privacy at the international level does not prejudice this principle.

The new cloud computing paradigm: the way to IT seen as a utility

In the present competitive environment, companies are wondering how to reduce their IT costs while increasing their efficiency and agility to react when changes in the business processes are required. Cloud Computing is the latest paradigm to optimize the use of IT resources considering ?everything as a service? and receiving these services from the Cloud (Internet) instead of owning and managing hardware and software assets. The benefits from the model are clear. However, there are also concerns and issues to be solved before Cloud Computing spreads across the different industries. This model will allow a pay-per-use model for the IT services and many benefits like cost savings, agility to react when business demands changes and simplicity because there will not be any infrastructure to operate and administrate. It will be comparable to the well known utilities like electricity, water or gas companies. However, this paper underlines several risk factors of the model. Leading technology companies should research on solutions to minimize the risks described in this article. Keywords - Cloud Computing, Utility Computing, Elastic Computing, Enterprise Agility

Accessing advanced computational resources in Africa through Cloud Computing

Low resources in many African locations do not allow many African scientists and physicians to access the latest advances in technology. This deficiency hinders the daily life of African professionals that often cannot afford, for instance, the cost of internet fees or software licenses. The AFRICA BUILD project, funded by the European Commission and formed by four European and four African institutions, intends to provide advanced computational tools to African institutions in order to solve current technological limitations. In the context of AFRICA BUILD we have carried out, a series of experiments to test the feasibility of using Cloud Computing technologies in two different locations in Africa: Egypt and Burundi. The project aims to create a virtual platform to provide access to a wide range of biomedical informatics and learning resources to professionals and researchers in Africa.

The University in the Cloud Computing.

With the advancement of Information and Communication Technology ICT which favors increasingly fast, easy, and accessible communication for all and which can reach large groups of people, there have been changes, in recent years in our society that have modified the way we interact, communicate and transmit information. Access to this, it is possible, not only through computers situated in a fixed location, but new mobile devices make it available, wherever the user happens to be located. Now, information "travels" with the user. These forms of communication, transmission and access to information, have also affected the way to conceive and manage business. To these new forms of business that the Internet has brought, is now added the concept of companies in the Cloud Computing ClC. The ClC technology is based on the supply and consumption of services on demand and pay per use, and it gives a 180 degree turn to the business management concept. Small and large businesses may use the latest developments in ICT, to manage their organizations without the need for expensive investments in them. This will enable enterprises to focus more specifically within the scope of their business, leaving the ICT control to the experts. We believe that education can also and should benefit from these new philosophies. ?Due to the global economic crisis in general and each country in particular, economic cutbacks have come to most universities. These are seen in the need to raise tuition rates, which makes increasingly fewer students have the opportunity to pursue higher education?. In this paper we propose using ClC technologies in universities and we make a dissertation on the advantages that it can provide to both: universities and students. For the universities, we expose two focuses, one: ?to reorganize university ICT structures with the ClC philosophy? and the other one, ?to extend the offer of the university education with education on demand?. Regarding the former we propose to use public or private Clouds, to reuse resources across the education community, to save costs on infrastructure investment, in upgrades and in maintenance of ICT, and paying only for what you use and with the ability to scale according to needs. Regarding the latter, we propose an educational model in the ClC, to increase the current university offerings, using educational units in the form of low-cost services and where students pay only for the units consumed on demand. For the students, they could study at any university in the world (virtually), from anywhere, without travel costs: money and time, and what is most important paying only for what they consume. We think that this proposal of education on demand may represent a great change in the current educational model, because strict registration deadlines disappear, and also the problem of economically disadvantaged students, who will not have to raise large amounts of money for an annual tuition. Also it will decrease the problem of loss of the money invested in an enrollment when the student dropout. In summary we think that this proposal is interesting for both, universities and students, we aim for "Higher education from anywhere, with access from any mobile device, at any time, without requiring large investments for students, and with reuse and optimization of resources by universities. Cost by consumption and consumption by service?. We argue for a Universal University "wisdom and knowledge accessible to all?

An Autonomic Framework for Mobile Cloud Computing

This thesis presents the formal definition of a novel Mobile Cloud Computing (MCC) extension of the Networked Autonomic Machine (NAM) framework, a general-purpose conceptual tool which describes large-scale distributed autonomic systems. The introduction of autonomic policies in the MCC paradigm has proved to be an effective technique to increase the robustness and flexibility of MCC systems. In particular, autonomic policies based on continuous resource and connectivity monitoring help automate context-aware decisions for computation offloading. We have also provided NAM with a formalization in terms of a transformational operational semantics in order to fill the gap between its existing Java implementation NAM4J and its conceptual definition. Moreover, we have extended NAM4J by adding several components with the purpose of managing large scale autonomic distributed environments. In particular, the middleware allows for the implementation of peer-to-peer (P2P) networks of NAM nodes. Moreover, NAM mobility actions have been implemented to enable the migration of code, execution state and data. Within NAM4J, we have designed and developed a component, denoted as context bus, which is particularly useful in collaborative applications in that, if replicated on each peer, it instantiates a virtual shared channel allowing nodes to notify and get notified about context events. Regarding the autonomic policies management, we have provided NAM4J with a rule engine, whose purpose is to allow a system to autonomously determine when offloading is convenient. We have also provided NAM4J with trust and reputation management mechanisms to make the middleware suitable for applications in which such aspects are of great interest. To this purpose, we have designed and implemented a distributed framework, denoted as DARTSense, where no central server is required, as reputation values are stored and updated by participants in a subjective fashion. We have also investigated the literature regarding MCC systems. The analysis pointed out that all MCC models focus on mobile devices, and consider the Cloud as a system with unlimited resources. To contribute in filling this gap, we defined a modeling and simulation framework for the design and analysis of MCC systems, encompassing both their sides. We have also implemented a modular and reusable simulator of the model. We have applied the NAM principles to two different application scenarios. First, we have defined a hybrid P2P/cloud approach where components and protocols are autonomically configured according to specific target goals, such as cost-effectiveness, reliability and availability. Merging P2P and cloud paradigms brings together the advantages of both: high availability, provided by the Cloud presence, and low cost, by exploiting inexpensive peers resources. As an example, we have shown how the proposed approach can be used to design NAM-based collaborative storage systems based on an autonomic policy to decide how to distribute data chunks among peers and Cloud, according to cost minimization and data availability goals. As a second application, we have defined an autonomic architecture for decentralized urban participatory sensing (UPS) which bridges sensor networks and mobile systems to improve effectiveness and efficiency. The developed application allows users to retrieve and publish different types of sensed information by using the features provided by NAM4J's context bus. Trust and reputation is managed through the application of DARTSense mechanisms. Also, the application includes an autonomic policy that detects areas characterized by few contributors, and tries to recruit new providers by migrating code necessary to sensing, through NAM mobility actions.

A számítási felhő az Európai Unió egén (Cloud computing on the sky of European Union)

Napjaink informatikai világának talán legkeresettebb hívó szava a cloud computing, vagy magyar fordításban, a számítási felhő. A fordítás forrása az EU-s (Digitális Menetrend magyar változata, 2010) A számítási felhő üzleti modelljének részletes leírását adja (Bőgel, 2009). Bőgel György ismerteti az új, közműszerű informatikai szolgáltatás kialakulását és gazdasági előnyeit, nagy jövőt jósolva a számítási felhőnek az üzleti modellek versenyében. A szerző – a számítási felhő üzleti előnyei mellett – nagyobb hangsúlyt fektet dolgozatában a gyors elterjedést gátló tényezőkre, és arra, hogy mit jelentenek az előnyök és a hátrányok egy üzleti, informatikai vagy megfelelőségi vezető számára. Nem csökkentve a cloud modell gazdasági jelentőségét, fontosnak tartja, hogy a problémákról és a kockázatokról is szóljon. Kiemeli, hogy a kockázatokban – különösen a biztonsági és adatvédelmi kockázatokban – lényeges különbségek vannak az Európai Gazdasági Térség és a világ többi része, pl. az Amerikai Egyesült Államok között. A cikkben rámutat ezekre a különbségekre, és az olvasó magyarázatot kap arra is, hogy miért várható a számítási felhő lassabb terjedése Európában, mint a világ más részein. Bemutatja az EU erőfeszítéseit is a számítási felhő európai terjedésének elősegítésére, tekintettel a modell versenyképességet növelő hatására. / === / One of the most popular concept of the recent web searches is cloud computing. Several authors present detailed description of the new service model and it's business benefits and cite the optimistic prognoses of the cloud experts regarding the competition of information system service models. The author analyses the operational benefits of the cloud application and give a detailed description of the inhibitors of the fast expansion of the service modell. He also analyses the pros and cons of the cloud for a business manager, an information and a compliance officer. When understanding the advantages of the cloud, it is equally important to review the problems and risks associated with the model. The paper gives a list of the expected cloud-specific risks. It also explains the differences in security and data protection approach between the European Economic Area and the rest of the world, including the USA. The explains why slower expansion of the cloud modell is expected in Europe than in the rest of the world. The efforts of the EU Committee in helping to spread the cloud model is also presented, as the EU's officers consider the model as an important element of competitiveness.

A Framework for Security Transparency in Cloud Computing

Individuals and corporate users are persistently considering cloud adoption due to its significant benefits compared to traditional computing environments. The data and applications in the cloud are stored in an environment that is separated, managed and maintained externally to the organisation. Therefore, it is essential for cloud providers to demonstrate and implement adequate security practices to protect the data and processes put under their stewardship. Security transparency in the cloud is likely to become the core theme that underpins the systematic disclosure of security designs and practices that enhance customer confidence in using cloud service and deployment models. In this paper, we present a framework that enables a detailed analysis of security transparency for cloud based systems. In particular, we consider security transparency from three different levels of abstraction, i.e., conceptual, organisation and technical levels, and identify the relevant concepts within these levels. This allows us to provide an elaboration of the essential concepts at the core of transparency and analyse the means for implementing them from a technical perspective. Finally, an example from a real world migration context is given to provide a solid discussion on the applicability of the proposed framework.

FlorNExT®, a cloud computing application to estimate growth and yield of maritime pine (Pinus pinaster Ait.) stands in Northeastern Portugal

Aim of the study: To introduce and describe FlorNExT®, a free cloud computing application to estimate growth and yield of maritime pine (Pinus pinaster Ait.) even-aged stands in the Northeast of Portugal (NE Portugal). Area of study: NE Portugal. Material and methods: FlorNExT® implements a dynamic growth and yield modelling framework which integrates transition functions for dominant height (site index curves) and basal area, as well as output functions for tree and stand volume, biomass, and carbon content. Main results: FlorNExT® is freely available from any device with an Internet connection at: http://flornext.esa.ipb.pt/. Research highlights: This application has been designed to make it possible for any stakeholder to easily estimate standing volume, biomass, and carbon content in maritime pine stands from stand data, as well as to estimate growth and yield based on four stand variables: age, density, dominant height, and basal area. FlorNExT® allows planning thinning treatments. FlorNExT® is a fundamental tool to support forest mobilization at local and regional scales in NE Portugal. Material and methods: FlorNExT® implements a dynamic growth and yield modelling framework which integrates transition functions for dominant height (site index curves) and basal area, as well as output functions for tree and stand volume, biomass, and carbon content. Main results: FlorNExT® is freely available from any device with an Internet connection at: http://flornext.esa.ipb.pt/. Research highlights: This application has been designed to make it possible for any stakeholder to easily estimate standing volume, biomass, and carbon content in maritime pine stands from stand data, as well as to estimate growth and yield based on four stand variables: age, density, dominant height, and basal area. FlorNExT® allows planning thinning treatments. FlorNExT® is a fundamental tool to support forest mobilization at local and regional scales in NE Portugal.

Cloud Computing to Automate Information Units

This article will address the main technical aspects that facilitate the use and growth of computer technology in the cloud, which go hand in hand with the emergence of more and better services on the Internet and technological development of the broadband. Finally, we know what is the impact that the cloud computing technologies in the automation of information units.

Estudo de adoção de tecnologias de cloud computing no contexto das PMES portuguesas

A sociedade atual é marcada por uma crescente competição entre as organizações, as quais para garantirem a sua sobrevivência têm que evoluir e utilizar a tecnologia de informação mais adequada. Num período em que as PMES portuguesas necessitam de reduzir custos e concentrarem-se mais no seu processo de negócio surge um novo paradigma tecnológico, a Computação na Nuvem, esta ainda é um paradigma bastante recente, onde os recursos de TI que apresentam características de elasticidade e dinâmicas são oferecidos ao cliente como um serviço, no qual este paga apenas o que consome (pagamento por utilização). A Cloud Computing com o seu novo paradigma permite às organizações, principalmente às PMES reduzirem custos, concentrarem-se no seu processo de negócio sem se preocuparem com a parte informática (cópias de segurança, assistência e perda de dados) e obter um retorno mais rápido do seu investimento. Este trabalho consiste no Estudo da Adoção Individual da Cloud Computing no contexto das PMEs Portuguesas, o qual apresenta como principais objetivos obter uma melhor compreensão do conceito Cloud Computing, entender a sua importância para os indivíduos das PMES, identificar os seus fatores de sucesso e insucesso e identificar as limitações sentidas de uma forma generalizada e de uma forma específica para as PMES Portuguesas. Para cumprir os objetivos mencionados anteriormente, efetuou-se uma extensa revisão de literatura, propôs-se um modelo conceptual de investigação com adaptações à Teoria Unificada de Aceitação e Uso de Tecnologia (UTAUT) e utilizou-se como instrumento de recolha de dados um questionário que foi enviado às PMES Portuguesas.

The adoption of cloud computing by SMEs: identifying and coping with external factors.

Comunicação apresentada na CAPSI 2011 - 11ª Conferência da Associação Portuguesa de Sistemas de Informação – A Gestão de Informação na era da Cloud Computing, Lisboa, ISEG/IUL-ISCTE/, 19 a 21 de Outubro de 2011.