Container-based Cloud Virtual Machine Benchmarking

With the availability of a wide range of cloud Virtual Machines (VMs) it is difficult to determine which VMs can maximise the performance of an application. Benchmarking is commonly used to this end for capturing the performance of VMs. Most cloud benchmarking techniques are typically heavyweight - time consuming processes which have to benchmark the entire VM in order to obtain accurate benchmark data. Such benchmarks cannot be used in real-time on the cloud and incur extra costs even before an application is deployed.

In this paper, we present lightweight cloud benchmarking techniques that execute quickly and can be used in near real-time on the cloud. The exploration of lightweight benchmarking techniques are facilitated by the development of DocLite - Docker Container-based Lightweight Benchmarking. DocLite is built on the Docker container technology which allows a user-defined portion (such as memory size and the number of CPU cores) of the VM to be benchmarked. DocLite operates in two modes, in the first mode, containers are used to benchmark a small portion of the VM to generate performance ranks. In the second mode, historic benchmark data is used along with the first mode as a hybrid to generate VM ranks. The generated ranks are evaluated against three scientific high-performance computing applications. The proposed techniques are up to 91 times faster than a heavyweight technique which benchmarks the entire VM. It is observed that the first mode can generate ranks with over 90% and 86% accuracy for sequential and parallel execution of an application. The hybrid mode improves the correlation slightly but the first mode is sufficient for benchmarking cloud VMs.

Executing bag of distributed tasks on the cloud: Investigating the trade-offs between performance and cost

Bag of Distributed Tasks (BoDT) can benefit from decentralised execution on the Cloud. However, there is a trade-off between the performance that can be achieved by employing a large number of Cloud VMs for the tasks and the monetary constraints that are often placed by a user. The research reported in this paper is motivated towards investigating this trade-off so that an optimal plan for deploying BoDT applications on the cloud can be generated. A heuristic algorithm, which considers the user's preference of performance and cost is proposed and implemented. The feasibility of the algorithm is demonstrated by generating execution plans for a sample application. The key result is that the algorithm generates optimal execution plans for the application over 91% of the time.

Optimal deployment of geographically distributed workflow engines on the cloud

When orchestrating Web service workflows, the geographical placement of the orchestration engine (s) can greatly affect workflow performance. Data may have to be transferred across long geographical distances, which in turn increases execution time and degrades the overall performance of a workflow. In this paper, we present a framework that, given a DAG-based workflow specification, computes the optimal Amazon EC2 cloud regions to deploy the orchestration engines and execute a workflow. The framework incorporates a constraint model that solves the workflow deployment problem, which is generated using an automated constraint modelling system. The feasibility of the framework is evaluated by executing different sample workflows representative of scientific workloads. The experimental results indicate that the framework reduces the workflow execution time and provides a speed up of 1.3x-2.5x over centralised approaches.

LS-ADT: Lightweight and Scalable Anomaly Detection for Cloud Datacentres

Cloud data centres are implemented as large-scale clusters with demanding requirements for service performance, availability and cost of operation. As a result of scale and complexity, data centres typically exhibit large numbers of system anomalies resulting from operator error, resource over/under provisioning, hardware or software failures and security issus anomalies are inherently difficult to identify and resolve promptly via human inspection. Therefore, it is vital in a cloud system to have automatic system monitoring that detects potential anomalies and identifies their source. In this paper we present a lightweight anomaly detection tool for Cloud data centres which combines extended log analysis and rigorous correlation of system metrics, implemented by an efficient correlation algorithm which does not require training or complex infrastructure set up. The LADT algorithm is based on the premise that there is a strong correlation between node level and VM level metrics in a cloud system. This correlation will drop significantly in the event of any performance anomaly at the node-level and a continuous drop in the correlation can indicate the presence of a true anomaly in the node. The log analysis of LADT assists in determining whether the correlation drop could be caused by naturally occurring cloud management activity such as VM migration, creation, suspension, termination or resizing. In this way, any potential anomaly alerts are reasoned about to prevent false positives that could be caused by the cloud operator’s activity. We demonstrate LADT with log analysis in a Cloud environment to show how the log analysis is combined with the correlation of systems metrics to achieve accurate anomaly detection.

DocLite: A Docker-Based Lightweight Cloud Benchmarking Tool

Existing benchmarking methods are time consuming processes as they typically benchmark the entire Virtual Machine (VM) in order to generate accurate performance data, making them less suitable for real-time analytics. The research in this paper is aimed to surmount the above challenge by presenting DocLite - Docker Container-based Lightweight benchmarking tool. DocLite explores lightweight cloud benchmarking methods for rapidly executing benchmarks in near real-time. DocLite is built on the Docker container technology, which allows a user-defined memory size and number of CPU cores of the VM to be benchmarked. The tool incorporates two benchmarking methods - the first referred to as the native method employs containers to benchmark a small portion of the VM and generate performance ranks, and the second uses historic benchmark data along with the native method as a hybrid to generate VM ranks. The proposed methods are evaluated on three use-cases and are observed to be up to 91 times faster than benchmarking the entire VM. In both methods, small containers provide the same quality of rankings as a large container. The native method generates ranks with over 90% and 86% accuracy for sequential and parallel execution of an application compared against benchmarking the whole VM. The hybrid method did not improve the quality of the rankings significantly.

Minimising the Execution of Unknown Bag-of-Task Jobs with Deadlines on the Cloud

Scheduling jobs with deadlines, each of which defines the latest time that a job must be completed, can be challenging on the cloud due to incurred costs and unpredictable performance. This problem is further complicated when there is not enough information to effectively schedule a job such that its deadline is satisfied, and the cost is minimised. In this paper, we present an approach to schedule jobs, whose performance are unknown before execution, with deadlines on the cloud. By performing a sampling phase to collect the necessary information about those jobs, our approach delivers the scheduling decision within 10% cost and 16% violation rate when compared to the ideal setting, which has complete knowledge about each of the jobs from the beginning. It is noted that our proposed algorithm outperforms existing approaches, which use a fixed amount of resources by reducing the violation cost by at least two times.

Cumulus momentum fluxes in cloud-resolving model simulations of toga-coare

O objectivo deste trabalho científico é o estudo do transporte vertical de momento linear horizontal (CMT) realizado por sistemas de nuvens de convecção profunda sobre o oceano tropical. Para realizar este estudo, foram utilizadas simulações tridimensionais produzidas por um modelo explícito de nuvens (CRM) para os quatro meses de duração da campanha observacional TOGA COARE que ocorreu sobre as águas quentes do Pacífico ocidental. O estudo foca essencialmente as características estatísticas e à escala da nuvem do CMT durante um episódio de fortes ventos de oeste e durante um período de tempo maior que incluí este evento de convecção profunda. As distribuições verticais e altitude-temporais de campos atmosféricos relacionados com o CMT são avaliadas relativamente aos campos observacionais disponíveis, mostrando um bom acordo com os resultados de estudos anteriores, confirmando assim a boa qualidade das primeiras e fornecendo a confiança necessária para continuar a investigação. A sensibilidade do CMT em relação do domínio espacial do model é analisada, utilizando dois tipos de simulações tridimensionais produzidas por domínios horizontais de diferente dimensão, sugerindo que o CMT não depende da dimensão do domínio espacial horizontal escolhido para simular esta variável. A capacidade da parameterização do comprimento de mistura simular o CMT é testada, destacando as regiões troposféricas onde os fluxos de momento linear horizontal são no sentido do gradiente ou contra o gradiente. Os fluxos no sentido do gradiente apresentam-se relacionados a uma fraca correlação entre os campos atmosféricos que caracterizam esta parameterização, sugerindo que as formulações dos fluxos de massa dentro da nuvem e o fenómeno de arrastamento do ar para dentro da nuvem devem ser revistos. A importância do ar saturado e não saturado para o CMT é estudada com o objectivo de alcançar um melhor entendimento acerca dos mecanismos físicos responsáveis pelo CMT. O ar não saturado e saturado na forma de correntes descendentes contribuem de forma determinante para o CMT e deverão ser considerados em futuras parameterizações do CMT e da convecção em nuvens cumulus. Métodos de agrupamento foram aplicados às contribuições do ar saturado e não saturado, analisando os campos da força de flutuação e da velocidade vertical da partícula de ar, concluindo-se a presença de ondas gravíticas internas como mecanismo responsável pelo ar não saturado. A força do gradiente de pressão dentro da nuvem é também avaliada, utilizando para este efeito a fórmula teórica proposta por Gregory et al. (1997). Uma boa correlação entre esta força e o produto entre efeito de cisalhamento do vento e a perturbação da velocidade vertical é registada, principalmente para as correntes ascendentes dentro da nuvem durante o episódio de convecção profunda. No entanto, o valor ideal para o coeficiente empírico c*, que caracteriza a influência da força do gradiente de pressão dentro da nuvem sobre a variação vertical da velocidade horizontal dentro da nuvem, não é satisfatoriamente alcançado. Bons resultados são alcançados através do teste feito à aproximação do fluxo de massa proposta por Kershaw e Gregory (1997) para o cálculo do CMT total, revelando mais uma vez a importância do ar não saturado para o CMT.

Energy-aware Workflow Scheduling and Fuzzy Logic Based Capacity Provisioning in Cloud Environment

Thesis (Master's)--University of Washington, 2015

Business Models for Cloud Computing: Experiences from Developing Modeling & Simulation as a Service Applications in Industry

The potential of cloud computing is gaining significant interest in Modeling & Simulation (M&S). The underlying concept of using computing power as a utility is very attractive to users that can access state-of-the-art hardware and software without capital investment. Moreover, the cloud computing characteristics of rapid elasticity and the ability to scale up or down according to workload make it very attractive to numerous applications including M&S. Research and development work typically focuses on the implementation of cloud-based systems supporting M&S as a Service (MSaaS). Such systems are typically composed of a supply chain of technology services. How is the payment collected from the end-user and distributed to the stakeholders in the supply chain? We discuss the business aspects of developing a cloud platform for various M&S applications. Business models from the perspectives of the stakeholders involved in providing and using MSaaS and cloud computing are investigated and presented.

"One of our hosts in another country": Challenges of data geolocation in cloud storage

Physical location of data in cloud storage is an increasingly urgent problem. In a short time, it has evolved from the concern of a few regulated businesses to an important consideration for many cloud storage users. One of the characteristics of cloud storage is fluid transfer of data both within and among the data centres of a cloud provider. However, this has weakened the guarantees with respect to control over data replicas, protection of data in transit and physical location of data. This paper addresses the lack of reliable solutions for data placement control in cloud storage systems. We analyse the currently available solutions and identify their shortcomings. Furthermore, we describe a high-level architecture for a trusted, geolocation-based mechanism for data placement control in distributed cloud storage systems, which are the basis of an on-going work to define the detailed protocol and a prototype of such a solution. This mechanism aims to provide granular control over the capabilities of tenants to access data placed on geographically dispersed storage units comprising the cloud storage.

Distributed Agent-Based Load Balancer for Cloud Computing

In this paper we present a concept of an agent-based strategy to allocate services on a Cloud system without overloading nodes and maintaining the system stability with minimum cost. To provide a base for our research we specify an abstract model of cloud resources utilization, including multiple types of resources as well as considerations for the service migration costs. We also present an early version of simulation environment and a prototype of agent-based load balancer implemented in functional language Scala and Akka framework.

A Meta-Heuristic Load Balancer for Cloud Computing Systems

This paper introduces a strategy to allocate services on a cloud system without overloading the nodes and maintaining the system stability with minimum cost. We specify an abstract model of cloud resources utilization, including multiple types of resources as well as considerations for the service migration costs. A prototype meta-heuristic load balancer is demonstrated and experimental results are presented and discussed. We also propose a novel genetic algorithm, where population is seeded with the outputs of other meta-heuristic algorithms.

The Data of Things: Strategies, Patterns and Practice of Cloud-based Participatory Sensing

The broad capabilities of current mobile devices have paved the way for Mobile Crowd Sensing (MCS) applications. The success of this emerging paradigm strongly depends on the quality of received data which, in turn, is contingent to mass user participation; the broader the participation, the more useful these systems become. However, there is an ongoing trend that tries to integrate MCS applications with emerging computing paradigms such as cloud computing. The intuition is that such a transition can significantly improve the overall efficiency while at the same time it offers stronger security and privacy-preserving mechanisms for the end-user. In this position paper, we dwell on the underpinnings of incorporating cloud computing techniques to facilitate the vast amount of data collected in MCS applications. That is, we present a list of core system, security and privacy requirements that must be met if such a transition is to be successful. To this end, we first address several competing challenges not previously considered in the literature such as the scarce energy resources of battery-powered mobile devices as well as their limited computational resources that they often prevent the use of computationally heavy cryptographic operations and thus offering limited security services to the end-user. Finally, we present a use case scenario as a comprehensive example. Based on our findings, we posit open issues and challenges, and discuss possible ways to address them, so that security and privacy do not hinder the migration of MCS systems to the cloud.