Data analytics in the cloud with flexible mapreduced workflows

Data analytic applications are characterized by large data sets that are subject to a series of processing phases. Some of these phases are executed sequentially but others can be executed concurrently or in parallel on clusters, grids or clouds. The MapReduce programming model has been applied to process large data sets in cluster and cloud environments. For developing an application using MapReduce there is a need to install/configure/access specific frameworks such as Apache Hadoop or Elastic MapReduce in Amazon Cloud. It would be desirable to provide more flexibility in adjusting such configurations according to the application characteristics. Furthermore the composition of the multiple phases of a data analytic application requires the specification of all the phases and their orchestration. The original MapReduce model and environment lacks flexible support for such configuration and composition. Recognizing that scientific workflows have been successfully applied to modeling complex applications, this paper describes our experiments on implementing MapReduce as subworkflows in the AWARD framework (Autonomic Workflow Activities Reconfigurable and Dynamic). A text mining data analytic application is modeled as a complex workflow with multiple phases, where individual workflow nodes support MapReduce computations. As in typical MapReduce environments, the end user only needs to define the application algorithms for input data processing and for the map and reduce functions. In the paper we present experimental results when using the AWARD framework to execute MapReduce workflows deployed over multiple Amazon EC2 (Elastic Compute Cloud) instances.

Aplicação da técnica Análise do Valor (AV) a um determinado porduto/serviço

Mestrado em Engenharia Mecânica - Gestão de Processos e Operações

Sistema Multi-Agente de prestação de serviços de seguimento adaptados aos requisitos do utilizador

Mestrado em Engenharia Electrotécnica e de Computadores - Área de Especialização de Telecomunicações

An high-speed parametric ADC and a co-designed mixer for CMOS RF receivers

Dissertação apresentada na faculdade de Ciências e Tecnologia da Universidade Nova de Lisboa para a obtenção do grau de Mestre em Engenharia Electrotécnica e de Computadores

Seqins: a sequencing tool for educational resources

The teaching-learning process is increasingly focused on the combination of the paradigms “learning by viewing” and “learning by doing.” In this context, educational resources, either expository or evaluative, play a pivotal role. Both types of resources are interdependent and their sequencing would create a richer educational experience to the end user. However, there is a lack of tools that support sequencing essentially due to the fact that existing specifications are complex. The Seqins is a sequencing tool of digital resources that has a fairly simple sequencing model. The tool communicates through the IMS LTI specification with a plethora of e-learning systems such as learning management systems, repositories, authoring and evaluation systems. In order to validate Seqins we integrate it in an e-learning Ensemble framework instance for the computer programming learning.

Multi-agent system for personalization of location-based services

The current ubiquitous network access and increase in network bandwidth are driving the sales of mobile location-aware user devices and, consequently, the development of context-aware applications, namely location-based services. The goal of this project is to provide consumers of location-based services with a richer end-user experience by means of service composition, personalization, device adaptation and continuity of service. Our approach relies on a multi-agent system composed of proxy agents that act as mediators and providers of personalization meta-services, device adaptation and continuity of service for consumers of pre-existing location-based services. These proxy agents, which have Web services interfaces to ensure a high level of interoperability, perform service composition and take in consideration the preferences of the users, the limitations of the user devices, making the usage of different types of devices seamless for the end-user. To validate and evaluate the performance of this approach, use cases were defined, tests were conducted and results gathered which demonstrated that the initial goals were successfully fulfilled.

Characterisation of noisy speech channels in 2G and 3G mobile networks

As the wireless cellular market reaches competitive levels never seen before, network operators need to focus on maintaining Quality of Service (QoS) a main priority if they wish to attract new subscribers while keeping existing customers satisfied. Speech Quality as perceived by the end user is one major example of a characteristic in constant need of maintenance and improvement. It is in this topic that this Master Thesis project fits in. Making use of an intrusive method of speech quality evaluation, as a means to further study and characterize the performance of speech codecs in second-generation (2G) and third-generation (3G) technologies. Trying to find further correlation between codecs with similar bit rates, along with the exploration of certain transmission parameters which may aid in the assessment of speech quality. Due to some limitations concerning the audio analyzer equipment that was to be employed, a different system for recording the test samples was sought out. Although the new designed system is not standard, after extensive testing and optimization of the system's parameters, final results were found reliable and satisfactory. Tests include a set of high and low bit rate codecs for both 2G and 3G, where values were compared and analysed, leading to the outcome that 3G speech codecs perform better, under the approximately same conditions, when compared with 2G. Reinforcing the idea that 3G is, with no doubt, the best choice if the costumer looks for the best possible listening speech quality. Regarding the transmission parameters chosen for the experiment, the Receiver Quality (RxQual) and Received Energy per Chip to the Power Density Ratio (Ec/N0), these were subject to speech quality correlation tests. Final results of RxQual were compared to those of prior studies from different researchers and, are considered to be of important relevance. Leading to the confirmation of RxQual as a reliable indicator of speech quality. As for Ec/N0, it is not possible to state it as a speech quality indicator however, it shows clear thresholds for which the MOS values decrease significantly. The studied transmission parameters show that they can be used not only for network management purposes but, at the same time, give an expected idea to the communications engineer (or technician) of the end-to-end speech quality consequences. With the conclusion of the work new ideas for future studies come to mind. Considering that the fourth-generation (4G) cellular technologies are now beginning to take an important place in the global market, as the first all-IP network structure, it seems of great relevance that 4G speech quality should be subject of evaluation. Comparing it to 3G, not only in narrowband but also adding wideband scenarios with the most recent standard objective method of speech quality assessment, POLQA. Also, new data found on Ec/N0 tests, justifies further research studies with the intention of validating the assumptions made in this work.

Plataforma de Dados de Saúde - Portal Institucional

O presente documento de dissertação retrata o desenvolvimento do projeto PDS-Portal Institucional cujo cerne é um sistema para recolha, armazenamento e análise de dados (plataforma de Business Intelligence). Este portal está enquadrado na área da saúde e é uma peça fundamental no sistema da Plataforma de dados da Saúde, que é constituído por quatro portais distintos. Esta plataforma tem como base um sistema totalmente centrado no utente, que agrega dados de saúde dos utentes e distribui pelos diversos intervenientes: utente, profissionais de saúde nacionais e internacionais e organizações de saúde. O objetivo principal deste projeto é o desenvolvimento do PDS-Portal Institucional, recorrendo a uma plataforma de Business Intelligence, com o intuito de potenciar os utilizadores de uma ferramenta analítica para análise de dados. Estando a informação armazenada em dois dos portais da Plataforma de dados da Saúde (PDS-Portal Utente e PDS-Portal Profissional), é necessário modular um armazém de dados que agregue a informação de ambos e, através do PDS-PI, distribua um conjunto de análises ao utilizador final. Para tal este sistema comtempla um mecanismo totalmente automatizado para extração, tratamento e carregamento de dados para o armazém central, assim como uma plataforma de BI que disponibiliza os dados armazenados sobre a forma de análises específicas. Esta plataforma permite uma evolução constante e é extremamente flexível, pois fornece um mecanismo de gestão de utilizadores e perfis, assim como capacita o utilizador de um ambiente Web para análise de dados, permitindo a partilha e acesso a partir de dispositivos móveis. Após a implementação deste sistema foi possível explorar os dados e tirar diversas conclusões que são de extrema importância tanto para a evolução da PDS como para os métodos de praticar os cuidados de saúde em Portugal. Por fim são identificados alguns pontos de melhoria do sistema atual e delineada uma perspetiva de evolução futura. É certo que a partir do momento que este projeto seja lançado para produção, novas oportunidades surgirão e o contributo dos utilizadores será útil para evoluir o sistema progressivamente.

A constructive technology assessment of stationary energy storage systems: prospective life cycle orientated analysis

Based on the presentation and discussion at the 3rd Winter School on Technology Assessment, December 2012, Universidade Nova de Lisboa (Portugal), Caparica Campus, PhD programme on Technology Assessment

TSKY: a dependable middleware solution for data privacy using public storage clouds

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

An agent-based simulator to estimate domestic energy use

Throughout recent years, there has been an increase in the population size, as well as a fast economic growth, which has led to an increase of the energy demand that comes mainly from fossil fuels. In order to reduce the ecological footprint, governments have implemented sustainable measures and it is expected that by 2035 the energy produced from renewable energy sources, such as wind and solar would be responsible for one-third of the energy produced globally. However, since the energy produced from renewable sources is governed by the availability of the respective primary energy source there is often a mismatch between production and demand, which could be solved by adding flexibility on the demand side through demand response (DR). DR programs influence the end-user electricity usage by changing its cost along the time. Under this scenario the user needs to estimate the energy demand and on-site production in advance to plan its energy demand according to the energy price. This work focuses on the development of an agent-based electrical simulator, capable of: (a) estimating the energy demand and on-site generation with a 1-min time resolution for a 24-h period, (b) calculating the energy price for a given scenario, (c) making suggestions on how to maximize the usage of renewable energy produced on-site and to lower the electricity costs by rescheduling the use of certain appliances. The results show that this simulator allows reducing the energy bill by 11% and almost doubling the use of renewable energy produced on-site.

The new economy: Essays in network economics and two-sided markets

Following the Introduction, which surveys existing literature on the technology advances and regulation in telecommunications and on two-sided markets, we address specific issues on the industries of the New Economy, featured by the existence of network effects. We seek to explore how each one of these industries work, identify potential market failures and find new solutions at the economic regulation level promoting social welfare. In Chapter 1 we analyze a regulatory issue on access prices and investments in the telecommunications market. The existing literature on access prices and investment has pointed out that networks underinvest under a regime of mandatory access provision with a fixed access price per end-user. We propose a new access pricing rule, the indexation approach, i.e., the access price, per end-user, that network i pays to network j is function of the investment levels set by both networks. We show that the indexation can enhance economic efficiency beyond what is achieved with a fixed access price. In particular, access price indexation can simultaneously induce lower retail prices and higher investment and social welfare as compared to a fixed access pricing or a regulatory holidays regime. Furthermore, we provide sufficient conditions under which the indexation can implement the socially optimal investment or the Ramsey solution, which would be impossible to obtain under fixed access pricing. Our results contradict the notion that investment efficiency must be sacrificed for gains in pricing efficiency. In Chapter 2 we investigate the effect of regulations that limit advertising airtime on advertising quality and on social welfare. We show, first, that advertising time regulation may reduce the average quality of advertising broadcast on TV networks. Second, an advertising cap may reduce media platforms and firms' profits, while the net effect on viewers (subscribers) welfare is ambiguous because the ad quality reduction resulting from a regulatory cap o¤sets the subscribers direct gain from watching fewer ads. We find that if subscribers are sufficiently sensitive to ad quality, i.e., the ad quality reduction outweighs the direct effect of the cap, a cap may reduce social welfare. The welfare results suggest that a regulatory authority that is trying to increase welfare via regulation of the volume of advertising on TV might necessitate to also regulate advertising quality or, if regulating quality proves impractical, take the effect of advertising quality into consideration. 3 In Chapter 3 we investigate the rules that govern Electronic Payment Networks (EPNs). In EPNs the No-Surcharge Rule (NSR) requires that merchants charge at most the same amount for a payment card transaction as for cash. In this chapter, we analyze a three- party model (consumers, merchants, and a proprietary EPN) with endogenous transaction volumes and heterogenous merchants' transactional benefits of accepting cards to assess the welfare impacts of the NSR. We show that, if merchants are local monopolists and the network externalities from merchants to cardholders are sufficiently strong, with the exception of the EPN, all agents will be worse o¤ with the NSR, and therefore the NSR is socially undesirable. The positive role of the NSR in terms of improvement of retail price efficiency for cardholders is also highlighted.

Understanding churn and its reasons: The GuestU case

In the age of computerisation, entrepreneurs around the world aim to revolutionise their field with ground-breaking ICTs. GuestU is one of those offering mobile application development to small and medium-sized tourism enterprises. To assure the young start-ups steady growth this paper aims to identify reasons for the high customer churn risk. Primary data is used to analyse end-user behaviour and particularly the success of acquisition and activation. Results show troubling download quantities as well as retention rates leading to the recommendations for GuestU to incentivise and guide their customers in increasing promotion for the applications.

Testing low-altitude infrared digital photography from a mini-UAV to retrieve information for biological conservation

Satellite remote sensing imagery is used for forestry, conservation and environmental applications, but insufficient spatial resolution, and, in particular, unavailability of images at the precise timing required for a given application, often prevent achieving a fully operational stage. Airborne remote sensing has the advantage of custom-tuned sensors, resolution and timing, but its price prevents using it as a routine technique for the mentioned fields. Some Unmanned Aerial Vehicles might provide a “third way” solution as low-cost techniques for acquiring remotely sensed information, under close control of the end-user, albeit at the expense of lower quality instrumentation and instability. This report evaluates a light remote sensing system based on a remotely-controlled mini-UAV (ATMOS-3) equipped with a color infra-red camera (VEGCAM-1) designed and operated by CATUAV. We conducted a testing mission over a Mediterranean landscape dominated by an evergreen woodland of Aleppo pine (Pinus halepensis) and (Holm) oak (Quercus ilex) in the Montseny National Park (Catalonia, NE Spain). We took advantage of state-of-the-art ortho-rectified digital aerial imagery (acquired by the Institut Cartogràfic de Catalunya over the area during the previous year) and used it as quality reference. In particular, we paid attention to: 1) Operationality of flight and image acquisition according to a previously defined plan; 2) Radiometric and geometric quality of the images; and 3) Operational use of the images in the context of applications. We conclude that the system has achieved an operational stage regarding flight activities, although with meteorological limits set by wind speed and turbulence. Appropriate landing areas can be sometimes limiting also, but the system is able to land on small and relatively rough terrains such as patches of grassland or short matorral, and we have operated the UAV as far as 7 km from the control unit. Radiometric quality is sufficient for interactive analysis, but probably insufficient for automated processing. A forthcoming camera is supposed to greatly improve radiometric quality and consistency. Conventional GPS positioning through time synchronization provides coarse orientation of the images, with no roll information.

Sky Net : sistema de sincronización y control de versiones de documentos on-line

La función de la aplicación Sky Net (un sistema de control de versiones y sincronización de documentos on-line) será la de ofrecer un conjunto de herramientas capaces de establecer una comunicación cliente-servidor de forma transparente utilizando carpetas del disco duro. De esta forma el usuario final podrá tener todos sus documentos guardados en un servidor, poderlos recuperar en el momento que se desee, y tener los mismos documentos en todas las estaciones de trabajo que estén conectadas a la aplicación. Además se le añade una función de control de versiones que permitirá guardar el contenido de una carpeta bajo un identificador (nombre o versión), y poderlo recuperar más adelante aunque ya hayamos hecho modificaciones en los documentos de dicha carpeta. El proyecto se compone de dos aplicaciones: Sky Catcher, la aplicación cliente, y Sky Node, la aplicación servidor.