Data analytics in the cloud with flexible mapreduced workflows

Data analytic applications are characterized by large data sets that are subject to a series of processing phases. Some of these phases are executed sequentially but others can be executed concurrently or in parallel on clusters, grids or clouds. The MapReduce programming model has been applied to process large data sets in cluster and cloud environments. For developing an application using MapReduce there is a need to install/configure/access specific frameworks such as Apache Hadoop or Elastic MapReduce in Amazon Cloud. It would be desirable to provide more flexibility in adjusting such configurations according to the application characteristics. Furthermore the composition of the multiple phases of a data analytic application requires the specification of all the phases and their orchestration. The original MapReduce model and environment lacks flexible support for such configuration and composition. Recognizing that scientific workflows have been successfully applied to modeling complex applications, this paper describes our experiments on implementing MapReduce as subworkflows in the AWARD framework (Autonomic Workflow Activities Reconfigurable and Dynamic). A text mining data analytic application is modeled as a complex workflow with multiple phases, where individual workflow nodes support MapReduce computations. As in typical MapReduce environments, the end user only needs to define the application algorithms for input data processing and for the map and reduce functions. In the paper we present experimental results when using the AWARD framework to execute MapReduce workflows deployed over multiple Amazon EC2 (Elastic Compute Cloud) instances.

4Sensing - decentralized processing for participatory sensing data

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática.

An Approach To Publish a Data Warehouse Content as Linked Data

Mestrado em Engenharia Informática - Área de Especialização em Tecnologias do Conhecimento e Decisão

A Communication and Resources Management Scheme to Support the Smart Grid Integration of Multiplayers Access to Resources Information

The increasing and intensive integration of distributed energy resources into distribution systems requires adequate methodologies to ensure a secure operation according to the smart grid paradigm. In this context, SCADA (Supervisory Control and Data Acquisition) systems are an essential infrastructure. This paper presents a conceptual design of a communication and resources management scheme based on an intelligent SCADA with a decentralized, flexible, and intelligent approach, adaptive to the context (context awareness). The methodology is used to support the energy resource management considering all the involved costs, power flows, and electricity prices leading to the network reconfiguration. The methodology also addresses the definition of the information access permissions of each player to each resource. The paper includes a 33-bus network used in a case study that considers an intensive use of distributed energy resources in five distinct implemented operation contexts.

Artificial Intelligence in geospatial analysis: applications of self-organizing maps in the context of geographic information science.

A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Information Systems.

Parallel Software framework for time-critical many-core systems

The recent technological advancements and market trends are causing an interesting phenomenon towards the convergence of High-Performance Computing (HPC) and Embedded Computing (EC) domains. On one side, new kinds of HPC applications are being required by markets needing huge amounts of information to be processed within a bounded amount of time. On the other side, EC systems are increasingly concerned with providing higher performance in real-time, challenging the performance capabilities of current architectures. The advent of next-generation many-core embedded platforms has the chance of intercepting this converging need for predictable high-performance, allowing HPC and EC applications to be executed on efficient and powerful heterogeneous architectures integrating general-purpose processors with many-core computing fabrics. To this end, it is of paramount importance to develop new techniques for exploiting the massively parallel computation capabilities of such platforms in a predictable way. P-SOCRATES will tackle this important challenge by merging leading research groups from the HPC and EC communities. The time-criticality and parallelisation challenges common to both areas will be addressed by proposing an integrated framework for executing workload-intensive applications with real-time requirements on top of next-generation commercial-off-the-shelf (COTS) platforms based on many-core accelerated architectures. The project will investigate new HPC techniques that fulfil real-time requirements. The main sources of indeterminism will be identified, proposing efficient mapping and scheduling algorithms, along with the associated timing and schedulability analysis, to guarantee the real-time and performance requirements of the applications.

A learning approach to swarm-based path detection and tracking

Dissertação para obtenção do Grau de Mestre em Engenharia Electrotécnica e de Computadores

Fasciola hepatica IN BOVINES IN BRAZIL: DATA AVAILABILITY AND SPATIAL DISTRIBUTION

Fasciolosis is a disease of importance for both veterinary and public health. For the first time, georeferenced prevalence data of Fasciola hepatica in bovines were collected and mapped for the Brazilian territory and data availability was discussed. Bovine fasciolosis in Brazil is monitored on a Federal, State and Municipal level, and to improve monitoring it is essential to combine the data collected on these three levels into one dataset. Data were collected for 1032 municipalities where livers were condemned by the Federal Inspection Service (MAPA/SIF) because of the presence of F. hepatica. The information was distributed over 11 states: Espírito Santo, Goiás, Minas Gerais, Mato Grosso do Sul, Mato Grosso, Pará, Paraná, Rio de Janeiro, Rio Grande do Sul, Santa Catarina and São Paulo. The highest prevalence of fasciolosis was observed in the southern states, with disease clusters along the coast of Paraná and Santa Catarina and in Rio Grande do Sul. Also, temporal variation of the prevalence was observed. The observed prevalence and the kriged prevalence maps presented in this paper can assist both animal and human health workers in estimating the risk of infection in their state or municipality.

Synthesis, characterization and applications of new schiff base fluorescent chemosensors for metal and DNA interactions: conventional and "green" approaches

Dissertação apresentada para a obtenção do Grau de Doutor em Química Sustentável, especialidade de Química-Física Inorgânica, pela Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia

Integrated data model and DSL modifications

Companies are increasingly more and more dependent on distributed web-based software systems to support their businesses. This increases the need to maintain and extend software systems with up-to-date new features. Thus, the development process to introduce new features usually needs to be swift and agile, and the supporting software evolution process needs to be safe, fast, and efficient. However, this is usually a difficult and challenging task for a developer due to the lack of support offered by programming environments, frameworks, and database management systems. Changes needed at the code level, database model, and the actual data contained in the database must be planned and developed together and executed in a synchronized way. Even under a careful development discipline, the impact of changing an application data model is hard to predict. The lifetime of an application comprises changes and updates designed and tested using data, which is usually far from the real, production, data. So, coding DDL and DML SQL scripts to update database schema and data, is the usual (and hard) approach taken by developers. Such manual approach is error prone and disconnected from the real data in production, because developers may not know the exact impact of their changes. This work aims to improve the maintenance process in the context of Agile Platform by Outsystems. Our goal is to design and implement new data-model evolution features that ensure a safe support for change and a sound migration process. Our solution includes impact analysis mechanisms targeting the data model and the data itself. This provides, to developers, a safe, simple, and guided evolution process.

Text mining na análise de sentimentos em contextos de big data

Dissertação de mestrado integrado em Engenharia e Gestão de Sistemas de Informação

Innovation sources and productivity in Catalonian firms: a quantile regression analysis

This paper explores the effects of two main sources of innovation -intramural and external R&D- on the productivity level in a sample of 3,267 Catalonian firms. The data set used is based on the official innovation survey of Catalonia which was a part of the Spanish sample of CIS4, covering the years 2002-2004. We compare empirical results by applying usual OLS and quantile regression techniques both in manufacturing and services industries. In quantile regression, results suggest different patterns at both innovation sources as we move across conditional quantiles. The elasticity of intramural R&D activities on productivity decreased when we move up the high productivity levels both in manufacturing and services sectors, while the effects of external R&D rise in high-technology industries but are more ambiguous in low-technology and knowledge-intensive services. JEL codes: O300, C100, O140. Keywords: Innovation sources, R&D, Productivity, Quantile regression

Barriers to innovation and public policy in Catalonia

The present paper analyses the link between firms’ decisions to innovate and the barriers that prevent them from being innovative. The aim is twofold. First, it analyses three groups of barriers to innovation: the cost of innovation projects, lack of knowledge and market conditions. Second, it presents the main steps taken by Catalan Government to promote the creation of new firms and to reduce barriers to innovation. The data set used is based on the 2004 official innovation survey of Catalonia which was taken from the Spanish CIS-4 sample. This sample includes individual information on 2,954 Catalan firms in manufacturing industries and knowledge-intensive services (KIS). The empirical analysis reveals pronounced differences regarding a firm’s propensity to innovate and its perception of barriers. Moreover, the results show that cost and knowledge barriers seem to be the most important and that there are substantial sectoral differences in the way that firms react to barriers. The results of this paper have important implications for the design of future public policy to promote entrepreneurship and innovation together.

Towards an emotion-driven interactive music system: ridging the gaps between affect, physiology and sound generation

Report for the scientific sojourn at the Stanford University from January until June 2007. Music is well known for affecting human emotional states, yet the relationship between specific musical parameters and emotional responses is still not clear. With the advent of new human-computer interaction (HCI) technologies, it is now possible to derive emotion-related information from physiological data and use it as an input to interactive music systems. Providing such implicit musical HCI will be highly relevant for a number of applications including music therapy, diagnosis, nteractive gaming, and physiologically-based musical instruments. A key question in such physiology-based compositions is how sound synthesis parameters can be mapped to emotional states of valence and arousal. We used both verbal and heart rate responses to evaluate the affective power of five musical parameters. Our results show that a significant correlation exists between heart rate and the subjective evaluation of well-defined musical parameters. Brightness and loudness showed to be arousing parameters on subjective scale while harmonicity and even partial attenuation factor resulted in heart rate changes typically associated to valence. This demonstrates that a rational approach to designing emotion-driven music systems for our public installations and music therapy applications is possible.

Innovation sources and productivity in Catalonian firms: a quantile regression analysis

This paper explores the effects of two main sources of innovation - intramural and external R&D— on the productivity level in a sample of 3,267 Catalonian firms. The data set used is based on the official innovation survey of Catalonia which was a part of the Spanish sample of CIS4, covering the years 2002-2004. We compare empirical results by applying usual OLS and quantile regression techniques both in manufacturing and services industries. In quantile regression, results suggest different patterns at both innovation sources as we move across conditional quantiles. The elasticity of intramural R&D activities on productivity decreased when we move up the high productivity levels both in manufacturing and services sectors, while the effects of external R&D rise in high-technology industries but are more ambiguous in low-technology and knowledge-intensive services. JEL codes: O300, C100, O140 Keywords: Innovation sources, R&D, Productivity, Quantile Regression