Phytoplankton analysis of portuguese reservoirs: a cluster analysis with R

In this work, cluster analysis is applied to a real dataset of biological features of several Portuguese reservoirs. All the statistical analysis is done using R statistical software. Several metrics and methods were explored, as well as the combination of Euclidean metric and the hierarchical Ward method. Although it did not present the best combination in terms of internal and stability validation, it was still a good solution and presented good results in terms of interpretation of the problem at hand.

Bases de Dados NoSQL

Com o advento da invenção do modelo relacional em 1970 por E.F.Codd, a forma como a informação era gerida numa base de dados foi totalmente revolucionada. Migrou‐se de sistemas hierárquicos baseados em ficheiros para uma base de dados relacional com tabelas relações e registos que simplificou em muito a gestão da informação e levou muitas empresas a adotarem este modelo. O que E.F.Codd não previu foi o facto de que cada vez mais a informação que uma base de dados teria de armazenar fosse de proporções gigantescas, nem que as solicitações às bases de dados fossem da mesma ordem. Tudo isto veio a acontecer com a difusão da internet que veio ligar todas as pessoas de qualquer parte do mundo que tivessem um computador. Com o número de adesões à internet a crescer, o número de sites que nela eram criados também cresceu (e ainda cresce exponencialmente). Os motores de busca que antigamente indexavam alguns sites por dia, atualmente indexam uns milhões de sites por segundo e, mais recentemente as redes sociais também estão a lidar com quantidades gigantescas de informação. Tanto os motores de busca como as redes sociais chegaram à conclusão que uma base de dados relacional não chega para gerir a enorme quantidade de informação que ambos produzem e como tal, foi necessário encontrar uma solução. Essa solução é NoSQL e é o assunto que esta tese vai tratar. O presente documento visa definir e apresentar o problema que as bases de dados relacionais têm quando lidam com grandes volumes de dados, introduzir os limites do modelo relacional que só até há bem pouco tempo começaram a ser evidenciados com o surgimento de movimentos, como o BigData, com o crescente número de sites que surgem por dia e com o elevado número de utilizadores das redes sociais. Será também ilustrada a solução adotada até ao momento pelos grandes consumidores de dados de elevado volume, como o Google e o Facebook, enunciando as suas características vantagens, desvantagens e os demais conceitos ligados ao modelo NoSQL. A presente tese tenciona ainda demonstrar que o modelo NoSQL é uma realidade usada em algumas empresas e quais as principias mudanças a nível programático e as boas práticas delas resultantes que o modelo NoSQL traz. Por fim esta tese termina com a explicação de que NoSQL é uma forma de implementar a persistência de uma aplicação que se inclui no novo modelo de persistência da informação.

Electricity markets modeling considering complex contracts and aggregators

All over the world, the liberalization of electricity markets, which follows different paradigms, has created new challenges for those involved in this sector. In order to respond to these challenges, electric power systems suffered a significant restructuring in its mode of operation and planning. This restructuring resulted in a considerable increase of the electric sector competitiveness. Particularly, the Ancillary Services (AS) market has been target of constant renovations in its operation mode as it is a targeted market for the trading of services, which have as main objective to ensure the operation of electric power systems with appropriate levels of stability, safety, quality, equity and competitiveness. In this way, with the increasing penetration of distributed energy resources including distributed generation, demand response, storage units and electric vehicles, it is essential to develop new smarter and hierarchical methods of operation of electric power systems. As these resources are mostly connected to the distribution network, it is important to consider the introduction of this kind of resources in AS delivery in order to achieve greater reliability and cost efficiency of electrical power systems operation. The main contribution of this work is the design and development of mechanisms and methodologies of AS market and for energy and AS joint market, considering different management entities of transmission and distribution networks. Several models developed in this work consider the most common AS in the liberalized market environment: Regulation Down; Regulation Up; Spinning Reserve and Non-Spinning Reserve. The presented models consider different rules and ways of operation, such as the division of market by network areas, which allows the congestion management of interconnections between areas; or the ancillary service cascading process, which allows the replacement of AS of superior quality by lower quality of AS, ensuring a better economic performance of the market. A major contribution of this work is the development an innovative methodology of market clearing process to be used in the energy and AS joint market, able to ensure viable and feasible solutions in markets, where there are technical constraints in the transmission network involving its division into areas or regions. The proposed method is based on the determination of Bialek topological factors and considers the contribution of the dispatch for all services of increase of generation (energy, Regulation Up, Spinning and Non-Spinning reserves) in network congestion. The use of Bialek factors in each iteration of the proposed methodology allows limiting the bids in the market while ensuring that the solution is feasible in any context of system operation. Another important contribution of this work is the model of the contribution of distributed energy resources in the ancillary services. In this way, a Virtual Power Player (VPP) is considered in order to aggregate, manage and interact with distributed energy resources. The VPP manages all the agents aggregated, being able to supply AS to the system operator, with the main purpose of participation in electricity market. In order to ensure their participation in the AS, the VPP should have a set of contracts with the agents that include a set of diversified and adapted rules to each kind of distributed resource. All methodologies developed and implemented in this work have been integrated into the MASCEM simulator, which is a simulator based on a multi-agent system that allows to study complex operation of electricity markets. In this way, the developed methodologies allow the simulator to cover more operation contexts of the present and future of the electricity market. In this way, this dissertation offers a huge contribution to the AS market simulation, based on models and mechanisms currently used in several real markets, as well as the introduction of innovative methodologies of market clearing process on the energy and AS joint market. This dissertation presents five case studies; each one consists of multiple scenarios. The first case study illustrates the application of AS market simulation considering several bids of market players. The energy and ancillary services joint market simulation is exposed in the second case study. In the third case study it is developed a comparison between the simulation of the joint market methodology, in which the player bids to the ancillary services is considered by network areas and a reference methodology. The fourth case study presents the simulation of joint market methodology based on Bialek topological distribution factors applied to transmission network with 7 buses managed by a TSO. The last case study presents a joint market model simulation which considers the aggregation of small players to a VPP, as well as complex contracts related to these entities. The case study comprises a distribution network with 33 buses managed by VPP, which comprises several kinds of distributed resources, such as photovoltaic, CHP, fuel cells, wind turbines, biomass, small hydro, municipal solid waste, demand response, and storage units.

Dynamic Analysis and Pattern Visualization of Forest Fires

This paper analyses forest fires in the perspective of dynamical systems. Forest fires exhibit complex correlations in size, space and time, revealing features often present in complex systems, such as the absence of a characteristic length-scale, or the emergence of long range correlations and persistent memory. This study addresses a public domain forest fires catalogue, containing information of events for Portugal, during the period from 1980 up to 2012. The data is analysed in an annual basis, modelling the occurrences as sequences of Dirac impulses with amplitude proportional to the burnt area. First, we consider mutual information to correlate annual patterns. We use visualization trees, generated by hierarchical clustering algorithms, in order to compare and to extract relationships among the data. Second, we adopt the Multidimensional Scaling (MDS) visualization tool. MDS generates maps where each object corresponds to a point. Objects that are perceived to be similar to each other are placed on the map forming clusters. The results are analysed in order to extract relationships among the data and to identify forest fire patterns.

State Space Analysis of Forest Fires

This paper studies forest fires from the perspective of dynamical systems. Burnt area, precipitation and atmospheric temperatures are interpreted as state variables of a complex system and the correlations between them are investigated by means of different mathematical tools. First, we use mutual information to reveal potential relationships in the data. Second, we adopt the state space portrait to characterize the system’s behavior. Third, we compare the annual state space curves and we apply clustering and visualization tools to unveil long-range patterns. We use forest fire data for Portugal, covering the years 1980–2003. The territory is divided into two regions (North and South), characterized by different climates and vegetation. The adopted methodology represents a new viewpoint in the context of forest fires, shedding light on a complex phenomenon that needs to be better understood in order to mitigate its devastating consequences, at both economical and environmental levels.

Integer and Fractional Order Entropy Analysis of Earthquake Data-series

This paper studies the statistical distributions of worldwide earthquakes from year 1963 up to year 2012. A Cartesian grid, dividing Earth into geographic regions, is considered. Entropy and the Jensen–Shannon divergence are used to analyze and compare real-world data. Hierarchical clustering and multi-dimensional scaling techniques are adopted for data visualization. Entropy-based indices have the advantage of leading to a single parameter expressing the relationships between the seismic data. Classical and generalized (fractional) entropy and Jensen–Shannon divergence are tested. The generalized measures lead to a clear identification of patterns embedded in the data and contribute to better understand earthquake distributions.

Manoeuvre Based Mission Control System for an Autonomous Surface Vehicle

In this work the mission control and supervision system developed for the ROAZ Autonomous Surface Vehicle is presented. Complexity in mission requirements coupled with flexibility lead to the design of a modular hierarchical mission control system based on hybrid systems control. Monitoring and supervision control for a vehicle such as ROAZ mission is not an easy task using tools with low complexity and yet powerful enough. A set of tools were developed to perform both on board mission control and remote planning and supervision. “ROAZ- Mission Control” was developed to be used in support to bathymetric and security missions performed in river and at seas.

An automated maneuver control framework for a remotely operated vehicle

OCEANS, 2001. MTS/IEEE Conference and Exhibition (Volume:2 )

New Features to Look at Natural Phenomena

Proceeding of the 3rd International Conference on Fractional Systems and Signals, at Ghent, Belgium

Analysis of Natural and Artificial Phenomena Using Signal Processing and Fractional Calculus

In this paper we study several natural and man-made complex phenomena in the perspective of dynamical systems. For each class of phenomena, the system outputs are time-series records obtained in identical conditions. The time-series are viewed as manifestations of the system behavior and are processed for analyzing the system dynamics. First, we use the Fourier transform to process the data and we approximate the amplitude spectra by means of power law functions. We interpret the power law parameters as a phenomenological signature of the system dynamics. Second, we adopt the techniques of non-hierarchical clustering and multidimensional scaling to visualize hidden relationships between the complex phenomena. Third, we propose a vector field based analogy to interpret the patterns unveiled by the PL parameters.

Multidimensional Scalling Analysis of the World Economy During the Period 1972-2012

The last 40 years of the world economy are analyzed by means of computer visualization methods. Multidimensional scaling and the hierarchical clustering tree techniques are used. The current Western downturn in favor of Asian partners may still be reversed in the coming decades.

Scalable and interference aware wi-fi mesh networks using cots devices

A crescente tendencia no acesso móvel tem sido potenciada pela tecnologia IEEE 802.11. Contudo, estas redes têm alcance rádio limitado. Para a extensão da sua cobertura é possível recorrer a redes emalhadas sem fios baseadas na tecnologia IEEE 802.11, com vantagem do ponto de vista do custo e da flexibilidade de instalação, face a soluções cabladas. Redes emalhadas sem fios constituídas por nós com apenas uma interface têm escalabilidade reduzida. A principal razão dessa limitação deve-se ao uso do mecanismo de acesso ao meio partilhado Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) em topologias multi-hop. Especificamente, o CSMA/CA não evita o problema do nó escondido levando ao aumento do número de colisões e correspondente degradação de desempenho com impacto direto no throughput e na latência. Com a redução da tecnologia rádio torna-se viável a utilização de múltiplos rádios por nó, sem com isso aumentar significativamente o custo da solução final de comunicações. A utilização de mais do que um rádio por nó de comuniações permite superar os problemas de desempenho inerentes ás redes formadas por nós com apenas um rádio. O objetivo desta tese, passa por desenvolver uma nova solução para redes emalhadas multi-cana, duar-radio, utilizando para isso novos mecanismos que complementam os mecanismos definidos no IEEE 802.11 para o estabelecimento de um Basic Service Set (BSS). A solução é baseada na solução WiFIX, um protocolo de routing para redes emalhadas de interface única e reutiliza os mecanismos já implementados nas redes IEEE 802.11 para difundir métricas que permitam à rede escalar de forma eficaz minimizando o impacto na performance. A rede multi-hop é formada por nós equipados com duas interfaces, organizados numa topologia hierárquica sobre múltiplas relações Access Point (AP) – Station (STA). Os resultados experimentais obtidos mostram a eficácia e o bom desempenho da solução proposta face à solução WiFIX original.

Clustering aplicado à Bolsa de Valores de Lisboa

O objetivo desta dissertação foi estudar um conjunto de empresas cotadas na bolsa de valores de Lisboa, para identificar aquelas que têm um comportamento semelhante ao longo do tempo. Para isso utilizamos algoritmos de Clustering tais como K-Means, PAM, Modelos hierárquicos, Funny e C-Means tanto com a distância euclidiana como com a distância de Manhattan. Para selecionar o melhor número de clusters identificado por cada um dos algoritmos testados, recorremos a alguns índices de avaliação/validação de clusters como o Davies Bouldin e Calinski-Harabasz entre outros.

Classificação dos créditos sobre a insolvência – Os créditos subordinados

Dissertação de Mestrado em Solicitaria