A MAP approach to evidence accumulation clustering

The Evidence Accumulation Clustering (EAC) paradigm is a clustering ensemble method which derives a consensus partition from a collection of base clusterings obtained using different algorithms. It collects from the partitions in the ensemble a set of pairwise observations about the co-occurrence of objects in a same cluster and it uses these co-occurrence statistics to derive a similarity matrix, referred to as co-association matrix. The Probabilistic Evidence Accumulation for Clustering Ensembles (PEACE) algorithm is a principled approach for the extraction of a consensus clustering from the observations encoded in the co-association matrix based on a probabilistic model for the co-association matrix parameterized by the unknown assignments of objects to clusters. In this paper we extend the PEACE algorithm by deriving a consensus solution according to a MAP approach with Dirichlet priors defined for the unknown probabilistic cluster assignments. In particular, we study the positive regularization effect of Dirichlet priors on the final consensus solution with both synthetic and real benchmark data.

Classificação e agregação automática de notícias desportivas

Mestrado em Engenharia Informática - Área de Especialização em Arquiteturas, Sistemas e Redes

Reducing the number of membership functions in linguistic variables

Dissertation presented at Universidade Nova de Lisboa, Faculdade de Ciências e Tecnologia in fulfilment of the requirements for the Masters degree in Mathematics and Applications, specialization in Actuarial Sciences, Statistics and Operations Research

Cultura organizacional e desempenho: o impacto da cultura organizacional no desempenho financeiro

Esta dissertação visa o estudo da influência da cultura organizacional no desempenho financeiro das organizações. Nesse contexto, procuramos analisar qual a cultura predominante das organizações, de forma a estabelecer posteriormente uma relação entre a cultura e o desempenho das empresas. Para isso a metodologia seguida foi a realização de um inquérito por questionário a empresas da região Douro de Portugal no sentido de obter, através de uma adaptação ao instrumento desenvolvido por Cameron e Quinn (2006), a cultura predominante da empresa, os indicadores financeiros necessários ao nosso estudo assim como, uma caracterização da amostra recolhida. Para análise e tratamento dos dados recolhidos através do inquérito por questionário foi utilizada a ferramenta estatística SPSS que nos permitiu retirar ilações sobre as características da amostra, assim como sobre a relação existente entre cultura organizacional e desempenho financeiro, esta relação foi avaliada através de testes de correlação e regressão linear múltipla. Os resultados sugerem que as variáveis culturais, cultura adocrática, mercado e hierárquica e o número de colaboradores explicam em cerca de 20% o resultado líquido ajustado. Também se verificou um efeito positivo da cultura adocrática e de mercado, embora o efeito da cultura de mercado seja mais forte que o da adocrática, e o efeito negativo da cultura hierárquica, ainda que estes resultados não sejam estatisticamente significativos. Não existem evidências que os tipos de cultura analisados (adocrática, de mercado e hierárquica) estão significativamente associados ao desempenho financeiro, avaliado pelos resultados líquidos ajustados, das empresas analisadas, quer pelos testes de correlação quer pelos resultados da estimação do modelo de regressão linear múltipla.

Determinants of frailty: the added value of assessing medication

This study aims to analyze which determinants predict frailty in general and each frailty domain (physical, psychological, and social), considering the integral conceptual model of frailty, and particularly to examine the contribution of medication in this prediction. A cross-sectional study was designed using a non-probabilistic sample of 252 community-dwelling elderly from three Portuguese cities. Frailty and determinants of frailty were assessed with the Tilburg Frailty Indicator. The amount and type of different daily-consumed medication were also examined. Hierarchical regression analysis were conducted. The mean age of the participants was 79.2 years (±7.3), and most of them were women (75.8%), widowed (55.6%) and with a low educational level (0–4 years: 63.9%). In this study, determinants explained 46% of the variance of total frailty, and 39.8, 25.3, and 27.7% of physical, psychological, and social frailty respectively. Age, gender, income, death of a loved one in the past year, lifestyle, satisfaction with living environment and self-reported comorbidity predicted total frailty, while each frailty domain was associated with a different set of determinants. The number of daily-consumed drugs was independently associated with physical frailty, and the consumption of medication for the cardiovascular system and for the blood and blood-forming organs explained part of the variance of total and physical frailty. The adverse effects of polymedication and its direct link with the level of comorbidities could explain the independent contribution of the amount of prescribed drugs to frailty prediction. On the other hand, findings in regard to medication type provide further evidence of the association of frailty with cardiovascular risk. In the present study, a significant part of frailty was predicted, and the different contributions of each determinant to frailty domains highlight the relevance of the integral model of frailty. The added value of a simple assessment of medication was considerable, and it should be taken into account for effective identification of frailty.

Multidimensional Frailty and Pain in Community Dwelling Elderly

Artigo científico disponível actualmente em Early View (Online Version of Record published before inclusion in an issue)

Phytoplankton analysis of portuguese reservoirs: a cluster analysis with R

In this work, cluster analysis is applied to a real dataset of biological features of several Portuguese reservoirs. All the statistical analysis is done using R statistical software. Several metrics and methods were explored, as well as the combination of Euclidean metric and the hierarchical Ward method. Although it did not present the best combination in terms of internal and stability validation, it was still a good solution and presented good results in terms of interpretation of the problem at hand.

A Data Mining Framework for Electric Load Profiling

This paper consists in the characterization of medium voltage (MV) electric power consumers based on a data clustering approach. It is intended to identify typical load profiles by selecting the best partition of a power consumption database among a pool of data partitions produced by several clustering algorithms. The best partition is selected using several cluster validity indices. These methods are intended to be used in a smart grid environment to extract useful knowledge about customers’ behavior. The data-mining-based methodology presented throughout the paper consists in several steps, namely the pre-processing data phase, clustering algorithms application and the evaluation of the quality of the partitions. To validate our approach, a case study with a real database of 1.022 MV consumers was used.

A Data-mining-based Methodology to support MV Electricity Customers' Characterization

This paper presents an electricity medium voltage (MV) customer characterization framework supportedby knowledge discovery in database (KDD). The main idea is to identify typical load profiles (TLP) of MVconsumers and to develop a rule set for the automatic classification of new consumers. To achieve ourgoal a methodology is proposed consisting of several steps: data pre-processing; application of severalclustering algorithms to segment the daily load profiles; selection of the best partition, corresponding tothe best consumers’ segmentation, based on the assessments of several clustering validity indices; andfinally, a classification model is built based on the resulting clusters. To validate the proposed framework,a case study which includes a real database of MV consumers is performed.

Load Profiling Tool to Support Smart Grid Operation Scenarios

This paper presents the characterization of high voltage (HV) electric power consumers based on a data clustering approach. The typical load profiles (TLP) are obtained selecting the best partition of a power consumption database among a pool of data partitions produced by several clustering algorithms. The choice of the best partition is supported using several cluster validity indices. The proposed data-mining (DM) based methodology, that includes all steps presented in the process of knowledge discovery in databases (KDD), presents an automatic data treatment application in order to preprocess the initial database in an automatic way, allowing time saving and better accuracy during this phase. These methods are intended to be used in a smart grid environment to extract useful knowledge about customers’ consumption behavior. To validate our approach, a case study with a real database of 185 HV consumers was used.

Screening pentachlorophenol degradation ability by environmental fungal strains belonging to the phyla Ascomycota and Zygomycota

Pentachlorophenol (PCP) bioremediation by the fungal strains amongst the cork- colonising community has not yet been analysed. In this paper, the co- and direct metabolism of PCP by each of the 17 fungal species selected from this community were studied. Using hierarchical data analysis, the isolates were ranked by their PCP bioremediation potential. Fifteen isolates were able to degrade PCP under co-metabolic conditions, and surprisingly Chrysonilia sitophila, Trichoderma longibrachiatum, Mucor plumbeus, Penicillium janczewskii and P. glandicola were able to directly metabolise PCP, leading to its complete depletion from media. PCP degradation intermediates are preliminarily discussed. Data emphasise the signiWcance of these fungi to have an interesting potential to be used in PCP bioremediation processes.

Hybrid systems biology: application to Escherichia coli

Dissertation presented to obtain a Master degree in Biotechnology

Bases de Dados NoSQL

Com o advento da invenção do modelo relacional em 1970 por E.F.Codd, a forma como a informação era gerida numa base de dados foi totalmente revolucionada. Migrou‐se de sistemas hierárquicos baseados em ficheiros para uma base de dados relacional com tabelas relações e registos que simplificou em muito a gestão da informação e levou muitas empresas a adotarem este modelo. O que E.F.Codd não previu foi o facto de que cada vez mais a informação que uma base de dados teria de armazenar fosse de proporções gigantescas, nem que as solicitações às bases de dados fossem da mesma ordem. Tudo isto veio a acontecer com a difusão da internet que veio ligar todas as pessoas de qualquer parte do mundo que tivessem um computador. Com o número de adesões à internet a crescer, o número de sites que nela eram criados também cresceu (e ainda cresce exponencialmente). Os motores de busca que antigamente indexavam alguns sites por dia, atualmente indexam uns milhões de sites por segundo e, mais recentemente as redes sociais também estão a lidar com quantidades gigantescas de informação. Tanto os motores de busca como as redes sociais chegaram à conclusão que uma base de dados relacional não chega para gerir a enorme quantidade de informação que ambos produzem e como tal, foi necessário encontrar uma solução. Essa solução é NoSQL e é o assunto que esta tese vai tratar. O presente documento visa definir e apresentar o problema que as bases de dados relacionais têm quando lidam com grandes volumes de dados, introduzir os limites do modelo relacional que só até há bem pouco tempo começaram a ser evidenciados com o surgimento de movimentos, como o BigData, com o crescente número de sites que surgem por dia e com o elevado número de utilizadores das redes sociais. Será também ilustrada a solução adotada até ao momento pelos grandes consumidores de dados de elevado volume, como o Google e o Facebook, enunciando as suas características vantagens, desvantagens e os demais conceitos ligados ao modelo NoSQL. A presente tese tenciona ainda demonstrar que o modelo NoSQL é uma realidade usada em algumas empresas e quais as principias mudanças a nível programático e as boas práticas delas resultantes que o modelo NoSQL traz. Por fim esta tese termina com a explicação de que NoSQL é uma forma de implementar a persistência de uma aplicação que se inclui no novo modelo de persistência da informação.

Electricity markets modeling considering complex contracts and aggregators

All over the world, the liberalization of electricity markets, which follows different paradigms, has created new challenges for those involved in this sector. In order to respond to these challenges, electric power systems suffered a significant restructuring in its mode of operation and planning. This restructuring resulted in a considerable increase of the electric sector competitiveness. Particularly, the Ancillary Services (AS) market has been target of constant renovations in its operation mode as it is a targeted market for the trading of services, which have as main objective to ensure the operation of electric power systems with appropriate levels of stability, safety, quality, equity and competitiveness. In this way, with the increasing penetration of distributed energy resources including distributed generation, demand response, storage units and electric vehicles, it is essential to develop new smarter and hierarchical methods of operation of electric power systems. As these resources are mostly connected to the distribution network, it is important to consider the introduction of this kind of resources in AS delivery in order to achieve greater reliability and cost efficiency of electrical power systems operation. The main contribution of this work is the design and development of mechanisms and methodologies of AS market and for energy and AS joint market, considering different management entities of transmission and distribution networks. Several models developed in this work consider the most common AS in the liberalized market environment: Regulation Down; Regulation Up; Spinning Reserve and Non-Spinning Reserve. The presented models consider different rules and ways of operation, such as the division of market by network areas, which allows the congestion management of interconnections between areas; or the ancillary service cascading process, which allows the replacement of AS of superior quality by lower quality of AS, ensuring a better economic performance of the market. A major contribution of this work is the development an innovative methodology of market clearing process to be used in the energy and AS joint market, able to ensure viable and feasible solutions in markets, where there are technical constraints in the transmission network involving its division into areas or regions. The proposed method is based on the determination of Bialek topological factors and considers the contribution of the dispatch for all services of increase of generation (energy, Regulation Up, Spinning and Non-Spinning reserves) in network congestion. The use of Bialek factors in each iteration of the proposed methodology allows limiting the bids in the market while ensuring that the solution is feasible in any context of system operation. Another important contribution of this work is the model of the contribution of distributed energy resources in the ancillary services. In this way, a Virtual Power Player (VPP) is considered in order to aggregate, manage and interact with distributed energy resources. The VPP manages all the agents aggregated, being able to supply AS to the system operator, with the main purpose of participation in electricity market. In order to ensure their participation in the AS, the VPP should have a set of contracts with the agents that include a set of diversified and adapted rules to each kind of distributed resource. All methodologies developed and implemented in this work have been integrated into the MASCEM simulator, which is a simulator based on a multi-agent system that allows to study complex operation of electricity markets. In this way, the developed methodologies allow the simulator to cover more operation contexts of the present and future of the electricity market. In this way, this dissertation offers a huge contribution to the AS market simulation, based on models and mechanisms currently used in several real markets, as well as the introduction of innovative methodologies of market clearing process on the energy and AS joint market. This dissertation presents five case studies; each one consists of multiple scenarios. The first case study illustrates the application of AS market simulation considering several bids of market players. The energy and ancillary services joint market simulation is exposed in the second case study. In the third case study it is developed a comparison between the simulation of the joint market methodology, in which the player bids to the ancillary services is considered by network areas and a reference methodology. The fourth case study presents the simulation of joint market methodology based on Bialek topological distribution factors applied to transmission network with 7 buses managed by a TSO. The last case study presents a joint market model simulation which considers the aggregation of small players to a VPP, as well as complex contracts related to these entities. The case study comprises a distribution network with 33 buses managed by VPP, which comprises several kinds of distributed resources, such as photovoltaic, CHP, fuel cells, wind turbines, biomass, small hydro, municipal solid waste, demand response, and storage units.

Estimação de Modelos de Adopção e Difusão de Tecnologias de Informação e Comunicação em Portugal

Dissertação apresentada como requisito parcial para obtenção do grau de Mestre em Estatística e Gestão de Informação