Clustering de dados biomédicos com algoritmos baseados em critérios entrópicos

A procura de padrões nos dados de modo a formar grupos é conhecida como aglomeração de dados ou clustering, sendo uma das tarefas mais realizadas em mineração de dados e reconhecimento de padrões. Nesta dissertação é abordado o conceito de entropia e são usados algoritmos com critérios entrópicos para fazer clustering em dados biomédicos. O uso da entropia para efetuar clustering é relativamente recente e surge numa tentativa da utilização da capacidade que a entropia possui de extrair da distribuição dos dados informação de ordem superior, para usá-la como o critério na formação de grupos (clusters) ou então para complementar/melhorar algoritmos existentes, numa busca de obtenção de melhores resultados. Alguns trabalhos envolvendo o uso de algoritmos baseados em critérios entrópicos demonstraram resultados positivos na análise de dados reais. Neste trabalho, exploraram-se alguns algoritmos baseados em critérios entrópicos e a sua aplicabilidade a dados biomédicos, numa tentativa de avaliar a adequação destes algoritmos a este tipo de dados. Os resultados dos algoritmos testados são comparados com os obtidos por outros algoritmos mais “convencionais" como o k-médias, os algoritmos de spectral clustering e um algoritmo baseado em densidade.

Probabilistic consensus clustering using evidence accumulation

Clustering ensemble methods produce a consensus partition of a set of data points by combining the results of a collection of base clustering algorithms. In the evidence accumulation clustering (EAC) paradigm, the clustering ensemble is transformed into a pairwise co-association matrix, thus avoiding the label correspondence problem, which is intrinsic to other clustering ensemble schemes. In this paper, we propose a consensus clustering approach based on the EAC paradigm, which is not limited to crisp partitions and fully exploits the nature of the co-association matrix. Our solution determines probabilistic assignments of data points to clusters by minimizing a Bregman divergence between the observed co-association frequencies and the corresponding co-occurrence probabilities expressed as functions of the unknown assignments. We additionally propose an optimization algorithm to find a solution under any double-convex Bregman divergence. Experiments on both synthetic and real benchmark data show the effectiveness of the proposed approach.

Enhancing the selection of a model-based clustering with external categorical variables

In cluster analysis, it can be useful to interpret the partition built from the data in the light of external categorical variables which are not directly involved to cluster the data. An approach is proposed in the model-based clustering context to select a number of clusters which both fits the data well and takes advantage of the potential illustrative ability of the external variables. This approach makes use of the integrated joint likelihood of the data and the partitions at hand, namely the model-based partition and the partitions associated to the external variables. It is noteworthy that each mixture model is fitted by the maximum likelihood methodology to the data, excluding the external variables which are used to select a relevant mixture model only. Numerical experiments illustrate the promising behaviour of the derived criterion.

A MAP approach to evidence accumulation clustering

The Evidence Accumulation Clustering (EAC) paradigm is a clustering ensemble method which derives a consensus partition from a collection of base clusterings obtained using different algorithms. It collects from the partitions in the ensemble a set of pairwise observations about the co-occurrence of objects in a same cluster and it uses these co-occurrence statistics to derive a similarity matrix, referred to as co-association matrix. The Probabilistic Evidence Accumulation for Clustering Ensembles (PEACE) algorithm is a principled approach for the extraction of a consensus clustering from the observations encoded in the co-association matrix based on a probabilistic model for the co-association matrix parameterized by the unknown assignments of objects to clusters. In this paper we extend the PEACE algorithm by deriving a consensus solution according to a MAP approach with Dirichlet priors defined for the unknown probabilistic cluster assignments. In particular, we study the positive regularization effect of Dirichlet priors on the final consensus solution with both synthetic and real benchmark data.

Comparing clustering solutions: the use of adjusted paired indices

In the present paper we compare clustering solutions using indices of paired agreement. We propose a new method - IADJUST - to correct indices of paired agreement, excluding agreement by chance. This new method overcomes previous limitations known in the literature as it permits the correction of any index. We illustrate its use in external clustering validation, to measure the accordance between clusters and an a priori known structure. The adjusted indices are intended to provide a realistic measure of clustering performance that excludes agreement by chance with ground truth. We use simulated data sets, under a range of scenarios - considering diverse numbers of clusters, clusters overlaps and balances - to discuss the pertinence and the precision of our proposal. Precision is established based on comparisons with the analytical approach for correction specific indices that can be corrected in this way are used for this purpose. The pertinence of the proposed correction is discussed when making a detailed comparison between the performance of two classical clustering approaches, namely Expectation-Maximization (EM) and K-Means (KM) algorithms. Eight indices of paired agreement are studied and new corrected indices are obtained.

O doente com patologia múltipla em medicina geral e familiar: comorbilidade de 4 doenças crónicas

RESUMO: A tese de doutoramento visa demonstrar duas proposições: a comorbilidade de 4 situações de doença prevalentes, hipertensão arterial (HTA), diabetes (DM), doença cardíaca isquémica (DCI) e asma é um assunto importante em Medicina Geral e Familiar e o seu estudo tem diversas implicações na forma como os cuidados de saúde são prestados, na sua organização e no ensino-aprendizagem da disciplina. O documento encontra-se dividido em 4 partes: 1) justificação do interesse do tema e finalidades da dissertação; 2) revisão sistemática de literatura publicada entre 1992 e 2002; 3) apresentação de dois trabalhos de investigação, descritivos e exploratórios que se debruçam sobre a mesma população de estudo, o primeiro intitulado “Comorbilidade de quatro doenças crónicas e sua relação com factores sócio demográficos” e o segundo, “Diferenças entre doentes, por médico e por sub-região, na comorbilidade de 4 doenças crónicas”; 4) conclusões e implicações dos resultados dos estudos na gestão da prática clínica, nos serviços, no ensino da disciplina da MGF e no desenvolvimento posterior de uma linha de investigação nesta área. O primeiro estudo tem como objectivos: descrever a prevalência da comorbilidade entre as 4 doenças-índice; verificar se existe relação entre o tempo da primeira doença e o tempo decorrido até ao aparecimento da 2ª e da 3ª doença, nas 4 doenças; determinar a comorbilidade associada às 4 doenças; identificar eventuais agrupamentos de doenças e verificar se existe relação entre comorbilidade e factores sociais e demográficos. O segundo estudo pretende verificar se existem diferenças na comorbilidade a nível local, por médico, e por Sub-Região de Saúde. O trabalho empírico é descritivo e exploratório. A população é constituída pelos doentes, com pelo menos uma das 4 doenças crónicas índice, das listas de utentes de 12 Médicos de Família a trabalharem em Centros de Saúde urbanos, suburbanos e rurais dos distritos de Lisboa e Beja. Os dados foram colhidos durante um ano através dos registos médicos. As variáveis sócio demográficas estudadas são: sexo, idade, etnia/raça, escolaridade, situação profissional, estado civil, tipo de família, funcionalidade familiar, condições de habitação. A comorbilidade é definida pela presença de duas ou mais doenças e estudada pelo número de doenças coexistentes. O tempo de duração da doença é definido como o número de anos decorridos entre o ano de diagnóstico e 2003. Os problemas de saúde crónicos são classificados pela ICPC2. Nas comparações efectuadas aplicaram-se os testes de Mann-Whitney e de Friedman, de homogeneidade e de análise de resíduos. A Análise Classificatória Hierárquica foi utilizada para determinar o agrupamento de doenças e a Análise de Regressão Categórica e Análise de Correspondências na relação entre as características sócio demográficas e a comorbilidade. Identificaram-se 3998 doentes. A idade média é de 64,3 anos (DP=15,70). Há uma correlação positiva significativa (r =0,350 r=0) entre “anos com a primeira doença”e “idade dos doentes” em todos os indivíduos (homens r=0,129 mulheres r=0,231). A comorbilidade entre as quatro doenças crónicas índice está presente em 1/3 da população. As associações mais prevalentes são HTA+DM (14,3%) e HTA+DCI (6,25%). Existe correlação positiva, expressiva, entre a duração da primeira doença, quando esta é a HTA ou a DM, e o intervalo de tempo até ao aparecimento da 2ª e da 3ª doenças. Identificaram-se 18 655 problemas crónicos de saúde que se traduziram em 244 códigos da ICPC2. O número médio de problemas foi de 5,94 (DP=3,04). A idade, a actividade profissional, a funcionalidade familiar e a escolaridade foram as variáveis que mais contribuíram para diferenciar os indivíduos quanto à comorbilidade. Foram encontradas diferenças significativas entre médicos(c2=1165,368 r=0) e entre os agrupamentos de doentes por Sub-Região de Saúde (c2= 157,108 r=0) no respeitante à comorbilidade. Na partição por Lisboa o número médio de problemas é de 6,45 e em Beja de 5,35. Deste trabalho ressaltam várias consequências para os profissionais, para os serviços, para o ensino e para a procura de mais saber nesta área. Os médicos, numa gestão eficiente de cuidados são chamados a desempenhar um papel de gestores da complexidade e de coordenadores assim como a trabalhar num modelo organizativo apoiado numa colaboração em equipa. Por sua vez os serviços de saúde têm que desenvolver medidas de avaliação de cuidados que integrem a comorbilidade como medida de risco. O contexto social da cronicidade e da comorbilidade deverá ser incluído como área de ensino. A concluir analisa-se o impacto do estudo nos colaboradores e o possível desenvolvimento da investigação nesta área.----------------------------------------ABSTRACT: The PhD Thesis has two propositions, co-morbidity of four chronic conditions (hypertension, asthma, diabetes, cardiac ischaemic disease) is a prevalent and complex issue and its study has several implications in the way care is provided and organised as well as in the learning and teaching of the discipline of General Practice. In the first part of the document arguments of different nature are given in order to sustain the dissertation aims; the second part describes a systematic study of literature review from 1992 to 2002; the third presents two research studies "Comorbidity of four chronic diseases and its relation with socio demographic factors” and “Differences between patients among GPs at local and regional level”; implications of study results for practice management, teaching and research are presented in the last part. The prevalence of the four chronic diseases co-morbidity, the relation of the first disease duration with the time of diagnose of the next index condition, the burden of co-morbidity in the four chronic diseases, the clustering of those diseases, the relation between demographic and social characteristics and co-morbidity, are the objectives of the first study. The second intends to verify differences in comorbidity between patients at local and regional level of practice. Research studies were descriptive and exploratory. The population under study were patients enlisted in 12 GPs working in urban and rural health centres, in Lisbon and Beja districts, with at least one of the four mentioned diseases. Data were collected through medical records during one year (2003) and 3998 patients were identified. The social demographic variables were: sex, age, ethnicity/race, education, profession, marriage status, family status, family functionality, home living conditions. Co-morbidity is defined by the presence of two or more diseases, and studied by the number of co-existing diseases. The time duration of the disease is defined by the number of years between the diagnostic year and 2003. The chronic disease problems are classified in accord with ICPC2. The characterization of population is descriptive. The effected comparisons applied the Mann-Whitney, Friedman, homogeneity and analysis of residuals tests. The Classificatory Hierarchy Analysis was utilized to determine the grouping of diseases and the Regression Categorization and Correspondences Analysis was used to study the relation of socio-demographic and co-morbidity. The median age of the population under study is 64,3 (SD= 15,70). There is a significant positive correlation (r =0,350 r=0)between “years with the first disease” and “patient age” for all individuals (men r=0,129 women r=0,231). Co-morbidity of the four index diseases is present in 1/3 of the studied population. The most prevalent associations for the four diseases are HTA+DM (14,03%) and HTA+IHD (6,25%). Expressive positive correlation between the duration of the first disease and the second and the third index disease interval is found. For the 3988 patients, 18 655 chronic health problems, translated in 244 ICPC2 codes, were identified. The mean number of problems is 5,94 (SD=3,04). Age, professional activity, family functionality and education level are the socio demographic characteristics that most contribute to differentiate individuals concerning the overall co-morbidity. Significant differences in co-morbidity between GP patients at local (c2=1165,368 r=0) and regional level (c2= 157,108 r=0) are found. This study has several consequences for professionals, for services, for the teaching and learning of General Practice and for the pursuit of knowledge in this area. New competences and performances have to be implemented. General Practitioners, assuming a role of co-ordination, have to perform the role of complexity managers in patient's care, working in practices supported by a strong team in collaboration with other specialists. In order to assess provided care, services have to develop tools where co-morbidity is included as a risk measure. The social context of comorbidity and chronicity has to be included in the curricula of General Practice learning and teaching areas. The dissertation ends describing the added value to participant's performance for their participation in the research and an agenda for further research, in this area, based on a community of practice.--------RÉSUMÉ:Cette thèse de doctorat prétend démontrer deux postulats : le premier, que la comorbidité de quatre maladies fréquentes, hypertension artérielle (HTA), diabète (DM), maladie cardiaque ischémique (DCI) et asthme, est un thème important en Médecine Générale et Familiale et que son étude a plusieurs implications au niveau de l'approche pour dispenser les soins, de leur organisation et de l'enseignement/apprentissage de la discipline. Le document comprend quatre parties distinctes : 1) justification de l'intérêt du sujet et objectifs de la dissertation ; 2) étude systématique de publications éditées entre 1992 et 2002 ; 3) présentation de deux travaux de recherche, descriptifs et exploratoires, un premier intitulée « Comorbidité de quatre maladies chroniques et leur relation avec des facteurs sociodémographiques » et un deuxième « Différences entre malades, selon le médecin et la sous région, dans la comorbilité de quatre maladies chroniques» ; 4) conclusions et conséquences des résultats des études dans la gestion de la pratique clinique, dans les services, dans l'enseignement de la discipline de MGF et dans le développement postérieur de la recherche dans ce domaine. Les objectifs de la première étude sont les suivants : décrire la prévalence de la comorbidité entre les quatre maladies chroniques, vérifier s'il existe une relation entre temps de durée de la première maladie et l'espace de temps jusqu'à le diagnostic de la 2ème ou 3ème maladie; déterminer la comorbidité entre les 4 maladies ; identifier d'éventuelles groupements de maladies et vérifier s'il existe une relation entre comorbidité et facteurs sociodémographiques. La deuxième étude prétend vérifier s'il existe des différences de comorbidité entre médecins et par groupement régional. Le travail empirique est descriptif et exploratoire. La population est composée des malades ayant au moins une des quatre maladies chroniques parmi les listes de malades de douze Médecins de Famille qui travaillent dans des Centres de Santé urbains, suburbains et ruraux (Districts de Lisbonne et Beja). Les données ont été extraites pendant l'année 2003 des registres des médecins. Les variables sociodémographiques étudiées sont : le sexe, l'âge, l'ethnie/race, la scolarité, la situation professionnelle, l'état civil, le type de famille, sa fonctionnalité, les conditions de logement. La comorbidité est définie lorsqu'il existe deux ou plusieurs maladies et est étudiée d'après le nombre de maladies coexistantes. La durée de la maladie est établie en comptant le nombre d'années écoulées entre le diagnostique et 2003. Les problèmes de santé chroniques sont classés par l'ICPC 2. Pour les comparaisons les tests de Mann-Whitney et Friedman, de homogénéité et analyse de résidues ont été appliqués. L'Analyse de Classification Hiérarchique a été utilisée pour procéder au regroupement des maladies et l'Analyse de Régression Catégorique et l'Analyse de Correspondances pour étudier la relation entre les caractéristiques sociodémographiques et la comorbilité. Les principaux résultats sont les suivants : les 3998 malades identifiés ont 64,3 ans d'âge moyen (DP=15,70). Il existe une corrélation positive significative (r =0,350 r=0) entre « les années avec la première maladie » et « l'âge des malades », chez tous les individus (hommes r=0,129 femmes r=0,231). La comorbidité entre les quatre maladies chroniques est une réalité chez 1/3 des patients. Les associations les plus fréquentes sont HTA+DM (14%) et HTA+DCI (6,25%). Il existe une corrélation positive significative entre la durée de la première maladie, HTA ou DM, et l'écart jusqu'à l'apparition de la deuxième et de la troisième maladie. Chez les malades, 18.655 problèmes chroniques de santé ont été identifiés et traduits en 244 codes de l'ICPC2. La moyenne des problèmes a été de 5,94 (DP=3,04). L'âge, l'activité professionnelle, la fonctionnalité familiale et la scolarité sont les variables qui ont le plus contribué à différencier les individus face à la comorbilité. Des différences notoires ont été trouvées entre médecins (c2=1165,368 r=0) et entre les groupements régionaux (c2=157,108 r=0) en ce qui concerne la comorbidité. Dans le groupe de patients de Lisbonne, le chiffre moyen de problèmes est de 6,45 et à Beja il est de 5,35. Cette étude met en évidence plusieurs conséquences pour les professionnels, les services, l'enseignement et l'élargissement du savoir dans ce domaine. Les médecins, soucieux de gérer efficacement les soins sont appelés à jouer un rôle de gestionnaires de la complexité et de coordinateurs, de même qu'à travailler dans un modèle d'organisation soutenus par un travail d'équipe. D'autre part, les services de santé doivent eux aussi développer des mesures d'évaluation des soins qui intègrent la comorbidité comme mesure de risque. Le contexte social de la chronicité et de la comorbidité devra être inclus comme domaines à étudier. La fin de cette thèse décrit l'impact de cette étude sur les collaborateurs et le développement futur de la recherche dans ce domaine.

A study on parallel versus sequential relational fuzzy clustering methods

Dissertação para obtenção do Grau de Mestre em Engenharia Informática

Algorithms for time series clustering applied to biomedical signals

Thesis submitted in the fulfillment of the requirements for the Degree of Master in Biomedical Engineering

Intake of milk, but not total dairy, yogurt, or cheese, is negatively associated with the clustering of cardiometabolic risk factors in adolescents

Epidemiologic studies have reported an inverse association between dairy product consumption and cardiometabolic risk factors in adults, but this relation is relatively unexplored in adolescents. We hypothesized that a higher dairy product intake is associated with lower cardiometabolic risk factor clustering in adolescents. To test this hypothesis, a cross-sectional study was conducted with 494 adolescents aged 15 to 18 years from the Azorean Archipelago, Portugal. We measured fasting glucose, insulin, total cholesterol, high-density lipoprotein cholesterol, triglycerides, systolic blood pressure, body fat, and cardiorespiratory fitness. We also calculated homeostatic model assessment and total cholesterol/high-density lipoprotein cholesterol ratio. For each one of these variables, a z score was computed using age and sex. A cardiometabolic risk score (CMRS) was constructed by summing up the z scores of all individual risk factors. High risk was considered to exist when an individual had at least 1 SD from this score. Diet was evaluated using a food frequency questionnaire, and the intake of total dairy (included milk, yogurt, and cheese), milk, yogurt, and cheese was categorized as low (equal to or below the median of the total sample) or “appropriate” (above the median of the total sample).The association between dairy product intake and CMRS was evaluated using separate logistic regression, and the results were adjusted for confounders. Adolescents with high milk intake had lower CMRS, compared with those with low intake (10.6% vs 18.1%, P = .018). Adolescents with appropriate milk intake were less likely to have high CMRS than those with low milk intake (odds ratio, 0.531; 95% confidence interval, 0.302-0.931). No association was found between CMRS and total dairy, yogurt, and cheese intake. Only milk intake seems to be inversely related to CMRS in adolescents.

An intelligent clustering scheme for distributed intrusion detection in vehicular cloud computing

In recent years, vehicular cloud computing (VCC) has emerged as a new technology which is being used in wide range of applications in the area of multimedia-based healthcare applications. In VCC, vehicles act as the intelligent machines which can be used to collect and transfer the healthcare data to the local, or global sites for storage, and computation purposes, as vehicles are having comparatively limited storage and computation power for handling the multimedia files. However, due to the dynamic changes in topology, and lack of centralized monitoring points, this information can be altered, or misused. These security breaches can result in disastrous consequences such as-loss of life or financial frauds. Therefore, to address these issues, a learning automata-assisted distributive intrusion detection system is designed based on clustering. Although there exist a number of applications where the proposed scheme can be applied but, we have taken multimedia-based healthcare application for illustration of the proposed scheme. In the proposed scheme, learning automata (LA) are assumed to be stationed on the vehicles which take clustering decisions intelligently and select one of the members of the group as a cluster-head. The cluster-heads then assist in efficient storage and dissemination of information through a cloud-based infrastructure. To secure the proposed scheme from malicious activities, standard cryptographic technique is used in which the auotmaton learns from the environment and takes adaptive decisions for identification of any malicious activity in the network. A reward and penalty is given by the stochastic environment where an automaton performs its actions so that it updates its action probability vector after getting the reinforcement signal from the environment. The proposed scheme was evaluated using extensive simulations on ns-2 with SUMO. The results obtained indicate that the proposed scheme yields an improvement of 10 % in detection rate of malicious nodes when compared with the existing schemes.

Clustering aplicado à Bolsa de Valores de Lisboa

O objetivo desta dissertação foi estudar um conjunto de empresas cotadas na bolsa de valores de Lisboa, para identificar aquelas que têm um comportamento semelhante ao longo do tempo. Para isso utilizamos algoritmos de Clustering tais como K-Means, PAM, Modelos hierárquicos, Funny e C-Means tanto com a distância euclidiana como com a distância de Manhattan. Para selecionar o melhor número de clusters identificado por cada um dos algoritmos testados, recorremos a alguns índices de avaliação/validação de clusters como o Davies Bouldin e Calinski-Harabasz entre outros.

Algorithms for information extraction and signal annotation on long-term biosignals using clustering techniques

Dissertação para obtenção do Grau de Mestre em Engenharia Biomédica

Planeamento de Trajetória para Operações de Busca e Salvamento com UAVs

Os sistemas autónomos trazem como mais valia aos cenários de busca e salvamento a possibilidade de minimizar a presença de Humanos em situações de perigo e a capacidade de aceder a locais de difícil acesso. Na dissertação propõe-se endereçar novos métodos para perceção e navegação de veículos aéreos não tripulados (UAV), tendo como foco principal o planeamento de trajetórias e deteção de obstáculos. No que respeita à perceção foi desenvolvido um método para gerar clusters tendo por base os voxels gerados pelo Octomap. Na área de navegação, foram desenvolvidos dois novos métodos de planeamento de trajetórias, GPRM (Grid Probabilistic Roadmap) e PPRM (Particle Probabilistic Roadmap), que tem como método base para o seu desenvolvimento o PRM. O primeiro método desenvolvido, GPRM, espalha as partículas numa grid pré-definida, construindo posteriormente o roadmap na área determinada pela grid e com isto estima o trajeto mais curto até ao ponto destino. O segundo método desenvolvido, PPRM, espalha as partículas pelo cenário de aplicação, gera o roadmap considerando o mapa total e atribui uma probabilidade que irá permitir definir a trajetória otimizada. Para analisar a performance de cada método em comparação com o PRM, efetua-se a sua avaliação em três cenários distintos com recurso ao simulador MORSE.

Clustering medical tourism in Algarve: setting the first steps for the the health authority

A Work Project, presented as part of the requirements for the Award of a Masters Degree in Management from the NOVA – School of Business and Economics

A clustering approach for vehicle routing problems with hard time windows

Dissertação para obtenção do Grau de Mestre em Logica Computicional