Development of Hierarchical Clustering Techniques for Gridded Data from Mixed Data Sequences

Knowledge discovery in databases is the non-trivial process of identifying valid, novel potentially useful and ultimately understandable patterns from data. The term Data mining refers to the process which does the exploratory analysis on the data and builds some model on the data. To infer patterns from data, data mining involves different approaches like association rule mining, classification techniques or clustering techniques. Among the many data mining techniques, clustering plays a major role, since it helps to group the related data for assessing properties and drawing conclusions. Most of the clustering algorithms act on a dataset with uniform format, since the similarity or dissimilarity between the data points is a significant factor in finding out the clusters. If a dataset consists of mixed attributes, i.e. a combination of numerical and categorical variables, a preferred approach is to convert different formats into a uniform format. The research study explores the various techniques to convert the mixed data sets to a numerical equivalent, so as to make it equipped for applying the statistical and similar algorithms. The results of clustering mixed category data after conversion to numeric data type have been demonstrated using a crime data set. The thesis also proposes an extension to the well known algorithm for handling mixed data types, to deal with data sets having only categorical data. The proposed conversion has been validated on a data set corresponding to breast cancer. Moreover, another issue with the clustering process is the visualization of output. Different geometric techniques like scatter plot, or projection plots are available, but none of the techniques display the result projecting the whole database but rather demonstrate attribute-pair wise analysis

Mining minimal non-redundant association rules using frequent closed itemsets

The problem of the relevance and the usefulness of extracted association rules is of primary importance because, in the majority of cases, real-life databases lead to several thousands association rules with high confidence and among which are many redundancies. Using the closure of the Galois connection, we define two new bases for association rules which union is a generating set for all valid association rules with support and confidence. These bases are characterized using frequent closed itemsets and their generators; they consist of the non-redundant exact and approximate association rules having minimal antecedents and maximal consequences, i.e. the most relevant association rules. Algorithms for extracting these bases are presented and results of experiments carried out on real-life databases show that the proposed bases are useful, and that their generation is not time consuming.

Towards semantic web mining

Semantic Web Mining aims at combining the two fast-developing research areas Semantic Web and Web Mining. The idea is to improve, on the one hand, the results of Web Mining by exploiting the new semantic structures in the Web; and to make use of Web Mining, on overview of where the two areas meet today, and sketches ways of how a closer integration could be profitable.

Semantic web mining

Semantic Web Mining aims at combining the two fast-developing research areas Semantic Web and Web Mining. This survey analyzes the convergence of trends from both areas: an increasing number of researchers is working on improving the results of Web Mining by exploiting semantic structures in the Web, and they make use of Web Mining techniques for building the Semantic Web. Last but not least, these techniques can be used for mining the Semantic Web itself. The Semantic Web is the second-generation WWW, enriched by machine-processable information which supports the user in his tasks. Given the enormous size even of today’s Web, it is impossible to manually enrich all of these resources. Therefore, automated schemes for learning the relevant information are increasingly being used. Web Mining aims at discovering insights about the meaning of Web resources and their usage. Given the primarily syntactical nature of the data being mined, the discovery of meaning is impossible based on these data only. Therefore, formalizations of the semantics of Web sites and navigation behavior are becoming more and more common. Furthermore, mining the Semantic Web itself is another upcoming application. We argue that the two areas Web Mining and Semantic Web need each other to fulfill their goals, but that the full potential of this convergence is not yet realized. This paper gives an overview of where the two areas meet today, and sketches ways of how a closer integration could be profitable.

Efficient mining of association rules based on formal concept analysis

Association rules are a popular knowledge discovery technique for warehouse basket analysis. They indicate which items of the warehouse are frequently bought together. The problem of association rule mining has first been stated in 1993. Five years later, several research groups discovered that this problem has a strong connection to Formal Concept Analysis (FCA). In this survey, we will first introduce some basic ideas of this connection along a specific algorithm, TITANIC, and show how FCA helps in reducing the number of resulting rules without loss of information, before giving a general overview over the history and state of the art of applying FCA for association rule mining.

Statistical Models for Co-occurrence Data

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. In this contribution we develop a statistical framework for analyzing co-occurrence data in a general setting where elementary observations are joint occurrences of pairs of abstract objects from two finite sets. The main challenge for statistical models in this context is to overcome the inherent data sparseness and to estimate the probabilities for pairs which were rarely observed or even unobserved in a given sample set. Moreover, it is often of considerable interest to extract grouping structure or to find a hierarchical data organization. A novel family of mixture models is proposed which explain the observed data by a finite number of shared aspects or clusters. This provides a common framework for statistical inference and structure discovery and also includes several recently proposed models as special cases. Adopting the maximum likelihood principle, EM algorithms are derived to fit the model parameters. We develop improved versions of EM which largely avoid overfitting problems and overcome the inherent locality of EM--based optimization. Among the broad variety of possible applications, e.g., in information retrieval, natural language processing, data mining, and computer vision, we have chosen document retrieval, the statistical analysis of noun/adjective co-occurrence and the unsupervised segmentation of textured images to test and evaluate the proposed algorithms.

Validating Co-Training Models for Web Image Classification

Co-training is a semi-supervised learning method that is designed to take advantage of the redundancy that is present when the object to be identified has multiple descriptions. Co-training is known to work well when the multiple descriptions are conditional independent given the class of the object. The presence of multiple descriptions of objects in the form of text, images, audio and video in multimedia applications appears to provide redundancy in the form that may be suitable for co-training. In this paper, we investigate the suitability of utilizing text and image data from the Web for co-training. We perform measurements to find indications of conditional independence in the texts and images obtained from the Web. Our measurements suggest that conditional independence is likely to be present in the data. Our experiments, within a relevance feedback framework to test whether a method that exploits the conditional independence outperforms methods that do not, also indicate that better performance can indeed be obtained by designing algorithms that exploit this form of the redundancy when it is present.

Informed recommender: basing recommendations on consumer product reviews

Recommender systems attempt to predict items in which a user might be interested, given some information about the user's and items' profiles. Most existing recommender systems use content-based or collaborative filtering methods or hybrid methods that combine both techniques (see the sidebar for more details). We created Informed Recommender to address the problem of using consumer opinion about products, expressed online in free-form text, to generate product recommendations. Informed recommender uses prioritized consumer product reviews to make recommendations. Using text-mining techniques, it maps each piece of each review comment automatically into an ontology

Recommender System Based on Consumer Product Reviews

Consumer reviews, opinions and shared experiences in the use of a product is a powerful source of information about consumer preferences that can be used in recommender systems. Despite the importance and value of such information, there is no comprehensive mechanism that formalizes the opinions selection and retrieval process and the utilization of retrieved opinions due to the difficulty of extracting information from text data. In this paper, a new recommender system that is built on consumer product reviews is proposed. A prioritizing mechanism is developed for the system. The proposed approach is illustrated using the case study of a recommender system for digital cameras

Visual management of sags and incidents gathered in distribution substations for power quality management

Monitor a distribution network implies working with a huge amount of data coining from the different elements that interact in the network. This paper presents a visualization tool that simplifies the task of searching the database for useful information applicable to fault management or preventive maintenance of the network

Medidas de intervención para promoción de conductas saludables y prevención de riesgos laborales relacionados con accidentalidad y enfermedad laboral en trabajadores de minas de Carbón

Colombia en su legislación normatiza el sector de la minería de carbón, sin embargo se considera que las estrategias no han sido suficientes para la identificación, prevención y control de la accidentalidad y enfermedad laboral. Durante el año 2013 el índice de fatalidad fue de 1,59. Estadísticas del año 2004 evidencian que las neumoconiosis fueron las mayores causas de invalidez de origen profesional. Objetivo: Categorizar actividades de intervención en promoción y prevención de accidentalidad y enfermedad laboral en trabajadores de la minería de carbón. Metodología: Se realizó una revisión de literatura sobre minería de carbón y salud la cual fue obtenida de las bases de datos PUBMED, Sciendirect, VHL, SINAB por literatura publicada sin límites de año, en idioma inglés, español o portugués. Para la búsqueda se utilizaron términos en lenguaje controlado (términos MESH), revisión por pares de títulos y resúmenes. Las publicaciones fueron seleccionadas para revisión de texto completo bajo criterios de inclusión y exclusión. Los códigos contemplados para esta revisión fueron: a) país donde la intervención se llevó a cabo, b) salud ocupacional, c) prevención de accidentalidad, d) programas de promoción, e) tecnologías, f) resultados obtenidos. Resultados: Del total de 2500 artículos seleccionados por los autores principales se realizó la revisión de los primeros 300 artículos, 32 hacen referencia al tema de salud ocupacional y minería de carbón, 10 contienen intervenciones consideradas de relevancia para esta revisión bibliográfica. Se presentan intervenciones estadísticamente significativas (p<0.05) y que han demostrado ser de impacto positivo en la minería de carbón en promoción y prevención de accidentalidad y enfermedad ocupacional. Conclusiones: Se identificaron las siguientes cuatro tipos de intervención: 1) las de carácter educativo que hacen referencia a las capacitaciones participativas, el entrenamiento por medio de “degraded image”, la realización de gestión de autocontrol y retroalimentación para el uso de elementos de protección personal (EPP), la aplicación del Modelo de Proceso Paralelo Extendido; 2) intervenciones preventivas como la medición de alcoholimetría antes del turno, la presencia de personal de enfermería en minas de carbón y el reconocimiento de los predictores de la enfermedad para optimizar la prevención primaria; 3)intervenciones de vigilancia como la promovida en la metodología Estadísticas Europeas de Accidentes de Trabajo (EEAT) para la investigación de los accidentes de trabajo, la aplicación de las recomendaciones de la Organización Mundial de la Salud (OMS) para la detección de la neumoconiosis y 4) De carácter tecnológico consistente en la intervención de tareas a partir de los resultados de la aplicación del software desarrollado por el Instituto Nacional para la Seguridad y Salud Ocupacional (NIOSH). Estas intervenciones han demostrado ser eficaces en la promoción y prevención de accidentalidad y enfermedad ocupacional por lo cual se recomienda su aplicación en Colombia posterior al análisis de costo-efectividad.

Reading hybrid texts : remarks on text/image transitions

The reading of printed materials implies the visual processing of information originated in two distinct semiotic systems. The rapid identification of redundancy, complementation or contradiction rhetoric strategies between the two information types may be crucial for an adequate interpretation of bimodal materials. Hybrid texts (verbal and visual) are particular instances of bimodal materials, where the redundant information is often neglected while the complementary and the contradictory ones are essential.Studies using the 504 ASL eye-tracking system while reading either additive or exhibiting captions (Baptista, 2009) revealed fixations on the verbal material and transitions between the written and the pictorial in a much higher number and duration than the initially foreseen as necessary to read the verbal text. We therefore hypothesized that confirmation strategies of the written information are taking place, by using information available in the other semiotic system.Such eye-gaze patterns obtained from denotative texts and pictures seem to contradict some of the scarce existing data on visual processing of texts and images, namely cartoons (Carroll, Young and Guertain, 1992), descriptive captions (Hegarty, 1992 a and b), and advertising images with descriptive and explanatory texts (cf. Rayner and Rotello, 2001, who refer to a previous reading of the whole text before looking at the image, or even Rayner, Miller and Rotello, 2008 who refer to an earlier and longer look at the picture) and seem to consolidate findings of Radach et al. (2003) on systematic transitions between text and image.By framing interest areas in the printed pictorial material of non redundant hybrid texts, we have identified the specific areas where transitions take place after fixations in the verbal text. The way those transitions are processed brings a new interest to further research.

Data assimilation for morphodynamic prediction and predictability

This paper gives an overview of the project Changing Coastlines: data assimilation for morphodynamic prediction and predictability. This project is investigating whether data assimilation could be used to improve coastal morphodynamic modeling. The concept of data assimilation is described, and the benefits that data assimilation could bring to coastal morphodynamic modeling are discussed. Application of data assimilation in a simple 1D morphodynamic model is presented. This shows that data assimilation can be used to improve the current state of the model bathymetry, and to tune the model parameter. We now intend to implement these ideas in a 2D morphodynamic model, for two study sites. The logistics of this are considered, including model design and implementation, and data requirement issues. We envisage that this work could provide a means for maintaining up-to-date information on coastal bathymetry, without the need for costly survey campaigns. This would be useful for a range of coastal management issues, including coastal flood forecasting.