Towards a model-driven engineering approach of data mining

Nowadays, data mining is based on low-level specications of the employed techniques typically bounded to a specic analysis platform. Therefore, data mining lacks a modelling architecture that allows analysts to consider it as a truly software-engineering process. Here, we propose a model-driven approach based on (i) a conceptual modelling framework for data mining, and (ii) a set of model transformations to automatically generate both the data under analysis (via data-warehousing technology) and the analysis models for data mining (tailored to a specic platform). Thus, analysts can concentrate on the analysis problem via conceptual data-mining models instead of low-level programming tasks related to the underlying-platform technical details. These tasks are now entrusted to the model-transformations scaffolding.

Integrating the development of data mining and data warehouses via model-driven engineering

Data mining is one of the most important analysis techniques to automatically extract knowledge from large amount of data. Nowadays, data mining is based on low-level specifications of the employed techniques typically bounded to a specific analysis platform. Therefore, data mining lacks a modelling architecture that allows analysts to consider it as a truly software-engineering process. Bearing in mind this situation, we propose a model-driven approach which is based on (i) a conceptual modelling framework for data mining, and (ii) a set of model transformations to automatically generate both the data under analysis (that is deployed via data-warehousing technology) and the analysis models for data mining (tailored to a specific platform). Thus, analysts can concentrate on understanding the analysis problem via conceptual data-mining models instead of wasting efforts on low-level programming tasks related to the underlying-platform technical details. These time consuming tasks are now entrusted to the model-transformations scaffolding. The feasibility of our approach is shown by means of a hypothetical data-mining scenario where a time series analysis is required.

Open business intelligence: on the importance of data quality awareness in user-friendly data mining

Citizens demand more and more data for making decisions in their daily life. Therefore, mechanisms that allow citizens to understand and analyze linked open data (LOD) in a user-friendly manner are highly required. To this aim, the concept of Open Business Intelligence (OpenBI) is introduced in this position paper. OpenBI facilitates non-expert users to (i) analyze and visualize LOD, thus generating actionable information by means of reporting, OLAP analysis, dashboards or data mining; and to (ii) share the new acquired information as LOD to be reused by anyone. One of the most challenging issues of OpenBI is related to data mining, since non-experts (as citizens) need guidance during preprocessing and application of mining algorithms due to the complexity of the mining process and the low quality of the data sources. This is even worst when dealing with LOD, not only because of the different kind of links among data, but also because of its high dimensionality. As a consequence, in this position paper we advocate that data mining for OpenBI requires data quality-aware mechanisms for guiding non-expert users in obtaining and sharing the most reliable knowledge from the available LOD.

Towards a reverse engineering approach for guiding user in applying data mining

Comunicación presentada en las XVI Jornadas de Ingeniería del Software y Bases de Datos, JISBD 2011, A Coruña, 5-7 septiembre 2011.

La comunicación de la innovación de las agencias de medios. Big data en las webs propias

Las agencias de medios constituyen uno de los sectores publicitarios en el que la aplicación de big data proporciona soluciones innovadoras que debieran ser comunicadas a través, entre otros canales, de sus webs corporativas. Con el propósito de facilitar tal comunicación, las webs deben estar preparadas para ser consultadas por varios dispositivos. Así, este estudio pretende averiguar su uso para comunicar innovaciones relacionadas con big data, cuidando el responsive design, de las 20 primeras agencias de medios españolas, y reflexiona sobre las debilidades, amenazas, fortalezas y oportunidades de tal innovación.

Novas tendências, o big data: deteção de fraudes e transações no comércio local: SAF-T com tecnologia big data

Dissertação apresentada à Escola Superior de Tecnologia do Instituto Politécnico de Castelo Branco para cumprimento dos requisitos necessários à obtenção do grau de Mestre em Desenvolvimento de Software e Sistemas Interactivos, realizada sob a orientação científica da categoria profissional do orientador Doutor Eurico Ribeiro Lopes, do Instituto Politécnico de Castelo Branco.

Ferramentas open source de Data Mining

Em época de crise financeira, as ferramentas open source de data mining representam uma nova tendência na investigação, educação e nas aplicações industriais, especialmente para as pequenas e médias empresas. Com o software open source, estas podem facilmente iniciar um projeto de data mining usando as tecnologias mais recentes, sem se preocuparem com os custos de aquisição das mesmas, podendo apostar na aprendizagem dos seus colaboradores. Os sistemas open source proporcionam o acesso ao código, facilitando aos colaboradores a compreensão dos sistemas e algoritmos e permitindo que estes o adaptem às necessidades dos seus projetos. No entanto, existem algumas questões inerentes ao uso deste tipo de ferramenta. Uma das mais importantes é a diversidade, e descobrir, tardiamente, que a ferramenta escolhida é inapropriada para os objetivos do nosso negócio pode ser um problema grave. Como o número de ferramentas de data mining continua a crescer, a escolha sobre aquela que é realmente mais apropriada ao nosso negócio torna-se cada vez mais difícil. O presente estudo aborda um conjunto de ferramentas de data mining, de acordo com as suas características e funcionalidades. As ferramentas abordadas provém da listagem do KDnuggets referente a Software Suites de Data Mining. Posteriormente, são identificadas as que reúnem melhores condições de trabalho, que por sua vez são as mais populares nas comunidades, e é feito um teste prático com datasets reais. Os testes pretendem identificar como reagem as ferramentas a cenários diferentes do tipo: performance no processamento de grandes volumes de dados; precisão de resultados; etc. Nos tempos que correm, as ferramentas de data mining open source representam uma oportunidade para os seus utilizadores, principalmente para as pequenas e médias empresas, deste modo, os resultados deste estudo pretendem ajudar no processo de tomada de decisão relativamente às mesmas.

Tecnologie per l’analisi in tempo reale di Big Data: prestazioni a confronto

Lo scopo di questo l'elaborato è l'analisi,lo studio e il confronto delle tecnologie per l'analisi in tempo reale di Big Data: Apache Spark Streaming, Apache Storm e Apache Flink. Per eseguire un adeguato confronto si è deciso di realizzare un sistema di rilevamento e riconoscimento facciale all’interno di un video, in maniera da poter parallelizzare le elaborazioni necessarie sfruttando le potenzialità di ogni architettura. Dopo aver realizzato dei prototipi realistici, uno per ogni architettura, si è passati alla fase di testing per misurarne le prestazioni. Attraverso l’impiego di cluster appositamente realizzati in ambiente locale e cloud, sono state misurare le caratteristiche che rappresentavano, meglio di altre, le differenze tra le architetture, cercando di dimostrarne quantitativamente l’efficacia degli algoritmi utilizzati e l’efficienza delle stesse. Si è scelto quindi il massimo input rate sostenibile e la latenza misurate al variare del numero di nodi. In questo modo era possibile osservare la scalabilità di architettura, per analizzarne l’andamento e verificare fino a che limite si potesse giungere per mantenere un compromesso accettabile tra il numero di nodi e l’input rate sostenibile. Gli esperimenti effettuati hanno mostrato che, all’aumentare del numero di worker le prestazioni del sistema migliorano, rendendo i sistemi studiati adatti all’utilizzo su larga scala. Inoltre sono state rilevate sostanziali differenze tra i vari framework, riportando pro e contro di ognuno, cercando di evidenziarne i più idonei al caso di studio.

Strategie di ingegnerizzazione e di sviluppo di sistemi software per big data management

Il lavoro svolto si concentra sullo studio e lo sviluppo dei sistemi software per la gestione dei big data. Inizialmente sono stati analizzati i settori nei quali i big data si stanno diffondendo maggiormente per poi studiare l'ingegnerizzazione e lo sviluppo dei sistemi in grado di gestire questo tipo di dati. Sono state studiate tutte le fasi del processo di realizzazione del software e i rischi e i problemi che si possono incontrare. Infine è stato presentato un software di analisi di big data: Google BigQuery.

Maine occupational staffing for selected nonmanufacturing industries : mining, construction, finance, insurance, and real estate services, except hospitals and education : data for second quarter 1990 /

"Issued July 1992."

Big Data Development: A Tool for Interpreting Institutional Impact

Thesis (Master's)--University of Washington, 2016-06

Fuzzy logic in web data mining for website assessment

This paper proposes a novel application of fuzzy logic to web data mining for two basic problems of a website: popularity and satisfaction. Popularity means that people will visit the website while satisfaction refers to the usefulness of the site. We will illustrate that the popularity of a website is a fuzzy logic problem. It is an important characteristic of a website in order to survive in Internet commerce. The satisfaction of a website is also a fuzzy logic problem that represents the degree of success in the application of information technology to the business. We propose a framework of fuzzy logic for the representation of these two problems based on web data mining techniques to fuzzify the attributes of a website.

Electricity market price spike forecast with data mining techniques

Electricity market price forecast is a changeling yet very important task for electricity market managers and participants. Due to the complexity and uncertainties in the power grid, electricity prices are highly volatile and normally carry with spikes. which may be (ens or even hundreds of times higher than the normal price. Such electricity spikes are very difficult to be predicted. So far. most of the research on electricity price forecast is based on the normal range electricity prices. This paper proposes a data mining based electricity price forecast framework, which can predict the normal price as well as the price spikes. The normal price can be, predicted by a previously proposed wavelet and neural network based forecast model, while the spikes are forecasted based on a data mining approach. This paper focuses on the spike prediction and explores the reasons for price spikes based on the measurement of a proposed composite supply-demand balance index (SDI) and relative demand index (RDI). These indices are able to reflect the relationship among electricity demand, electricity supply and electricity reserve capacity. The proposed model is based on a mining database including market clearing price, trading hour. electricity), demand, electricity supply and reserve. Bayesian classification and similarity searching techniques are used to mine the database to find out the internal relationships between electricity price spikes and these proposed. The mining results are used to form the price spike forecast model. This proposed model is able to generate forecasted price spike, level of spike and associated forecast confidence level. The model is tested with the Queensland electricity market data with promising results. Crown Copyright (C) 2004 Published by Elsevier B.V. All rights reserved.