New kernel functions and learning methods for text and data mining

Recent advances in machine learning methods enable increasingly the automatic construction of various types of computer assisted methods that have been difficult or laborious to program by human experts. The tasks for which this kind of tools are needed arise in many areas, here especially in the fields of bioinformatics and natural language processing. The machine learning methods may not work satisfactorily if they are not appropriately tailored to the task in question. However, their learning performance can often be improved by taking advantage of deeper insight of the application domain or the learning problem at hand. This thesis considers developing kernel-based learning algorithms incorporating this kind of prior knowledge of the task in question in an advantageous way. Moreover, computationally efficient algorithms for training the learning machines for specific tasks are presented. In the context of kernel-based learning methods, the incorporation of prior knowledge is often done by designing appropriate kernel functions. Another well-known way is to develop cost functions that fit to the task under consideration. For disambiguation tasks in natural language, we develop kernel functions that take account of the positional information and the mutual similarities of words. It is shown that the use of this information significantly improves the disambiguation performance of the learning machine. Further, we design a new cost function that is better suitable for the task of information retrieval and for more general ranking problems than the cost functions designed for regression and classification. We also consider other applications of the kernel-based learning algorithms such as text categorization, and pattern recognition in differential display. We develop computationally efficient algorithms for training the considered learning machines with the proposed kernel functions. We also design a fast cross-validation algorithm for regularized least-squares type of learning algorithm. Further, an efficient version of the regularized least-squares algorithm that can be used together with the new cost function for preference learning and ranking tasks is proposed. In summary, we demonstrate that the incorporation of prior knowledge is possible and beneficial, and novel advanced kernels and cost functions can be used in algorithms efficiently.

Evaluating multidimensional visualization techniques in data mining tasks

Visual data mining (VDM) tools employ information visualization techniques in order to represent large amounts of high-dimensional data graphically and to involve the user in exploring data at different levels of detail. The users are looking for outliers, patterns and models – in the form of clusters, classes, trends, and relationships – in different categories of data, i.e., financial, business information, etc. The focus of this thesis is the evaluation of multidimensional visualization techniques, especially from the business user’s perspective. We address three research problems. The first problem is the evaluation of projection-based visualizations with respect to their effectiveness in preserving the original distances between data points and the clustering structure of the data. In this respect, we propose the use of existing clustering validity measures. We illustrate their usefulness in evaluating five visualization techniques: Principal Components Analysis (PCA), Sammon’s Mapping, Self-Organizing Map (SOM), Radial Coordinate Visualization and Star Coordinates. The second problem is concerned with evaluating different visualization techniques as to their effectiveness in visual data mining of business data. For this purpose, we propose an inquiry evaluation technique and conduct the evaluation of nine visualization techniques. The visualizations under evaluation are Multiple Line Graphs, Permutation Matrix, Survey Plot, Scatter Plot Matrix, Parallel Coordinates, Treemap, PCA, Sammon’s Mapping and the SOM. The third problem is the evaluation of quality of use of VDM tools. We provide a conceptual framework for evaluating the quality of use of VDM tools and apply it to the evaluation of the SOM. In the evaluation, we use an inquiry technique for which we developed a questionnaire based on the proposed framework. The contributions of the thesis consist of three new evaluation techniques and the results obtained by applying these evaluation techniques. The thesis provides a systematic approach to evaluation of various visualization techniques. In this respect, first, we performed and described the evaluations in a systematic way, highlighting the evaluation activities, and their inputs and outputs. Secondly, we integrated the evaluation studies in the broad framework of usability evaluation. The results of the evaluations are intended to help developers and researchers of visualization systems to select appropriate visualization techniques in specific situations. The results of the evaluations also contribute to the understanding of the strengths and limitations of the visualization techniques evaluated and further to the improvement of these techniques.

Heat Demand Forecasting Models’ Development: Use of Data Mining Tools in SQL Server Analysis Services

This thesis introduces heat demand forecasting models which are generated by using data mining algorithms. The forecast spans one full day and this forecast can be used in regulating heat consumption of buildings. For training the data mining models, two years of heat consumption data from a case building and weather measurement data from Finnish Meteorological Institute are used. The thesis utilizes Microsoft SQL Server Analysis Services data mining tools in generating the data mining models and CRISP-DM process framework to implement the research. Results show that the built models can predict heat demand at best with mean average percentage errors of 3.8% for 24-h profile and 5.9% for full day. A deployment model for integrating the generated data mining models into an existing building energy management system is also discussed.

Exploring the feasibility of applying data mining for library reference service improvement : a case study of Turku Main Library

Data mining, as a heatedly discussed term, has been studied in various fields. Its possibilities in refining the decision-making process, realizing potential patterns and creating valuable knowledge have won attention of scholars and practitioners. However, there are less studies intending to combine data mining and libraries where data generation occurs all the time. Therefore, this thesis plans to fill such a gap. Meanwhile, potential opportunities created by data mining are explored to enhance one of the most important elements of libraries: reference service. In order to thoroughly demonstrate the feasibility and applicability of data mining, literature is reviewed to establish a critical understanding of data mining in libraries and attain the current status of library reference service. The result of the literature review indicates that free online data resources other than data generated on social media are rarely considered to be applied in current library data mining mandates. Therefore, the result of the literature review motivates the presented study to utilize online free resources. Furthermore, the natural match between data mining and libraries is established. The natural match is explained by emphasizing the data richness reality and considering data mining as one kind of knowledge, an easy choice for libraries, and a wise method to overcome reference service challenges. The natural match, especially the aspect that data mining could be helpful for library reference service, lays the main theoretical foundation for the empirical work in this study. Turku Main Library was selected as the case to answer the research question: whether data mining is feasible and applicable for reference service improvement. In this case, the daily visit from 2009 to 2015 in Turku Main Library is considered as the resource for data mining. In addition, corresponding weather conditions are collected from Weather Underground, which is totally free online. Before officially being analyzed, the collected dataset is cleansed and preprocessed in order to ensure the quality of data mining. Multiple regression analysis is employed to mine the final dataset. Hourly visits are the independent variable and weather conditions, Discomfort Index and seven days in a week are dependent variables. In the end, four models in different seasons are established to predict visiting situations in each season. Patterns are realized in different seasons and implications are created based on the discovered patterns. In addition, library-climate points are generated by a clustering method, which simplifies the process for librarians using weather data to forecast library visiting situation. Then the data mining result is interpreted from the perspective of improving reference service. After this data mining work, the result of the case study is presented to librarians so as to collect professional opinions regarding the possibility of employing data mining to improve reference services. In the end, positive opinions are collected, which implies that it is feasible to utilizing data mining as a tool to enhance library reference service.

Data Mining in Promoting Flight Safety

The incredible rapid development to huge volumes of air travel, mainly because of jet airliners that appeared to the sky in the 1950s, created the need for systematic research for aviation safety and collecting data about air traffic. The structured data can be analysed easily using queries from databases and running theseresults through graphic tools. However, in analysing narratives that often give more accurate information about the case, mining tools are needed. The analysis of textual data with computers has not been possible until data mining tools have been developed. Their use, at least among aviation, is still at a moderate level. The research aims at discovering lethal trends in the flight safety reports. The narratives of 1,200 flight safety reports from years 1994 – 1996 in Finnish were processed with three text mining tools. One of them was totally language independent, the other had a specific configuration for Finnish and the third originally created for English, but encouraging results had been achieved with Spanish and that is why a Finnish test was undertaken, too. The global rate of accidents is stabilising and the situation can now be regarded as satisfactory, but because of the growth in air traffic, the absolute number of fatal accidents per year might increase, if the flight safety will not be improved. The collection of data and reporting systems have reached their top level. The focal point in increasing the flight safety is analysis. The air traffic has generally been forecasted to grow 5 – 6 per cent annually over the next two decades. During this period, the global air travel will probably double also with relatively conservative expectations of economic growth. This development makes the airline management confront growing pressure due to increasing competition, signify cant rise in fuel prices and the need to reduce the incident rate due to expected growth in air traffic volumes. All this emphasises the urgent need for new tools and methods. All systems provided encouraging results, as well as proved challenges still to be won. Flight safety can be improved through the development and utilisation of sophisticated analysis tools and methods, like data mining, using its results supporting the decision process of the executives.

A Dependency Parsing Approach to Biomedical Text Mining

Biomedical research is currently facing a new type of challenge: an excess of information, both in terms of raw data from experiments and in the number of scientific publications describing their results. Mirroring the focus on data mining techniques to address the issues of structured data, there has recently been great interest in the development and application of text mining techniques to make more effective use of the knowledge contained in biomedical scientific publications, accessible only in the form of natural human language. This thesis describes research done in the broader scope of projects aiming to develop methods, tools and techniques for text mining tasks in general and for the biomedical domain in particular. The work described here involves more specifically the goal of extracting information from statements concerning relations of biomedical entities, such as protein-protein interactions. The approach taken is one using full parsing—syntactic analysis of the entire structure of sentences—and machine learning, aiming to develop reliable methods that can further be generalized to apply also to other domains. The five papers at the core of this thesis describe research on a number of distinct but related topics in text mining. In the first of these studies, we assessed the applicability of two popular general English parsers to biomedical text mining and, finding their performance limited, identified several specific challenges to accurate parsing of domain text. In a follow-up study focusing on parsing issues related to specialized domain terminology, we evaluated three lexical adaptation methods. We found that the accurate resolution of unknown words can considerably improve parsing performance and introduced a domain-adapted parser that reduced the error rate of theoriginal by 10% while also roughly halving parsing time. To establish the relative merits of parsers that differ in the applied formalisms and the representation given to their syntactic analyses, we have also developed evaluation methodology, considering different approaches to establishing comparable dependency-based evaluation results. We introduced a methodology for creating highly accurate conversions between different parse representations, demonstrating the feasibility of unification of idiverse syntactic schemes under a shared, application-oriented representation. In addition to allowing formalism-neutral evaluation, we argue that such unification can also increase the value of parsers for domain text mining. As a further step in this direction, we analysed the characteristics of publicly available biomedical corpora annotated for protein-protein interactions and created tools for converting them into a shared form, thus contributing also to the unification of text mining resources. The introduced unified corpora allowed us to perform a task-oriented comparative evaluation of biomedical text mining corpora. This evaluation established clear limits on the comparability of results for text mining methods evaluated on different resources, prompting further efforts toward standardization. To support this and other research, we have also designed and annotated BioInfer, the first domain corpus of its size combining annotation of syntax and biomedical entities with a detailed annotation of their relationships. The corpus represents a major design and development effort of the research group, with manual annotation that identifies over 6000 entities, 2500 relationships and 28,000 syntactic dependencies in 1100 sentences. In addition to combining these key annotations for a single set of sentences, BioInfer was also the first domain resource to introduce a representation of entity relations that is supported by ontologies and able to capture complex, structured relationships. Part I of this thesis presents a summary of this research in the broader context of a text mining system, and Part II contains reprints of the five included publications.

A platform for assessing the current state of data repositories in Korea, Japan and China

Presentation at Open Repositories 2014, Helsinki, Finland, June 9-13, 2014

Data and Inventory Management in Spare Part Business: Developing Operations in the Case Company

After sales business is an effective way to create profit and increase customer satisfaction in manufacturing companies. Despite this, some special business characteristics that are linked to these functions, make it exceptionally challenging in its own way. This Master’s Thesis examines the current situation of the data and inventory management in the case company regarding possibilities and challenges related to the consolidation of current business operations. The research examines process steps, procedures, data requirements, data mining practices and data storage management of spare part sales process, whereas the part focusing on inventory management is reviewing the current stock value and examining current practices and operational principles. There are two global after sales units which supply spare parts and issues reviewed in this study are examined from both units’ perspective. The analysis is focused on the operations of that unit where functions would be centralized by default, if change decisions are carried out. It was discovered that both data and inventory management include clear shortcomings, which result from lack of internal instructions and established processes as well as lack of cooperation with other stakeholders related to product’s lifecycle. The main product of data management was a guideline for consolidating the functions, tailored for the company’s needs. Additionally, potentially scrapped spare part were listed and a proposal of inventory management instructions was drafted. If the suggested spare part materials will be scrapped, stock value will decrease 46 percent. A guideline which was reviewed and commented in this thesis was chosen as the basis of the inventory management instructions.

Methods of Telecommunication Technology Forecasting

This master's thesis coversthe concepts of knowledge discovery, data mining and technology forecasting methods in telecommunications. It covers the various aspects of knowledge discoveryin data bases and discusses in detail the methods of data mining and technologyforecasting methods that are used in telecommunications. Main concern in the overall process of this thesis is to emphasize the methods that are being used in technology forecasting for telecommunications and data mining. It tries to answer to some extent to the question of do forecasts create a future? It also describes few difficulties that arise in technology forecasting. This thesis was done as part of my master's studies in Lappeenranta University of Technology.

Unsupervised visual object categorization

The number of digital images has been increasing exponentially in the last few years. People have problems managing their image collections and finding a specific image. An automatic image categorization system could help them to manage images and find specific images. In this thesis, an unsupervised visual object categorization system was implemented to categorize a set of unknown images. The system is unsupervised, and hence, it does not need known images to train the system which needs to be manually obtained. Therefore, the number of possible categories and images can be huge. The system implemented in the thesis extracts local features from the images. These local features are used to build a codebook. The local features and the codebook are then used to generate a feature vector for an image. Images are categorized based on the feature vectors. The system is able to categorize any given set of images based on the visual appearance of the images. Images that have similar image regions are grouped together in the same category. Thus, for example, images which contain cars are assigned to the same cluster. The unsupervised visual object categorization system can be used in many situations, e.g., in an Internet search engine. The system can categorize images for a user, and the user can then easily find a specific type of image.

Asiakkuuksien strateginen arvottaminen ja luokittelu

Tämän tutkimuksen kohdeorganisaatio on suuren teollisuusyrityksen sisäinen raaka-aineen hankkija ja toimittaja. Tutkimuksessa selvitetään, mistä kohdeorganisaation hankinta-asiakkuuksien arvo muodostuu ja kuinka olemassa olevan liiketoimintadatan perusteella voidaan tutkia, arvioida ja luokitella kauppojen ja asiakkuuksien arvokkuutta aikaan sitomatta, objektiivisesti ja luotettavasti. Tutkimuksen teoriaosiossa esitellään lähestymistapoja ja menetelmiä, joiden avulla voidaan jalostaa olemassa olevasta datasta uutta sidosryhmätietämystä liiketoiminnan käyttöön, sekä tarkastellaan asiakaskannattavuusanalyysin, portfolioanalyysin, sekä asiakassegmentoinnin perusteita ja malleja. Näiden teorioiden ja mallien pohjalta rakennetaan kohdeorganisaatiolle räätälöity, indeksoituihin hinta-, määrä- ja kauppojen toistuvuus-muuttujiin perustuva, asiakkuuksien arvottamis- ja luokittelumalli. Arvottamis- ja luokittelumalli testataan vuosien 2003–2007 liiketoimintadatasta muodostetulla 389 336 kaupparivin otoksella, joka sisältää 42 186 arvioitavaa asiakkuussuhdetta. Merkittävin esille nouseva havainto on noin 5 000:n keskimääräistä selkeästi kalliimman asiakkuuden ryhmä. Aineisto ja sen poikkeavuudet testataan tilastollisin menetelmin, jotta saadaan selville asiakkuuden arvoon vaikuttavat ja arvoa selittävät tekijät. Lopuksi pohditaan arvottamismallin merkitystä analyyttisemman ostotoiminnan ja asiakkuudenhallinnan välineenä, sekä esitetään muutamia parannusehdotuksia.

Business intelligence –järjestelmä integraation tukena kansainvälisessä yrityksessä

Business intelligencellä tarkoitetaan liiketoimintatiedon hallintaan liittyviä prosesseja ja tekniikoita. Se pitää sisällään tiedon keräämiseen, tallentamiseen, analysointiin ja jakamiseen käytettyt tuotteet, tekniikat ja prosessit, joiden tavoitteena on auttaa yrityksen työntekijöitä liiketoimintaan liittyvässä päätöksenteossa. Tutkimuksen tavoitteena on tutkia uuden yritysryhmän laajuisen BI-tietojärjestelmän suunnitteluun ja käyttöönotoon liittyviä seikkoja ja luoda valmiudet BI-tietojärjestelmän kehitys- ja käyttöönottoprojektin kohdeyrityksessä, jonka toimiala on kansainvälinen terveydenhoitoalan tukkuliiketoiminta. Uuden BI-järjestelmän halutaan tukeva yritysryhmän yritysten välistä integraatiota ja tehostavan tiedonhakuun ja analysointiin liittyviä prosesseja. Tutkimus toteutettiin konstruktiivisena tutkimuksena, joka kattaa kohdeyrityksen IT-arkkitehtuurin, tietosisällön, prosessit ja organisaation raportoinnin kannalta. Lisäksi työssä suoritettiin ohjelmistovertailu kahden markkinoilla toimivan merkittävän ohjelmistotalon BI-tuotteiden välillä. Työssä havaittiin, että BI-projekti on laaja-alainen ja suuri hanke, joka ulottuu läpi koko organisaation. BI-ohjelmiston tehokas hyödyntäminen asettaa vaatimuksia erityisesti taustajärjestelmien tiedon huolelliseen mallintamiseen liittyen. Työssä saatiin pilotoinnin kautta käytännön kokemuksia uudesta järjestelmästä ja sen tarjoamista mahdollisuuksista kohdeyrityksessä.

Työpaikkailmoitusten sisällön ja osaamisvaatimusten käsittely

Työpaikkailmoitusten etsiminen internetistä on hyvin yleistä nykyään, mutta kysei- nen prosessi ei ole kehittynyt vuosien varrella muiden palvelujen tapaan. Tämän ta- kia tehokkaan ja omiin taitoihin kohdistetun haun tekeminen on hyvin vaikeaa. Tässä työssä toteutetaan verkkopalvelu, jonka avulla käyttäjä voi tutkia useasta läh- teestä haettuja IT-alan työpaikkailmoituksia ja etsiä niistä omille taidoilleen parhai- ten sopivia. Palvelun taustalla toimiva järjestelmä hakee ilmoitukset ja analysoi ne tarvittavan datan saamiseksi. Samalla ilmoituksista luodaan tilastoja, joita käyttäjät voivat tutkia. Kerätyistä tiedoista saadaan myös selville millaisia yhteyksiä eri am- mattien ja termien välillä on. Palvelun avulla on helppoa tehdä hakuja painottaen omia osaamisalueita. Haun tu- lokset tulostetaan parhaiten sopivasta huonoimmin sopivaan. Jokaisen ilmoituksen mukana tulostetaan listaus ilmoituksessa olleista ammattitermeistä ja jokaisen haun loppuun tulostetaan myös listaus kaikista haun tuloksista löytyneistä ilmoituksista. Kohdistetut haut ovat mahdollisia, koska palvelu kerää tietoja ilmoituksista löytyvis- tä termeistä luokitellen niitä. Tilastoista käyttäjällä on mahdollisuus seurata työpaikkailmoitusmäärien muutoksia viikoittain niin mol:n kuin monsterin järjestelmissä. Pelkkien ilmoitusmäärien lisäksi tilastoista voi seurata yksittäisten ammattitermien esiintymistä, sekä tietyn ammat- tialan ilmoitusten määriä.

Legal challenges for next generation science

Presentation at the Nordic Perspectives on Open Access and Open Science seminar, Helsinki, October 15, 2013