Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

First dice your dill – new methods and techniques in sample handling

This book is dedicated to celebrate the 60th birthday of Professor Rainer Huopalahti. Professor Rainer “Repe” Huopalahti has had, and in fact is still enjoying a distinguished career in the analysis of food and food related flavor compounds. One will find it hard to make any progress in this particular field without a valid and innovative sample handling technique and this is a field in which Professor Huopalahti has made great contributions. The title and the front cover of this book honors Professor Huopahti’s early steps in science. His PhD thesis which was published on 1985 is entitled “Composition and content of aroma compounds in the dill herb, Anethum graveolens L., affected by different factors”. At that time, the thesis introduced new technology being applied to sample handling and analysis of flavoring compounds of dill. Sample handling is an essential task that in just about every analysis. If one is working with minor compounds in a sample or trying to detect trace levels of the analytes, one of the aims of sample handling may be to increase the sensitivity of the analytical method. On the other hand, if one is working with a challenging matrix such as the kind found in biological samples, one of the aims is to increase the selectivity. However, quite often the aim is to increase both the selectivity and the sensitivity. This book provides good and representative examples about the necessity of valid sample handling and the role of the sample handling in the analytical method. The contributors of the book are leading Finnish scientists on the field of organic instrumental analytical chemistry. Some of them are also Repe’ s personal friends and former students from the University of Turku, Department of Biochemistry and Food Chemistry. Importantly, the authors all know Repe in one way or another and are well aware of his achievements on the field of analytical chemistry. The editorial team had a great time during the planning phase and during the “hard work editorial phase” of the book. For example, we came up with many ideas on how to publish the book. After many long discussions, we decided to have a limited edition as an “old school hard cover book” – and to acknowledge more modern ways of disseminating knowledge by publishing an internet version of the book on the webpages of the University of Turku. Downloading the book from the webpage for personal use is free of charge. We believe and hope that the book will be read with great interest by scientists working in the fascinating field of organic instrumental analytical chemistry. We decided to publish our book in English for two main reasons. First, we believe that in the near future, more and more teaching in Finnish Universities will be delivered in English. To facilitate this process and encourage students to develop good language skills, it was decided to be published the book in English. Secondly, we believe that the book will also interest scientists outside Finland – particularly in the other member states of the European Union. The editorial team thanks all the authors for their willingness to contribute to this book – and to adhere to the very strict schedule. We also want to thank the various individuals and enterprises who financially supported the book project. Without that support, it would not have been possible to publish the hardcover book.

Communicative and Anticipatory Decision-Making Supported by Bayesian Networks

The purpose of this research is to draw up a clear construction of an anticipatory communicative decision-making process and a successful implementation of a Bayesian application that can be used as an anticipatory communicative decision-making support system. This study is a decision-oriented and constructive research project, and it includes examples of simulated situations. As a basis for further methodological discussion about different approaches to management research, in this research, a decision-oriented approach is used, which is based on mathematics and logic, and it is intended to develop problem solving methods. The approach is theoretical and characteristic of normative management science research. Also, the approach of this study is constructive. An essential part of the constructive approach is to tie the problem to its solution with theoretical knowledge. Firstly, the basic definitions and behaviours of an anticipatory management and managerial communication are provided. These descriptions include discussions of the research environment and formed management processes. These issues define and explain the background to further research. Secondly, it is processed to managerial communication and anticipatory decision-making based on preparation, problem solution, and solution search, which are also related to risk management analysis. After that, a solution to the decision-making support application is formed, using four different Bayesian methods, as follows: the Bayesian network, the influence diagram, the qualitative probabilistic network, and the time critical dynamic network. The purpose of the discussion is not to discuss different theories but to explain the theories which are being implemented. Finally, an application of Bayesian networks to the research problem is presented. The usefulness of the prepared model in examining a problem and the represented results of research is shown. The theoretical contribution includes definitions and a model of anticipatory decision-making. The main theoretical contribution of this study has been to develop a process for anticipatory decision-making that includes management with communication, problem-solving, and the improvement of knowledge. The practical contribution includes a Bayesian Decision Support Model, which is based on Bayesian influenced diagrams. The main contributions of this research are two developed processes, one for anticipatory decision-making, and the other to produce a model of a Bayesian network for anticipatory decision-making. In summary, this research contributes to decision-making support by being one of the few publicly available academic descriptions of the anticipatory decision support system, by representing a Bayesian model that is grounded on firm theoretical discussion, by publishing algorithms suitable for decision-making support, and by defining the idea of anticipatory decision-making for a parallel version. Finally, according to the results of research, an analysis of anticipatory management for planned decision-making is presented, which is based on observation of environment, analysis of weak signals, and alternatives to creative problem solving and communication.

Search Engine Optimization Methods & Search Engine Indexing for CMS Applications

Search engine optimization & marketing is a set of processes widely used on websites to improve search engine rankings which generate quality web traffic and increase ROI. Content is the most important part of any website. CMS web development is now become very essential for most of organizations and online businesses to develop their online system and websites. Every online business using a CMS wants to get users (customers) to make profit and ROI. This thesis comprises a brief study of existing SEO methods, tools and techniques and how they can be implemented to optimize a content base website. In results, the study provides recommendations about how to use SEO methods; tools and techniques to optimize CMS based websites on major search engines. This study compares popular CMS systems like Drupal, WordPress and Joomla SEO features and how implementing SEO can be improved on these CMS systems. Having knowledge of search engine indexing and search engine working is essential for a successful SEO campaign. This work is a complete guideline for web developers or SEO experts who want to optimize a CMS based website on all major search engines.

Computational methods for tactical simulations

Tämä taktiikan tutkimus keskittyy tietokoneavusteisen simuloinnin laskennallisiin menetelmiin, joita voidaan käyttää taktisen tason sotapeleissä. Työn tärkeimmät tuotokset ovat laskennalliset mallit todennäköisyyspohjaisen analyysin mahdollistaviin taktisen tason taistelusimulaattoreihin, joita voidaan käyttää vertailevaan analyysiin joukkue-prikaatitason tarkastelutilanteissa. Laskentamallit keskittyvät vaikuttamiseen. Mallit liittyvät vahingoittavan osuman todennäköisyyteen, jonka perusteella vaikutus joukossa on mallinnettu tilakoneina ja Markovin ketjuina. Edelleen näiden tulokset siirretään tapahtumapuuanalyysiin operaation onnistumisen todennäköisyyden osalta. Pienimmän laskentayksikön mallinnustaso on joukkue- tai ryhmätasolla, jotta laskenta-aika prikaatitason sotapelitarkasteluissa pysyisi riittävän lyhyenä samalla, kun tulokset ovat riittävän tarkkoja suomalaiseen maastoon. Joukkueiden mies- ja asejärjestelmävahvuudet ovat jakaumamuodossa, eivätkä yksittäisiä lukuja. Simuloinnin integroinnissa voidaan käyttää asejärjestelmäkohtaisia predictor corrector –parametreja, mikä mahdollistaa aika-askelta lyhytaikaisempien taistelukentän ilmiöiden mallintamisen. Asemallien pohjana ovat aiemmat tutkimukset ja kenttäkokeet, joista osa kuuluu tähän väitöstutkimukseen. Laskentamallien ohjelmoitavuus ja käytettävyys osana simulointityökalua on osoitettu tekijän johtaman tutkijaryhmän ohjelmoiman ”Sandis”- taistelusimulointiohjelmiston avulla, jota on kehitetty ja käytetty Puolustusvoimien Teknillisessä Tutkimuslaitoksessa. Sandikseen on ohjelmoitu karttakäyttöliittymä ja taistelun kulkua simuloivia laskennallisia malleja. Käyttäjä tai käyttäjäryhmä tekee taktiset päätökset ja syöttää nämä karttakäyttöliittymän avulla simulointiin, jonka tuloksena saadaan kunkin joukkuetason peliyksikön tappioiden jakauma, keskimääräisten tappioiden osalta kunkin asejärjestelmän aiheuttamat tappiot kuhunkin maaliin, ammuskulutus ja radioyhteydet ja niiden tila sekä haavoittuneiden evakuointi-tilanne joukkuetasolta evakuointisairaalaan asti. Tutkimuksen keskeisiä tuloksia (kontribuutio) ovat 1) uusi prikaatitason sotapelitilanteiden laskentamalli, jonka pienin yksikkö on joukkue tai ryhmä; 2) joukon murtumispisteen määritys tappioiden ja haavoittuneiden evakuointiin sitoutuvien taistelijoiden avulla; 3) todennäköisyyspohjaisen riskianalyysin käyttömahdollisuus vertailevassa tutkimuksessa sekä 4) kokeellisesti testatut tulen vaikutusmallit ja 5) toimivat integrointiratkaisut. Työ rajataan maavoimien taistelun joukkuetason todennäköisyysjakaumat luovaan laskentamalliin, kenttälääkinnän malliin ja epäsuoran tulen malliin integrointimenetelmineen sekä niiden antamien tulosten sovellettavuuteen. Ilmasta ja mereltä maahan -asevaikutusta voidaan tarkastella, mutta ei ilma- ja meritaistelua. Menetelmiä soveltavan Sandis -ohjelmiston malleja, käyttötapaa ja ohjelmistotekniikkaa kehitetään edelleen. Merkittäviä jatkotutkimuskohteita mallinnukseen osalta ovat muun muassa kaupunkitaistelu, vaunujen kaksintaistelu ja maaston vaikutus tykistön tuleen sekä materiaalikulutuksen arviointi.

Bankruptcy prediction using fuzzy methods

In this thesis, a classi cation problem in predicting credit worthiness of a customer is tackled. This is done by proposing a reliable classi cation procedure on a given data set. The aim of this thesis is to design a model that gives the best classi cation accuracy to e ectively predict bankruptcy. FRPCA techniques proposed by Yang and Wang have been preferred since they are tolerant to certain type of noise in the data. These include FRPCA1, FRPCA2 and FRPCA3 from which the best method is chosen. Two di erent approaches are used at the classi cation stage: Similarity classi er and FKNN classi er. Algorithms are tested with Australian credit card screening data set. Results obtained indicate a mean classi cation accuracy of 83.22% using FRPCA1 with similarity classi- er. The FKNN approach yields a mean classi cation accuracy of 85.93% when used with FRPCA2, making it a better method for the suitable choices of the number of nearest neighbors and fuzziness parameters. Details on the calibration of the fuzziness parameter and other parameters associated with the similarity classi er are discussed.

Techno-Economic Feasibility of Novel On-Line Condition Monitoring Methods in Low Voltage Distribution Networks

The focus in this thesis is to study both technical and economical possibilities of novel on-line condition monitoring techniques in underground low voltage distribution cable networks. This thesis consists of literature study about fault progression mechanisms in modern low voltage cables, laboratory measurements to determine the base and restrictions of novel on-line condition monitoring methods, and economic evaluation, based on fault statistics and information gathered from Finnish distribution system operators. This thesis is closely related to master’s thesis “Channel Estimation and On-line Diagnosis of LV Distribution Cabling”, which focuses more on the actual condition monitoring methods and signal theory behind them.

A Case Study of Agile Software Engineering Methods and Practices in the MeeGo Harmattan Open Source and Product Context

The aim of this master’s thesis is to study how Agile method (Scrum) and open source software are utilized to produce software for a flagship product in a complex production environment. The empirical case and the used artefacts are taken from the Nokia MeeGo N9 product program, and from the related software program, called as the Harmattan. The single research case is analysed by using a qualitative method. The Grounded Theory principles are utilized, first, to find out all the related concepts from artefacts. Second, these concepts are analysed, and finally categorized to a core category and six supported categories. The result is formulated as the operation of software practices conceivable in circumstances, where the accountable software development teams and related context accepts a open source software nature as a part of business vision and the whole organization supports the Agile methods.

Novel Regression Methods For Spectral Data

Singular Value Decomposition (SVD), Principal Component Analysis (PCA) and Multiple Linear Regression (MLR) are some of the mathematical pre- liminaries that are discussed prior to explaining PLS and PCR models. Both PLS and PCR are applied to real spectral data and their di erences and similarities are discussed in this thesis. The challenge lies in establishing the optimum number of components to be included in either of the models but this has been overcome by using various diagnostic tools suggested in this thesis. Correspondence analysis (CA) and PLS were applied to ecological data. The idea of CA was to correlate the macrophytes species and lakes. The di erences between PLS model for ecological data and PLS for spectral data are noted and explained in this thesis. i

Development of Cost Estimation with Feedback Information in EPCM Business

Target company of this study is a large machinery company, which is, inter alia, engaged in energy and pulp engineering, procurement and construction management (EPCM) supply business. The main objective of this study was to develop cost estimation of the target company by providing more accurate, reliable and up-to-date information through enterprise resource planning (ERP) system. Another objective was to find cost-effective methods to collect total cost of ownership information to support more informed supplier selection decision making. This study is primarily action-oriented, but also constructive, and it can be divided in two sections: theoretical literature review and empirical study on the abovementioned part of the target company’s business. Development of information collection is, in addition to literature review, based on nearly 30 qualitative interviews of employees at various organizational units, functions and levels at the target company. At the core of development was to make initial data more accurate, reliable and available, a necessary prerequisite for informed use of the information. Certain development suggestions and paths were presented in order to regain confidence in ERP system as information source by reorganizing work breakdown structure and by complementing mere cost information with quantitative, technical and scope information. Several methods to use the information ever more effectively were also discussed. While implementation of the development suggestions outreached the scope of this study, it was forwarded in test environment and interest groups.

Simulation methods in the modelling of bioaffinity assays

Computational model-based simulation methods were developed for the modelling of bioaffinity assays. Bioaffinity-based methods are widely used to quantify a biological substance in biological research, development and in routine clinical in vitro diagnostics. Bioaffinity assays are based on the high affinity and structural specificity between the binding biomolecules. The simulation methods developed are based on the mechanistic assay model, which relies on the chemical reaction kinetics and describes the forming of a bound component as a function of time from the initial binding interaction. The simulation methods were focused on studying the behaviour and the reliability of bioaffinity assay and the possibilities the modelling methods of binding reaction kinetics provide, such as predicting assay results even before the binding reaction has reached equilibrium. For example, a rapid quantitative result from a clinical bioaffinity assay sample can be very significant, e.g. even the smallest elevation of a heart muscle marker reveals a cardiac injury. The simulation methods were used to identify critical error factors in rapid bioaffinity assays. A new kinetic calibration method was developed to calibrate a measurement system by kinetic measurement data utilizing only one standard concentration. A nodebased method was developed to model multi-component binding reactions, which have been a challenge to traditional numerical methods. The node-method was also used to model protein adsorption as an example of nonspecific binding of biomolecules. These methods have been compared with the experimental data from practice and can be utilized in in vitro diagnostics, drug discovery and in medical imaging.

Improving product profitability control in continuous IT-services

The target of the thesis is to improve product profitability control in continuous IT-services. Accurate product cost accounting and correctly allocated revenues are a necessity for good product profitability control. The focus of the study is on costs and revenues that are not traced directly to services. The thesis is focused on revenue allocations as revenue allocation methods have not been used in the case company before. In order to achieve the target revenue allocation methods, which improve the product profitability accounting and control, are presented. The research methods used in the thesis are literature review and empirical case study. The research approach is constructive. The theoretical part is composed of literature and articles that create a base for the empirical part. Internal interviews describe the current situation in the company and based on it development actions are planned. The part of the empirical case study is seen mostly in the limitations as the research is limited to concern only one department in the company. Problems in the revenue tracing are caused by customer specific services and lack of service definitions because of which the revenues are not traced correctly. Methods to allocate revenues are presented in the thesis and stand-alone revenue allocation method is the most suitable one because it is fair and it can be modified. Approximate product profitability analysis is done in the thesis and the results of it indicate that some services are profitable and some unprofitable.

Developing the measurement and management of knowledge: applying the constructive research approach in a knowledge intensive service environment

Intellectual assets have attained continuous attention in the academic field, as they are vital sources of competitive advantage and organizational performance in the contemporary knowledge intensive business environment. Intellectual capital measurement is quite thoroughly addressed in the accounting literature. However, the purpose of the measurement is to support the management of intellectual assets, but the reciprocal relationship between measurement and management has not been comprehensively considered in the literature. The theoretical motivation for this study rose from this paradox, as in order to maximise the effectiveness of knowledge management the two initiatives need to be closely integrated. The research approach of this interventionist case study is constructive. The objective is to develop the case organization’s knowledge management and intellectual capital measurement in a way that they would be closely integrated and the measurement would support the management of intellectual assets. The case analysis provides valuable practical considerations about the integration and related issues as the case company is a knowledge intensive organization in which the know-how of the employees is the central competitive asset and therefore, the management and measurement of knowledge are essential for its future success. The results suggest that the case organization is confronting challenges in managing knowledge. In order to appropriately manage knowledge processes and control the related risks, support from intellectual capital measurement is required. However, challenges in measuring intellectual capital, especially knowledge, could be recognized in the organization. By reflecting the knowledge management situation and the constructed strategy map, a new intellectual measurement system was developed for the case organization. The construction of the system as well as its indicators can be perceived to contribute to the literature, emphasizing of the importance of properly considering the organization’s knowledge situation in developing an intellectual capital measurement system.