Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

Optimal trading of wind power in the short term market

The energy reform, which is happening all over the world, is caused by the common concern of the future of the humankind in our shared planet. In order to keep the effects of the global warming inside of a certain limit, the use of fossil fuels must be reduced. The marginal costs of the renewable sources, RES are quite high, since they are new technology. In order to induce the implementation of RES to the power grid and lower the marginal costs, subsidies were developed in order to make the use of RES more profitable. From the RES perspective the current market is developed to favor conventional generation, which mainly uses fossil fuels. Intermittent generation, like wind power, is penalized in the electricity market since it is intermittent and thus diffi-cult to control. Therefore, the need of regulation and thus the regulation costs to the producer differ, depending on what kind of generation market participant owns. In this thesis it is studied if there is a way for market participant, who has wind power to use the special characteristics of electricity market Nord Pool and thus reach the gap between conventional generation and the intermittent generation only by placing bids to the market. Thus, an optimal bid is introduced, which purpose is to minimize the regulation costs and thus lower the marginal costs of wind power. In order to make real life simulations in Nord Pool, a wind power forecast model was created. The simulations were done in years 2009 and 2010 by using a real wind power data provided by Hyötytuuli, market data from Nord Pool and wind forecast data provided by Finnish Meteorological Institute. The optimal bid needs probability intervals and therefore the methodology to create probability distributions is introduced in this thesis. In the end of the thesis it is shown that the optimal bidding improves the position of wind power producer in the electricity market.

Auger-type granular fertylizer distributor: matemathical model and dynamic simulation

The objective of this study was to model mathematically and to simulate the dynamic behavior of an auger-type fertilizer applicator (AFA) in order to use the variable-rate application (VRA) and reduce the coefficient of variation (CV) of the application, proposing an angular speed controller θ' for the motor drive shaft. The input model was θ' and the response was the fertilizer mass flow, due to the construction, density of fertilizer, fill factor and the end position of the auger. The model was used to simulate a control system in open loop, with an electric drive for AFA using an armature voltage (V A) controller. By introducing a sinusoidal excitation signal in V A with amplitude and delay phase optimized and varying θ' during an operation cycle, it is obtained a reduction of 29.8% in the CV (constant V A) to 11.4%. The development of the mathematical model was a first step towards the introduction of electric drive systems and closed loop control for the implementation of AFA with low CV in VRA.

Marketing in the pre-development process of agricultural machines: a reference model

The theme of the research is the development of the domain of marketing knowledge in the design of agricultural machinery. It is developed throughout the design of agricultural machinery in order to identify the corporate and customers needs and to develop strategies to satisfy these needs. The central problem of the research questions which marketing tools to apply on pre-development process of farm machinery, in order to increase the market value of the products and of the company and, consequently, generate competitive advantage to the manufacturers of agricultural machinery. As methodology, it was developed bibliographical research and multicase study of the development process of agricultural machinery developed by small, medium and large companies and the academy. As a result, a marketing reference model was elaborated for the pre-development stage of agricultural machinery, which outlines the activities, tasks, mechanisms and controls that can be used in strategic planning and in products planning of agricultural machinery manufacturers, contributing to explain the explicit knowledge in the marketing field.

Powder flow criteria for design of vertical silo walls

For design of vertical silos walls involving the storage of bulk solids to be safe and reliable, it is important knowing the largest possible number of variables such as: flow properties, silo geometry and pattern of flow desired. In order to validate the theories of flow prediction and design of conical hoppers, the flow properties of two bulk solids were determined, the theories of Jenike's flowability and Enstad and Walker for hopper design were analyzed and the results were compared with those experimentally obtained in a reduced model of a semicircular-section silo. Results show that Enstad theory for the hopper design is adequate to occur mass flow inside the silo, and for the sizing of the discharge outlet, the Walker's theory was closer to the appropriate than Jenike's theory, which was higher around 100% than the experimental hopper outlet.

Evaluation of roof slope and exposure with different roofing materials in reduced models of animal production facilities in spring and summer

The present study was conducted at the Department of Rural Engineering and the Department of Animal Morphology and Physiology of FCAV/Unesp, Jaboticabal, SP, Brazil. The objective was to verify the influence of roof slope, exposure and roofing material on the internal temperature of reduced models of animal production facilities. For the development of the research, 48 reduced and dissemble models with dimensions 1.00 × 1.00 × 0.50 m were used. The roof was shed-type, and the models faced to the North or South directions, with 24 models for each side of exposure. Ceramic, galvanized-steel and fibro tiles were used to build the roofs. Slopes varied between 20, 30, 40 and 50% for the ceramic tile and 10, 30, 40 and 50% for the other two. Inside the models, temperature readings were performed at every hour, for 12 months. The results were evaluated in a general linear model in a nested 3 × 4 × 2 factorial arrangement, in which the effects of roofing material and exposure were nested on the factor Slope. Means were compared by the Tukey test at 5% of probability. After analyzing the data, we observed that with the increase in the slope and exposure to the South, there was a drop in the internal temperature within the model at the geographic coordinates of Jaboticabal city (SP/Brazil).

Alueellinen energiatasemalli

Kiinnostus energia-asioiden käsittelemiseen alueellisesti yksittäisen rakennuksen asioiden käsittelyn sijaan on lisääntynyt voimakkaasti viime vuosina. Keskustelu alueellisesta ener-gia-asioista on kuitenkin vasta alussa ja alueellisiin energiatarkasteluihin tarvitaan työväli-neitä. Tämän työn päätavoitteena oli luoda alueellinen energiatasemalli, jolla voidaan laskea alueen rakennuskannan energiankulutusta ja -tuotantoa sekä energiantuotannon hiilidi-oksidipäästöjä helposti ja nopeasti. Työssä perehdyttiin alueellisten energiatarkasteluiden tämän hetken tasoon, työvälineisiin ja ohjeistukseen. Myös Suomen rakentamismääräyskokoelman rakennusten energiatehok-kuusmääräyksiin tutustuttiin. Alueellisen energiatasemallin perustana käytettiin uusien rakentamismääräysten E-lukulaskennan ohjeistusta. Mallin toimivuutta testattiin Helsingin Salmenkallioon rakennettavaan uudisrakennusalueeseen. Testattaessa mallilla laskettiin kyseisen alueen energiankulutus sekä erilaisia energiantuo-tantovaihtoehtoja ja todettiin, että erityisesti energiankulutuksen laskenta on hyvin helppoa, kun alueen rakennusten kerrosalat ovat tiedossa. Tuotantopuolen uusiutuvan energian laskeminen luotettavasti vaatii hiukan enemmän taustatyötä, mutta varsinainen mallin käyt-täminen on helppoa ja tulokset saadaan suoraan selkeinä kuvaajina. Työssä onnistuttiin luomaan tavoitteiden mukainen malli, jota voidaan soveltaa erilaisissa alueellisissa energiatarkasteluissa. Tarkasteltaessa energia-asioita alueellisesti, saadaan esimerkiksi alueen huipputehon tarvetta pienennettyä ja energiantuotantoa optimoitua. Malli ei kuitenkaan ole vielä valmis laskentatyökalu, vaan vaatii jatkokehitystä, mikäli sitä halutaan hyödyntää laajemmin.

Project Master Document adaption in order fulfillment process

This thesis discusses adaption of new project management tool at ABB Oy Motors and Generators business unit, Synchronous Machines profit centre. Thesis studies project modeling in general and buries in the Gate Model used at ABB Synchronous Machines. It is essential to understand Gate Model because this new project management tool, called Project Master Document, is created on the base of the existing project model. Thesis also analyzes goals and structure of Project Master Document in order to ease implementation of this new tool. Project Master Document aims to improved customer order fulfillment by clearing order handover interface. Office process, especially responsibilities and target dates, become also clearer after Master Document implementation. The document is built to be frame for whole order fulfillment process including check points for each gate of project model and updated memos from all project meetings. Furthermore, project progress will be clearly stated by status markings and visualized with colors.

Reflexive model of research-based theatre – processing innovation at the crossroads of theatre, reflection and practice-based innovation activities

Contemporary organisations have to embrace the notion of doing ‘more with less’. This challenges knowledge production within companies and public organisations, forcing them to reorganise their structures and rethink what knowledge production actually means in the context of innovation and how knowledge is actually produced among various professional groups within the organisation in their everyday actions. Innovations are vital for organisational survival, and ‘ordinary’ employees and customers are central but too-often ignored producers of knowledge for contemporary organisations. Broader levels of participation and reflexive practices are needed. This dissertation discusses the missing links between innovation research conducted in the context of industrial management, arts, and culture; applied drama and theatre practices (specifically post-Boalian approaches); and learning – especially organising reflection – in organisational settings. This dissertation (1) explores and extends the role of research-based theatre to organising reflection and reflexive practices in the context of practice-based innovation, (2) develops a reflexive model of RBT for investigating and developing practice-based organisational process innovations in order to contribute to the development of a tool for innovation management and analysis, and (3) operationalises this model within private- and publicsector organisations. The proposed novel reflexive model of research-based theatre for investigating and developing practice-based organisational process innovations extends existing methods and offers a different way of organising reflection and reflexive practices in the context of general innovation management. The model was developed through five participatory action research processes conducted in four different organisations. The results provide learning steps – a reflection path – for understanding complex organisational life, people, and relations amid renewal and change actions. The proposed model provides a new approach to organising and cultivating reflexivity in practice-based innovation activities via research-based theatre. The results can be utilised as a guideline when processing practice-based innovation within private or public organisations. The model helps innovation managers to construct, together with their employees, temporary communities where they can learn together through reflecting on their own and each others’ experiences and to break down assumptions related to their own perspectives. The results include recommendations for practical development steps applicable in various organisations with regard to (i) application of research-based theatre and (ii) related general innovation management. The dissertation thus contributes to the development of novel learning approaches in knowledge production. Keywords: practice-based innovation, research-based theatre, learning, reflection, mode 2b knowledge production

Asiakasohjautuvan kokoonpanon vaikutus huonekalualan yrityksen kannattavuuteen

Työn tavoitteena oli selvittää, miten tuotantomuodon muutos varasto-ohjautuvasta ns. litteiden pakettien tuotannosta asiakasohjautuvaan kokoonpanoon, vaikuttaa tutkittavan tuoteryhmän kannattavuuteen. Tutkimuksen avulla pyrittiin saamaan vastaukset kysymyksiin: Kuinka paljon varastoon sitoutunut pääoma pienenee? Riittääkö nykyinen tuotantokapasiteetti uudessa mallissa? Miten tuotantoa ja varastoja ohjataan uudessa mallissa? Tutkimus koostuu teorian tarjoamien keinojen ja syiden soveltamisesta kohdeyrityksen tarpeisiin ja muutoksen laskennallisen kannattavuuden selvityksestä, jota mitataan absoluuttisen ja suhteellisen mittarien avulla. Lisäksi teorian avulla luodaan periaatteellinen malli uuden tuotantomuodon kapasiteetinhallinnalle sekä tuotannon- ja varastonohjaukselle. Saadut tulokset osoittavat, että tuotantomuodon muutos lisää yrityksen suhteellista kannattavuutta ja mahdollistaa absoluuttisen kannattavuuden lisäyksen, joka riippuu vapautuvien resurssien uudelleenohjauksesta. Lisäksi voidaan todeta, että tutkimuksessa laadullisina muuttujina pidetyt kuljetuskustannukset ja asiakastyytyväisyys, eivät ole lähtökohtainen este muutokselle.

Experimentally induced intravaginal Tritrichomonas foetus infection in a mouse model

The interest to develop research on the host-parasite relationship in bovine tritrichomonosis has accomplished the use of experimental models alternative to cattle. The BALB/c mouse became the most appropriate species susceptible to vaginal Tritrichomonas foetus infection requiring previous estrogenization. For the need of an experimental model without persistent estrogenization and with normal estrous cycles, the establishment and persistence of vaginal infection on BALB/c mouse with different concentrations of T. foetus in two experimental groups was evaluated. Group A was treated with 5mg of b-estradiol 3-benzoate to synchronize the estrous, 48 hours before the T. foetus vaginal inoculation, and Group B was inoculated in natural estrus. At 5-7 days after treatment, estrogenic effect decreased allowing all animals to cycle regularly during the experiment. From the first week post-infection, samples of vaginal mucus were taken from all animals during 34 weeks, in order to evaluate the course of infection and the stage of the estrus cycle. Group A showed 93.6% of infected animals, and Group B showed 38%. Different doses of T. foetus were assayed to establish the vaginal infection, with a persistence of 34 weeks. Although different behavior was observed in each subgroup belonging to either Group A or Group B, there were no significant differences among the infecting doses used. The b-estradiol 3-benzoate treatment had a favorable effect on the establishment of the infection (P<0.0001), but it did not influence its persistence (P=0.1097). According to the results, an experimental mouse model is presented, appropriate for further studies on mechanisms of pathogenicity, immune response, protective evaluation of immunogen and therapeutic effect of drugs.

Validation of a 3D model for oxygen combustion in a circulating fluidized bed

In this thesis, a model called CFB3D is validated for oxygen combustion in circulating fluidized bed boiler. The first part of the work consists of literature review in which circulating fluidized bed and oxygen combustion technologies are studied. In addition, the modeling of circulating fluidized bed furnaces is discussed and currently available industrial scale three-dimensional furnace models are presented. The main features of CFB3D model are presented along with the theories and equations related to the model parameters used in this work. The second part of this work consists of the actual research and modeling work including measurements, model setup, and modeling results. The objectives of this thesis is to study how well CFB3D model works with oxygen combustion compared to air combustion in circulating fluidized bed boiler and what model parameters need to be adjusted when changing from air to oxygen combustion. The study is performed by modeling two air combustion cases and two oxygen combustion cases with comparable boiler loads. The cases are measured at Ciuden 30 MWth Flexi-Burn demonstration plant in April 2012. The modeled furnace temperatures match with the measurements as well in oxygen combustion cases as in air combustion cases but the modeled gas concentrations differ from the measurements clearly more in oxygen combustion cases. However, the same model parameters are optimal for both air and oxygen combustion cases. When the boiler load is changed, some combustion and heat transfer related model parameters need to be adjusted. To improve the accuracy of modeling results, better flow dynamics model should be developed in the CFB3D model. Additionally, more measurements are needed from the lower furnace to find the best model parameters for each case. The validation work needs to be continued in order to improve the modeling results and model predictability.

Reflexive Space. A Constructionist Model of the Russian Reflexive Marker

This study examines the structure of the Russian Reflexive Marker ( ся/-сь) and offers a usage-based model building on Construction Grammar and a probabilistic view of linguistic structure. Traditionally, reflexive verbs are accounted for relative to non-reflexive verbs. These accounts assume that linguistic structures emerge as pairs. Furthermore, these accounts assume directionality where the semantics and structure of a reflexive verb can be derived from the non-reflexive verb. However, this directionality does not necessarily hold diachronically. Additionally, the semantics and the patterns associated with a particular reflexive verb are not always shared with the non-reflexive verb. Thus, a model is proposed that can accommodate the traditional pairs as well as for the possible deviations without postulating different systems. A random sample of 2000 instances marked with the Reflexive Marker was extracted from the Russian National Corpus and the sample used in this study contains 819 unique reflexive verbs. This study moves away from the traditional pair account and introduces the concept of Neighbor Verb. A neighbor verb exists for a reflexive verb if they share the same phonological form excluding the Reflexive Marker. It is claimed here that the Reflexive Marker constitutes a system in Russian and the relation between the reflexive and neighbor verbs constitutes a cross-paradigmatic relation. Furthermore, the relation between the reflexive and the neighbor verb is argued to be of symbolic connectivity rather than directionality. Effectively, the relation holding between particular instantiations can vary. The theoretical basis of the present study builds on this assumption. Several new variables are examined in order to systematically model variability of this symbolic connectivity, specifically the degree and strength of connectivity between items. In usage-based models, the lexicon does not constitute an unstructured list of items. Instead, items are assumed to be interconnected in a network. This interconnectedness is defined as Neighborhood in this study. Additionally, each verb carves its own niche within the Neighborhood and this interconnectedness is modeled through rhyme verbs constituting the degree of connectivity of a particular verb in the lexicon. The second component of the degree of connectivity concerns the status of a particular verb relative to its rhyme verbs. The connectivity within the neighborhood of a particular verb varies and this variability is quantified by using the Levenshtein distance. The second property of the lexical network is the strength of connectivity between items. Frequency of use has been one of the primary variables in functional linguistics used to probe this. In addition, a new variable called Constructional Entropy is introduced in this study building on information theory. It is a quantification of the amount of information carried by a particular reflexive verb in one or more argument constructions. The results of the lexical connectivity indicate that the reflexive verbs have statistically greater neighborhood distances than the neighbor verbs. This distributional property can be used to motivate the traditional observation that the reflexive verbs tend to have idiosyncratic properties. A set of argument constructions, generalizations over usage patterns, are proposed for the reflexive verbs in this study. In addition to the variables associated with the lexical connectivity, a number of variables proposed in the literature are explored and used as predictors in the model. The second part of this study introduces the use of a machine learning algorithm called Random Forests. The performance of the model indicates that it is capable, up to a degree, of disambiguating the proposed argument construction types of the Russian Reflexive Marker. Additionally, a global ranking of the predictors used in the model is offered. Finally, most construction grammars assume that argument construction form a network structure. A new method is proposed that establishes generalization over the argument constructions referred to as Linking Construction. In sum, this study explores the structural properties of the Russian Reflexive Marker and a new model is set forth that can accommodate both the traditional pairs and potential deviations from it in a principled manner.

Oxidation rates of carbon and nitrogen in char residues from solid fuels

Computational fluid dynamics (CFD) modeling is an important tool in designing new combustion systems. By using CFD modeling, entire combustion systems can be modeled and the emissions and the performance can be predicted. CFD modeling can also be used to develop new and better combustion systems from an economical and environmental point of view. In CFD modeling of solid fuel combustion, the combustible fuel is generally treated as single fuel particles. One of the limitations with the CFD modeling concerns the sub-models describing the combustion of single fuel particles. Available models in the scientific literature are in many cases not suitable as submodels for CFD modeling since they depend on a large number of input parameters and are computationally heavy. In this thesis CFD-applicable models are developed for the combustion of single fuel particles. The single particle models can be used to improve the combustion performance in various combustion devices or develop completely new technologies. The investigated fields are oxidation of carbon (C) and nitrogen (N) in char residues from solid fuels. Modeled char-C oxidation rates are compared to experimental oxidation rates for a large number of pulverized solid fuel chars under relevant combustion conditions. The experiments have been performed in an isothermal plug flow reactor operating at 1123-1673 K and 3-15 vol.% O2. In the single particle model, the char oxidation is based on apparent kinetics and depends on three fuel specific parameters: apparent pre-exponential factor, apparent activation energy, and apparent reaction order. The single particle model can be incorporated as a sub-model into a CFD code. The results show that the modeled char oxidation rates are in good agreement with experimental char oxidation rates up to around 70% of burnout. Moreover, the results show that the activation energy and the reaction order can be assumed to be constant for a large number of bituminous coal chars under conditions limited by the combined effects of chemical kinetics and pore diffusion. Based on this, a new model based on only one fuel specific parameter is developed (Paper III). The results also show that reaction orders of bituminous coal chars and anthracite chars differ under similar conditions (Paper I and Paper II); reaction orders of bituminous coal chars were found to be one, while reaction orders of anthracite chars were determined to be zero. This difference in reaction orders has not previously been observed in the literature and should be considered in future char oxidation models. One of the most frequently used comprehensive char oxidation models could not explain the difference in the reaction orders. In the thesis (Paper II), a modification to the model is suggested in order to explain the difference in reaction orders between anthracite chars and bituminous coal chars. Two single particle models are also developed for the NO formation and reduction during the oxidation of single biomass char particles. In the models the char-N is assumed to be oxidized to NO and the NO is partly reduced inside the particle. The first model (Paper IV) is based on the concentration gradients of NO inside and outside the particle and the second model is simplified to such an extent that it is based on apparent kinetics and can be incorporated as a sub-model into a CFD code (Paper V). Modeled NO release rates from both models were in good agreement with experimental measurements from a single particle reactor of quartz glass operating at 1173-1323 K and 3-19 vol.% O2. In the future, the models can be used to reduce NO emissions in new combustion systems.

Improved order fulfillment in paper production

For the paper mill profitability it is crucial to minimize overproduction and underproduction. Overproduction and underproduction both generate undesired costs and profit losses in paper mill production. This master’s thesis examines paper production order fulfillment subject from paper production level point of view. Research and development approach is selected due to clarification requirements in lately implemented manufacturing execution system. Manufacturing execution systems are generally expected to offer reliable and accurate information about mill production details. However, confusions are likely to occur after implementation of new manufacturing execution system. These confusions are usually harmful and become cumulatively more influential the longer they keep occurring. In this master’s thesis is presented actions to improve order fulfillment at paper mill production level. Central points of the improvement actions are a model for successful order fulfillment in paper mill production and manufacturing execution system catalogue configuration redesign. Improvement actions are implemented in Jokilaakso paper production plant and it is examined as a case study. In the end of this master’s thesis is presented performance measurements which demonstrate order fulfillment from case Jokilaakso.