Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

Hybrid warfare – just a twist of compound warfare? : views on warfare from the United States Armed Forces perspective

The wars the Western armies are involved with today are different from those that were fought in the end of 20th century. To explain this change, the Western military thinkers have come up with various different types of definitions of warfare over the last 30 years, each describing the tendencies involved in the conflicts of the time. The changing nature of conflicts surfaced a new term – hybrid warfare. The term was to describe and explain the multi-modality and complexity of modern day conflict. This thesis seeks the answer for the question: what is the development of thought behind hybrid warfare? In this thesis the Vietnam War (1965-1975) is used as an example of compound warfare focusing on the American involvement in the war. The Second Lebanon War (2006) serves as an example of hybrid warfare. Both case studies include an irregular opposing force, namely National Liberation Front in Vietnam War and Hezbollah in the Second Lebanon War. These two case studies are compared with the term full spectrum operations introduced in the current U.S. Department of Army Field Manual No. 3-0 Operations to see the differences and similarities of each term. The perspective of this thesis is the American point of view. This thesis concludes that hybrid warfare, compound warfare and full spectrum operations are very similar. The first two terms are included in the last one. Although hybrid warfare is not officially defined, it will most likely remain to be used in the discussion in the future, since hybrid wars and hybrid threats are officially accepted terms.

Development of a social service in a multi-party network

Tämän pro gradu-tutkimuksen tarkoituksena oli tutkia monen toimijan sosiaalipalvelukehittäjäverkoston toimivuutta ja sen toimivuuteen vaikuttavia tekijöitä. Aihetta lähestyttiin erilaisten teoreettisten kokonaisuuksien kautta, joiden avulla saatiin luotua tutkimukselle pohja. Viitekehys tutkimukselle luotiin yhdistäen erilaisia teoreettisia aihealueita verkostoista, verkostojen johtamisesta ja palveluista. Tutkimuksessa korostuu motivaation, yhteisen, tarpeeseen perustuvan tavoitteen, sitoutumisen ja orkestroinnin merkitys verkostotoiminnassa hyvän lopputuloksen aikaansaamiseksi. Tutkimuksen empiirisessä osuudessa tehty kvalitatiivinen case-tutkimus keskittyy tiettyyn verkostoon, joka on Socomin koordinoimana kehittänyt Kaakkois-Suomen alueelle uudenlaista sosiaalipalvelua liittyen henkilökohtaiseen apuun. Verkosto on monen toimijan verkosto, jonka jäsenet edustavat erilaisia tahoja ja organisaatioita. Tutkimuksen perusteella verkosto on toiminut hyvin ja tehokkaasti ja saanut luotua toimivan sosiaalipalvelun. Verkosto tukee kirjallisuuskatsauksessa löydettyjen tekijöiden, kuten verkosto-orkestroinnin, sitoutumisen ja yhteisen päämäärän, vaikutusta verkoston toimintaan ja lopputulokseen.

Generalized Differential Evolution for Global Multi-Objective Optimization with Constraints

The objective of this thesis work is to develop and study the Differential Evolution Algorithm for multi-objective optimization with constraints. Differential Evolution is an evolutionary algorithm that has gained in popularity because of its simplicity and good observed performance. Multi-objective evolutionary algorithms have become popular since they are able to produce a set of compromise solutions during the search process to approximate the Pareto-optimal front. The starting point for this thesis was an idea how Differential Evolution, with simple changes, could be extended for optimization with multiple constraints and objectives. This approach is implemented, experimentally studied, and further developed in the work. Development and study concentrates on the multi-objective optimization aspect. The main outcomes of the work are versions of a method called Generalized Differential Evolution. The versions aim to improve the performance of the method in multi-objective optimization. A diversity preservation technique that is effective and efficient compared to previous diversity preservation techniques is developed. The thesis also studies the influence of control parameters of Differential Evolution in multi-objective optimization. Proposals for initial control parameter value selection are given. Overall, the work contributes to the diversity preservation of solutions in multi-objective optimization.

Fraud detection in the banking sector : a multi-agent approach

Fraud is an increasing phenomenon as shown in many surveys carried out by leading international consulting companies in the last years. Despite the evolution of electronic payments and hacking techniques there is still a strong human component in fraud schemes. Conflict of interest in particular is the main contributing factor to the success of internal fraud. In such cases anomaly detection tools are not always the best instruments, since the fraud schemes are based on faking documents in a context dominated by lack of controls, and the perpetrators are those ones who should control possible irregularities. In the banking sector audit team experts can count only on their experience, whistle blowing and the reports sent by their inspectors. The Fraud Interactive Decision Expert System (FIDES), which is the core of this research, is a multi-agent system built to support auditors in evaluating suspicious behaviours and to speed up the evaluation process in order to detect or prevent fraud schemes. The system combines Think-map, Delphi method and Attack trees and it has been built around audit team experts and their needs. The output of FIDES is an attack tree, a tree-based diagram to ”systematically categorize the different ways in which a system can be attacked”. Once the attack tree is built, auditors can choose the path they perceive as more suitable and decide whether or not to start the investigation. The system is meant for use in the future to retrieve old cases in order to match them with new ones and find similarities. The retrieving features of the system will be useful to simplify the risk management phase, since similar countermeasures adopted for past cases might be useful for present ones. Even though FIDES has been built with the banking sector in mind, it can be applied in all those organisations, like insurance companies or public organizations, where anti-fraud activity is based on a central anti-fraud unit and a reporting system.

Multi-axis solutions in packaging machines in Europe

The objective of this thesis was to examine the potential of multi-axis solutions in packaging machines produced in Europe. The definition of a multi-axis solution in this study is a construction that uses a common DC bus power supply for different amplifiers running the axes and the intelligence is centralized into one unit. The cost structure of a packaging machine was gained from an automation research, which divided the machines according to automation categories. The automation categories were then further divided into different sub-components by evaluating the ratio of multi-axis solutions compared to other automation components in packaging machines. A global motion control study was used for further information. With the help of the ratio, an estimation of the potential of multi-axis solutions in each country and packaging machine sector was completed. In addition to the research, a specific questionnaire was sent to five companies to gain information about the present situation and possible trends in packaging machinery. The greatest potential markets are in Germany and Italy, which are also the largest producers of packaging machinery in Europe. The greatest growth in the next few years will be seen in Turkey where the annual growth rate equals the general machinery production rate in Asia. The greatest market potential of the Nordic countries is found in Sweden in 35th position on the list. According to the interviews, motion control products in packaging machines will retain their current power levels, as well as the number of axes in the future. Integrated machine safety features together with a universal programming language are the desired attributes of the future. Unlike generally in industry, the energy saving objectives are and will remain insignificant in the packaging industry.

Innovation in a Non-Formal Adult Education Organisation – Multi-Case Study in Four Adult Education Centres

More than ever, education organisations are experiencing the need to develop new services and processes to satisfy expanding and changing customer needs and to adapt to the environmental changes and continually tightening economic situation. Innovation has been found in many studies to have a crucial role in the success of an organisation, both in the private and public sectors, in formal education and in manufacturing and services alike. However, studies concerning innovation in non-formal adult education organisations, such as adult education centres (AECs) in Finland, are still lacking. This study investigates innovation in the non-formal adult education organisation context from the perspective of organisational culture types and social networks. The objective is to determine the significant characteristics of an innovative non-formal adult education organisation. The analysis is based on data from interviews with the principals and fulltime staff of four case AECs. Before the case study, a pre-study phase is accomplished in order to obtain a preliminary understanding of innovation at AECs. The research found strong support for the need of innovation in AECs. Innovation is basically needed to accomplish the AEC system’s primary mission mentioned in the ACT on Liberal Adult Education. In addition, innovation is regarded vital to institutes and may prevent their decline. It helps the institutes to be more attractive, to enter new market, to increase customer satisfaction and to be on the cutting edge. Innovation is also seen as a solution to the shortage of resources. Innovative AECs search actively for additional resources for development work through project funding and subsidies, cooperation networks and creating a conversational and joyful atmosphere in the institute. The findings also suggest that the culture type that supports innovation at AECs is multidimensional, with an emphasis on the clan and adhocratic culture types and such values as: dynamism, future orientation, acquiring new resources, mistake tolerance, openness, flexibility, customer orientation, a risk-taking attitude, and community spirit. Active and creative internal and external cooperation also promote innovation at AECs. This study also suggests that the behaviour of a principal is crucial. The way he or she shows appreciation the staff, encouragement and support to the staff and his or her approachability and concrete participation in innovation activities have a strong effect on innovation attitudes and activities in AECs.

Outbreak of infectious laryngotracheitis in large multi-age egg layer chicken flocks in Minas Gerais, Brazil

A recent (November 2010) outbreak of infectious laryngotracheitis (ILT) in a multi-age laying hen facility in Minas Gerais state, Brazil, is described. Previous ILT outbreak in laying hens was only notified in São Paulo state, Brazil, in 2002. In the outbreak described here, the affected population was approximately eight million hens, with flock sizes ranging from 100,000 to 2,900,000 chickens. The average mortality ranged from 1 to 6%, and morbidity was around 90% (most of the twenty seven farms of the area were positive for ILT virus). Three multi-age laying farms from one company were selected for this report. Clinical signs included prostration, dyspnea, conjunctivitis, occasional swelling of the paranasal sinuses and bloody mucous nasal discharge. Severely affected chickens presented with dyspnea, gasping and became cyanotic before death. At necropsy, these chickens had fibrinous exudate blocking the larynx and the lumen of cranial part of the trachea. In addition, conjunctivitis with intense hyperemia, edema and sinuses with caseous exudate were present. On histopathology, there were marked necrosis and desquamation of respiratory ephitelium and conjunctiva with numerous syncytial cells formation and fibrinous exudate. Moderate to marked non suppurative (especially lymphocytes and plasma cells) infiltration in the lamina propria also was observed. Sixteen out of 20 examined chickens, eosinophilic intranuclear inclusion bodies were observed in the syncytial cells. The DNA extracted from larynx and trachea produced positive PCR results for ILT virus (ILTV) DNA using formalin-fixed, paraffin embedded (FFPE) samples. Amplicons from a small region of ICP4 gene were submitted to sequencing and showed 100% identity with ILTV EU104910.1 (USA strain), 99% with ILTV JN596963.1 (Australian strain) and 91% with ILTV JN580316.1 (Gallid herpesvirus 1 CEO vaccine strain) and JN580315.1 (Gallid herpesvirus 1 TCO vaccine strain).

Non-Linear Unsteady Aerodynamic Response Approximation Using Multi-Layer Functionals

Non-linear functional representation of the aerodynamic response provides a convenient mathematical model for motion-induced unsteady transonic aerodynamic loads response, that accounts for both complex non-linearities and time-history effects. A recent development, based on functional approximation theory, has established a novel functional form; namely, the multi-layer functional. For a large class of non-linear dynamic systems, such multi-layer functional representations can be realised via finite impulse response (FIR) neural networks. Identification of an appropriate FIR neural network model is facilitated by means of a supervised training process in which a limited sample of system input-output data sets is presented to the temporal neural network. The present work describes a procedure for the systematic identification of parameterised neural network models of motion-induced unsteady transonic aerodynamic loads response. The training process is based on a conventional genetic algorithm to optimise the network architecture, combined with a simplified random search algorithm to update weight and bias values. Application of the scheme to representative transonic aerodynamic loads response data for a bidimensional airfoil executing finite-amplitude motion in transonic flow is used to demonstrate the feasibility of the approach. The approach is shown to furnish a satisfactory generalisation property to different motion histories over a range of Mach numbers in the transonic regime.

Petri Net approach for modelling system integration in intelligent buildings

In this paper, a Petri Net approach is introduced for modelling and simulation of control strategies in Intelligent Building. In this context, it is claimed that integration with other building systems can be achieved in a more systematic way considering a mechatronic approach (i.e. multidisciplinary concepts applied to the development of systems). The case study is the Ambulatory Building of Medical School Hospital of University of São Paulo. Particularly, the developed methodology is applied to the elevator system and to the HVAC (Heating, Ventilation and Air Conditioning) system. It is shown that using this approach, the control systems could be integrated, improving performance.

Magnetic properties of multi-walled nanotubes on polysterene substrate

Nanotubes are one of the most perspective materials in modern nanotechologies. It makes present investigation very actual. In this work magnetic properties of multi-walled nanotubes on polystyrene substrate are investigated by using quantum magnetometer SQUID. Main purpose was to obtain magnetic field and temperature dependences of magnetization and to compare them to existing theoretical models of magnetism in carbon-bases structures. During data analysis a mathematical algorithm for obtained data filtration was developed because measurement with quantum magnetometer assume big missives of number data, which contain accidental errors. Nature of errors is drift of SQUID signal, errors of different parts of measurement station. Nanotube samples on polystyrene substrate were studied with help of atomic force microscope. On the surface traces of nanotube were found contours, which were oriented in horizontal plane. This feature was caused by rolling method for samples. Detailed comparison of obtained dependences with information of other researches on this topic allows to obtain some conclusions about nature of magnetism in the samples. It emphasizes importance and actuality of this scientific work.