On the differential evolution algorithm and its appliation to training radial basis function networks

The parameter setting of a differential evolution algorithm must meet several requirements: efficiency, effectiveness, and reliability. Problems vary. The solution of a particular problem can be represented in different ways. An algorithm most efficient in dealing with a particular representation may be less efficient in dealing with other representations. The development of differential evolution-based methods contributes substantially to research on evolutionary computing and global optimization in general. The objective of this study is to investigatethe differential evolution algorithm, the intelligent adjustment of its controlparameters, and its application. In the thesis, the differential evolution algorithm is first examined using different parameter settings and test functions. Fuzzy control is then employed to make control parameters adaptive based on an optimization process and expert knowledge. The developed algorithms are applied to training radial basis function networks for function approximation with possible variables including centers, widths, and weights of basis functions and both having control parameters kept fixed and adjusted by fuzzy controller. After the influence of control variables on the performance of the differential evolution algorithm was explored, an adaptive version of the differential evolution algorithm was developed and the differential evolution-based radial basis function network training approaches were proposed. Experimental results showed that the performance of the differential evolution algorithm is sensitive to parameter setting, and the best setting was found to be problem dependent. The fuzzy adaptive differential evolution algorithm releases the user load of parameter setting and performs better than those using all fixedparameters. Differential evolution-based approaches are effective for training Gaussian radial basis function networks.

Business competence and training needs in Finnish companies operating in the Leningrad Region

Tässä pro gradu -työssä tutkitaan Leningradin alueella, Venäjällä, toimivien suomalaisyritysten liiketoimintaosaamisen koulutustarpeita. Tavoitteena on ollut tutkia, millaisia yritysten koulutustarpeet ovat, sekä lisäksi selvittää yleisemmällä tasolla, miten liiketoimintaosaaminen määritellään. Useat tutkimusta varten haastatellut johtajat pitävät liiketoimintaosaamista erityisesti markkinoilla toimimiseen liittyvänä osaamisena. Myös johtaminen, sekä tuotteet ja teknologia nähdään liiketoimintaosaamisen tärkeinä osina. Yrityksillä on koulutustarpeita seuraavilla alueilla: johtaminen; myynti, markkinat ja asiakkaat; yrityksen sisäinen yhteistyö; kielet, sekä juridiikka ja laskentatoimi. Haastateltavien mukaan markkinoiden nopea kehitys sekä yrityksen kasvu luovat yrityksille koulutustarpeita. Yllättäen myös Venäjän koulutusjärjestelmää itsessään pidetään koulutustarpeiden syynä. Tutkimuksessa mukana olleiden yritysten koulutuskäytännöt ovat keskenään melko erilaisia: koulutusbudjetti, koulutuspäivien määrä ja koulutusorganisaation valintakriteerit vaihtelevatyrityksestä riippuen. Joka tapauksessa yleisin koulutusmuoto näyttää olevan yrityksen sisäinen koulutus. Monet haastateltavat painottavat suuresti uusien työntekijöiden kouluttamista. Selvästikin rekrytointi ja uusien työntekijöiden koulutus vievät suuren osan tutkimusta varten haastateltujen johtajien ajasta. Tärkeä huomio koulutusmarkkinoihin liittyen on se, että lyhyiden, kaikille avoimien koulutusten kohdalla markkinat ovat Pietarissa täynnä. Suurimpana uhkana nähdään alalla vallitseva kouluttajapula.

New kernel functions and learning methods for text and data mining

Recent advances in machine learning methods enable increasingly the automatic construction of various types of computer assisted methods that have been difficult or laborious to program by human experts. The tasks for which this kind of tools are needed arise in many areas, here especially in the fields of bioinformatics and natural language processing. The machine learning methods may not work satisfactorily if they are not appropriately tailored to the task in question. However, their learning performance can often be improved by taking advantage of deeper insight of the application domain or the learning problem at hand. This thesis considers developing kernel-based learning algorithms incorporating this kind of prior knowledge of the task in question in an advantageous way. Moreover, computationally efficient algorithms for training the learning machines for specific tasks are presented. In the context of kernel-based learning methods, the incorporation of prior knowledge is often done by designing appropriate kernel functions. Another well-known way is to develop cost functions that fit to the task under consideration. For disambiguation tasks in natural language, we develop kernel functions that take account of the positional information and the mutual similarities of words. It is shown that the use of this information significantly improves the disambiguation performance of the learning machine. Further, we design a new cost function that is better suitable for the task of information retrieval and for more general ranking problems than the cost functions designed for regression and classification. We also consider other applications of the kernel-based learning algorithms such as text categorization, and pattern recognition in differential display. We develop computationally efficient algorithms for training the considered learning machines with the proposed kernel functions. We also design a fast cross-validation algorithm for regularized least-squares type of learning algorithm. Further, an efficient version of the regularized least-squares algorithm that can be used together with the new cost function for preference learning and ranking tasks is proposed. In summary, we demonstrate that the incorporation of prior knowledge is possible and beneficial, and novel advanced kernels and cost functions can be used in algorithms efficiently.

Learning Preferences with Kernel-Based Methods

Learning of preference relations has recently received significant attention in machine learning community. It is closely related to the classification and regression analysis and can be reduced to these tasks. However, preference learning involves prediction of ordering of the data points rather than prediction of a single numerical value as in case of regression or a class label as in case of classification. Therefore, studying preference relations within a separate framework facilitates not only better theoretical understanding of the problem, but also motivates development of the efficient algorithms for the task. Preference learning has many applications in domains such as information retrieval, bioinformatics, natural language processing, etc. For example, algorithms that learn to rank are frequently used in search engines for ordering documents retrieved by the query. Preference learning methods have been also applied to collaborative filtering problems for predicting individual customer choices from the vast amount of user generated feedback. In this thesis we propose several algorithms for learning preference relations. These algorithms stem from well founded and robust class of regularized least-squares methods and have many attractive computational properties. In order to improve the performance of our methods, we introduce several non-linear kernel functions. Thus, contribution of this thesis is twofold: kernel functions for structured data that are used to take advantage of various non-vectorial data representations and the preference learning algorithms that are suitable for different tasks, namely efficient learning of preference relations, learning with large amount of training data, and semi-supervised preference learning. Proposed kernel-based algorithms and kernels are applied to the parse ranking task in natural language processing, document ranking in information retrieval, and remote homology detection in bioinformatics domain. Training of kernel-based ranking algorithms can be infeasible when the size of the training set is large. This problem is addressed by proposing a preference learning algorithm whose computation complexity scales linearly with the number of training data points. We also introduce sparse approximation of the algorithm that can be efficiently trained with large amount of data. For situations when small amount of labeled data but a large amount of unlabeled data is available, we propose a co-regularized preference learning algorithm. To conclude, the methods presented in this thesis address not only the problem of the efficient training of the algorithms but also fast regularization parameter selection, multiple output prediction, and cross-validation. Furthermore, proposed algorithms lead to notably better performance in many preference learning tasks considered.

MCMC Analysis Of Classical Time Series Algorithms

Identification of order of an Autoregressive Moving Average Model (ARMA) by the usual graphical method is subjective. Hence, there is a need of developing a technique to identify the order without employing the graphical investigation of series autocorrelations. To avoid subjectivity, this thesis focuses on determining the order of the Autoregressive Moving Average Model using Reversible Jump Markov Chain Monte Carlo (RJMCMC). The RJMCMC selects the model from a set of the models suggested by better fitting, standard deviation errors and the frequency of accepted data. Together with deep analysis of the classical Box-Jenkins modeling methodology the integration with MCMC algorithms has been focused through parameter estimation and model fitting of ARMA models. This helps to verify how well the MCMC algorithms can treat the ARMA models, by comparing the results with graphical method. It has been seen that the MCMC produced better results than the classical time series approach.

Evaluating End-User Training Effectiveness in ERP System Implementation

Today’s business world demands more and more internal and external integration and transparency among companies at all fields. Integrated ERP (enterprise resource planning) systems offer a possibility to improve business practices and procedures by providing a unified view on the business including all functions and departments. Due to the obvious benefits, the popularity of integrated ERP systems keeps growing. The implementation of ERP systems has however proven risky. The implementation projects tend to be long, extensive, and costly – and often they end up in a failure. Due to the significant task and role changes ERP implementation brings to almost everybody in the company, training has been identified as one of the most critical success factors of an ERP implementation. To ensure that the training is conducted in the most effective and successful manner, the training outcomes should be evaluated. So far, training evaluation has however gained only limited attention at most companies investing in different training programs. Uponor corporation has initiated a large ERP implementation and process harmonization program in 2004. Thousands of end-users have been trained during this project so far, and the work still continues until the project is completed in 2010. In this thesis, the evaluation of end-user training in Uponor’s ERP program is brought further from the current state of performing the basic participant satisfaction survey in the end of each class. The results show that in order to reach reliable training effectiveness evaluation results, not only the reaction towards training but also transfer of skills and attitudes and the final results of the training program should be evaluated.

Development of Calculation Procedures for Wastewater Treatment Technologies for a Training System

Efficient designs and operations of water and wastewater treatment systems are largely based on mathematical calculations. This even applies to training in the treatment systems. Therefore, it is necessary that calculation procedures are developed and computerised a priori for such applications to ensure effectiveness. This work was aimed at developing calculation procedures for gas stripping, depth filtration, ion exchange, chemical precipitation, and ozonation wastewater treatment technologies to include them in ED-WAVE, a portable computer based tool used in design, operations and training in wastewater treatment. The work involved a comprehensive online and offline study of research work and literature, and application of practical case studies to generate ED-WAVE compatible representations of the treatment technologies which were then uploaded into the tool.

Eye Fundus Image Analysis for Automatic Detection of Diabetic Retinopathy

Diabetes is a rapidly increasing worldwide problem which is characterised by defective metabolism of glucose that causes long-term dysfunction and failure of various organs. The most common complication of diabetes is diabetic retinopathy (DR), which is one of the primary causes of blindness and visual impairment in adults. The rapid increase of diabetes pushes the limits of the current DR screening capabilities for which the digital imaging of the eye fundus (retinal imaging), and automatic or semi-automatic image analysis algorithms provide a potential solution. In this work, the use of colour in the detection of diabetic retinopathy is statistically studied using a supervised algorithm based on one-class classification and Gaussian mixture model estimation. The presented algorithm distinguishes a certain diabetic lesion type from all other possible objects in eye fundus images by only estimating the probability density function of that certain lesion type. For the training and ground truth estimation, the algorithm combines manual annotations of several experts for which the best practices were experimentally selected. By assessing the algorithm’s performance while conducting experiments with the colour space selection, both illuminance and colour correction, and background class information, the use of colour in the detection of diabetic retinopathy was quantitatively evaluated. Another contribution of this work is the benchmarking framework for eye fundus image analysis algorithms needed for the development of the automatic DR detection algorithms. The benchmarking framework provides guidelines on how to construct a benchmarking database that comprises true patient images, ground truth, and an evaluation protocol. The evaluation is based on the standard receiver operating characteristics analysis and it follows the medical practice in the decision making providing protocols for image- and pixel-based evaluations. During the work, two public medical image databases with ground truth were published: DIARETDB0 and DIARETDB1. The framework, DR databases and the final algorithm, are made public in the web to set the baseline results for automatic detection of diabetic retinopathy. Although deviating from the general context of the thesis, a simple and effective optic disc localisation method is presented. The optic disc localisation is discussed, since normal eye fundus structures are fundamental in the characterisation of DR.

Training system for conceptual design and evaluation for wastewater treatment

Rising population, rapid urbanisation and growing industrialisation have severely stressed water quality and its availability in Malawi. In addition, financial and institutional problems and the expanding agro industry have aggravated this problem. The situation is worsened by depleting water resources and pollution from untreated sewage and industrial effluent. The increasing scarcity of clean water calls for the need for appropriate management of available water resources. There is also demand for a training system for conceptual design and evaluation for wastewater treatment in order to build the capacity for technical service providers and environmental practitioners in the country. It is predicted that Malawi will face a water stress situation by 2025. In the city of Blantyre, this situation is aggravated by the serious pollution threat from the grossly inadequate sewage treatment capacity. This capacity is only 23.5% of the wastewater being generated presently. In addition, limited or non-existent industrial effluent treatment has contributed to the severe water quality degradation. This situation poses a threat to the ecologically fragile and sensitive receiving water courses within the city. This water is used for domestic purposes further downstream. This manuscript outlines the legal and policy framework for wastewater treatment in Malawi. The manuscript also evaluates the existing wastewater treatment systems in Blantyre. This evaluation aims at determining if the effluent levels at the municipal plants conform to existing standards and guidelines and other associated policy and regulatory frameworks. The raw material at all the three municipal plants is sewage. The typical wastewater parameters are Biochemical Oxygen Demand (BOD5), Chemical Oxygen Demand (COD), and Total Suspended Solids (TSS). The treatment target is BOD5, COD, and TSS reduction. Typical wastewater parameters at the wastewater treatment plant at MDW&S textile and garments factory are BOD5 and COD. The treatment target is to reduce BOD5 and COD. The manuscript further evaluates a design approach of the three municipal wastewater treatment plants in the city and the wastewater treatment plant at Mapeto David Whitehead & Sons (MDW&S) textile and garments factory. This evaluation utilises case-based design and case-based reasoning principles in the ED-WAVE tool to determine if there is potential for the tool in Blantyre. The manuscript finally evaluates the technology selection process for appropriate wastewater treatment systems for the city of Blantyre. The criteria for selection of appropriate wastewater treatment systems are discussed. Decision support tools and the decision tree making process for technology selection are also discussed. Based on the treatment targets and design criteria at the eight cases evaluated in this manuscript in reference to similar cases in the ED-WAVE tool, this work confirms the practical use of case-based design and case-based reasoning principles in the ED-WAVE tool in the design and evaluation of wastewater treatment 6 systems in sub-Sahara Africa, using Blantyre, Malawi, as the case study area. After encountering a new situation, already collected decision scenarios (cases) are invoked and modified in order to arrive at a particular design alternative. What is necessary, however, is to appropriately modify the case arrived at through the Case Study Manager in order to come up with a design appropriate to the local situation taking into account technical, socio-economic and environmental aspects. This work provides a training system for conceptual design and evaluation for wastewater treatment.

The family school : the impact of a group training programme on overactive hard-to-manage preschool children and their parents

Föräldraskap upplevs som en utmanande uppgift i dag och det påstås att föräldrar oftare än förr skulle var i behov av råd och stöd beträffande barnuppfostran. Denna uppgift kan ytterligare försvåras om det i familjen finns ett hyperaktivt okoncentrerat barn att uppfostra. Detta arbete undersökte effekterna av ett kortvarigt gruppbaserat interventionsprogram benämnt Familjeskolan POP (Preschool Overactivity Programme). Familjeskolan är avsedd för familjer med barn i lekåldern, som visar beteendesvårigheter såsom ADHD (Attention Deficit Hyperactivity Disorder), ODD (Oppositional Deficit Disorder) eller CD (Conduct Disorder). Målet för Familjeskolan är att öka föräldrarnas kunskaper och självförtroende då de har ett krävande svårhanterligt barn att uppfostra. Familjeskolan strävar också till att reducera barns icke-önskvärda beteenden genom att öka deras sociala färdigheter och koncentrationsförmåga. Familjeskolan verkställdes i Helsingfors vid ADHD- centrets lokaliteter. 45 mödrar och deras barn från huvudstadsregionen deltog i denna undersökning. Av dessa deltog 33 i Familjeskola-programmet medan de 12 övriga bildade den s.k. kontrollgruppen. Undersökningsresultaten tyder på förbättringar beträffande både moderns och faderns föräldrakunskaper efter Familjeskola-interventionen. Det är att lägga märke till att enbart mödrar deltog i interventionsprogrammet. Efter programmet klarade mödrar enligt egen utsaga vardagen bättre. Speciellt hade de blivit bättre på att hantera barnens beteendesvårigheter och hyperaktivt okoncentrerat beteende. Resultaten påvisade också att programmet var effektivast för de mödrar som före Familjeskolan upplevde sig besitta ringa föräldrakunskaper. Mödrarna rapporterade en signifikant minskning i barnens totala beteendesvårigheter. Efter interventionen ansåg mödrarna att deras barn var mindre olydiga, hyperaktiva samt att deras beteendesvårigheter var lindrigare. Enligt dagvårdspersonalen hade barnens totala beteendesvårigheter och problem med koncentration och hyperaktivitet också minskat. Motsvarande förbättringar uppnåddes inte i kontrollgruppen. Resultaten från uppföljningsintervjun, visade också att barnens beteendeförändringar var bestående både hemma och i daghemmet. Både föräldrar och dagvårdspersonalen rapporterade en signifikant minskning i barnens totala svårigheter jämfört med innan familjerna påbörjade interventionen. Föräldrarna rapporterade en marginell minskning i barnens ADHD-liknande beteende, beteendesvårigheter och i svårigheter med kamrater, dagvårdspersonalen däremot rapporterade en signifikant minskning i barnens beteendesvårigheter, hyperaktivt/okoncentrerat beteende samt i svårigheter med kamrater mellan innan familjerna påbörjade interventionen och uppföljningen ett år efter. Resultaten av denna undersökning stödjer hypotesen att kortvariga gruppbaserade interventionsprogram kan åstadkomma permanenta förbättringar i föräldrakunskaper och barns beteende. Detta gäller främst hyperaktivitet, koncentrationssvårigheter och trotsighet.

Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.