Learning Preferences with Kernel-Based Methods

Learning of preference relations has recently received significant attention in machine learning community. It is closely related to the classification and regression analysis and can be reduced to these tasks. However, preference learning involves prediction of ordering of the data points rather than prediction of a single numerical value as in case of regression or a class label as in case of classification. Therefore, studying preference relations within a separate framework facilitates not only better theoretical understanding of the problem, but also motivates development of the efficient algorithms for the task. Preference learning has many applications in domains such as information retrieval, bioinformatics, natural language processing, etc. For example, algorithms that learn to rank are frequently used in search engines for ordering documents retrieved by the query. Preference learning methods have been also applied to collaborative filtering problems for predicting individual customer choices from the vast amount of user generated feedback. In this thesis we propose several algorithms for learning preference relations. These algorithms stem from well founded and robust class of regularized least-squares methods and have many attractive computational properties. In order to improve the performance of our methods, we introduce several non-linear kernel functions. Thus, contribution of this thesis is twofold: kernel functions for structured data that are used to take advantage of various non-vectorial data representations and the preference learning algorithms that are suitable for different tasks, namely efficient learning of preference relations, learning with large amount of training data, and semi-supervised preference learning. Proposed kernel-based algorithms and kernels are applied to the parse ranking task in natural language processing, document ranking in information retrieval, and remote homology detection in bioinformatics domain. Training of kernel-based ranking algorithms can be infeasible when the size of the training set is large. This problem is addressed by proposing a preference learning algorithm whose computation complexity scales linearly with the number of training data points. We also introduce sparse approximation of the algorithm that can be efficiently trained with large amount of data. For situations when small amount of labeled data but a large amount of unlabeled data is available, we propose a co-regularized preference learning algorithm. To conclude, the methods presented in this thesis address not only the problem of the efficient training of the algorithms but also fast regularization parameter selection, multiple output prediction, and cross-validation. Furthermore, proposed algorithms lead to notably better performance in many preference learning tasks considered.

Intelligent Internet Data Searching

In this thesis author approaches the problem of automated text classification, which is one of basic tasks for building Intelligent Internet Search Agent. The work discusses various approaches to solving sub-problems of automated text classification, such as feature extraction and machine learning on text sources. Author also describes her own multiword approach to feature extraction and pres-ents the results of testing this approach using linear discriminant analysis based classifier, and classifier combining unsupervised learning for etalon extraction with supervised learning using common backpropagation algorithm for multilevel perceptron.

Analysis of Satellite Data for Forest Segmentation

In this work we study the classification of forest types using mathematics based image analysis on satellite data. We are interested in improving classification of forest segments when a combination of information from two or more different satellites is used. The experimental part is based on real satellite data originating from Canada. This thesis gives summary of the mathematics basics of the image analysis and supervised learning , methods that are used in the classification algorithm. Three data sets and four feature sets were investigated in this thesis. The considered feature sets were 1) histograms (quantiles) 2) variance 3) skewness and 4) kurtosis. Good overall performances were achieved when a combination of ASTERBAND and RADARSAT2 data sets was used.

Deep generative models for facial keypoints detection

A new area of machine learning research called deep learning, has moved machine learning closer to one of its original goals: artificial intelligence and general learning algorithm. The key idea is to pretrain models in completely unsupervised way and finally they can be fine-tuned for the task at hand using supervised learning. In this thesis, a general introduction to deep learning models and algorithms are given and these methods are applied to facial keypoints detection. The task is to predict the positions of 15 keypoints on grayscale face images. Each predicted keypoint is specified by an (x,y) real-valued pair in the space of pixel indices. In experiments, we pretrained deep belief networks (DBN) and finally performed a discriminative fine-tuning. We varied the depth and size of an architecture. We tested both deterministic and sampled hidden activations and the effect of additional unlabeled data on pretraining. The experimental results show that our model provides better results than publicly available benchmarks for the dataset.

Pretraining Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks (CNN) have become the state-of-the-art methods on many large scale visual recognition tasks. For a lot of practical applications, CNN architectures have a restrictive requirement: A huge amount of labeled data are needed for training. The idea of generative pretraining is to obtain initial weights of the network by training the network in a completely unsupervised way and then fine-tune the weights for the task at hand using supervised learning. In this thesis, a general introduction to Deep Neural Networks and algorithms are given and these methods are applied to classification tasks of handwritten digits and natural images for developing unsupervised feature learning. The goal of this thesis is to find out if the effect of pretraining is damped by recent practical advances in optimization and regularization of CNN. The experimental results show that pretraining is still a substantial regularizer, however, not a necessary step in training Convolutional Neural Networks with rectified activations. On handwritten digits, the proposed pretraining model achieved a classification accuracy comparable to the state-of-the-art methods.

Sentence Compression with Supervised Machine Learning

The subject of the thesis is automatic sentence compression with machine learning, so that the compressed sentences remain both grammatical and retain their essential meaning. There are multiple possible uses for the compression of natural language sentences. In this thesis the focus is generation of television program subtitles, which often are compressed version of the original script of the program. The main part of the thesis consists of machine learning experiments for automatic sentence compression using different approaches to the problem. The machine learning methods used for this work are linear-chain conditional random fields and support vector machines. Also we take a look which automatic text analysis methods provide useful features for the task. The data used for machine learning is supplied by Lingsoft Inc. and consists of subtitles in both compressed an uncompressed form. The models are compared to a baseline system and comparisons are made both automatically and also using human evaluation, because of the potentially subjective nature of the output. The best result is achieved using a CRF - sequence classification using a rich feature set. All text analysis methods help classification and most useful method is morphological analysis. Tutkielman aihe on suomenkielisten lauseiden automaattinen tiivistäminen koneellisesti, niin että lyhennetyt lauseet säilyttävät olennaisen informaationsa ja pysyvät kieliopillisina. Luonnollisen kielen lauseiden tiivistämiselle on monta käyttötarkoitusta, mutta tässä tutkielmassa aihetta lähestytään television ohjelmien tekstittämisen kautta, johon käytännössä kuuluu alkuperäisen tekstin lyhentäminen televisioruudulle paremmin sopivaksi. Tutkielmassa kokeillaan erilaisia koneoppimismenetelmiä tekstin automaatiseen lyhentämiseen ja tarkastellaan miten hyvin erilaiset luonnollisen kielen analyysimenetelmät tuottavat informaatiota, joka auttaa näitä menetelmiä lyhentämään lauseita. Lisäksi tarkastellaan minkälainen lähestymistapa tuottaa parhaan lopputuloksen. Käytetyt koneoppimismenetelmät ovat tukivektorikone ja lineaarisen sekvenssin mallinen CRF. Koneoppimisen tukena käytetään tekstityksiä niiden eri käsittelyvaiheissa, jotka on saatu Lingsoft OY:ltä. Luotuja malleja vertaillaan Lopulta mallien lopputuloksia evaluoidaan automaattisesti ja koska teksti lopputuksena on jossain määrin subjektiivinen myös ihmisarviointiin perustuen. Vertailukohtana toimii kirjallisuudesta poimittu menetelmä. Tutkielman tuloksena paras lopputulos saadaan aikaan käyttäen CRF sekvenssi-luokittelijaa laajalla piirrejoukolla. Kaikki kokeillut teksin analyysimenetelmät auttavat luokittelussa, joista tärkeimmän panoksen antaa morfologinen analyysi.

Supervised localimage feature detection

This thesis is about detection of local image features. The research topic belongs to the wider area of object detection, which is a machine vision and pattern recognition problem where an object must be detected (located) in an image. State-of-the-art object detection methods often divide the problem into separate interest point detection and local image description steps, but in this thesis a different technique is used, leading to higher quality image features which enable more precise localization. Instead of using interest point detection the landmark positions are marked manually. Therefore, the quality of the image features is not limited by the interest point detection phase and the learning of image features is simplified. The approach combines both interest point detection and local description into one phase for detection. Computational efficiency of the descriptor is therefore important, leaving out many of the commonly used descriptors as unsuitably heavy. Multiresolution Gabor features has been the main descriptor in this thesis and improving their efficiency is a significant part. Actual image features are formed from descriptors by using a classifierwhich can then recognize similar looking patches in new images. The main classifier is based on Gaussian mixture models. Classifiers are used in one-class classifier configuration where there are only positive training samples without explicit background class. The local image feature detection method has been tested with two freely available face detection databases and a proprietary license plate database. The localization performance was very good in these experiments. Other applications applying the same under-lying techniques are also presented, including object categorization and fault detection.

Nursing student-patient relationship and associated factors

The purpose of this study was to analyse the nursing student-patient relationship and factors associated with this relationship from the point of view of both students and patients, and to identify factors that predict the type of relationship. The ultimate goal is to improve supervised clinical practicum with a view to supporting students in their reciprocal collaborative relationships with patients, increase their preparedness to meet patients’ health needs, and thus to enhance the quality of patient care. The study was divided into two phases. In the first phase (1999-2005), a literature review concerning the student-patient relationship was conducted (n=104 articles) and semi-structured interviews carried out with nursing students (n=30) and internal medicine patients (n=30). Data analysis was by means of qualitative content analysis and Student-Patient Relationship Scales, which were specially developed for this research. In the second phase (2005-2007), the data were collected by SPR scales among nursing students (n=290) and internal medicine patients (n=242). The data were analysed statistically by SPSS 12.0 software. The results revealed three types of student-patient relationship: a mechanistic relationship focusing on the student’s learning needs; an authoritative relationship focusing on what the student assumes is in the patient’s best interest; and a facilitative relationship focusing on the common good of both student and patient. Students viewed their relationship with patients more often as facilitative and authoritative than mechanistic, while in patients’ assessments the authoritative relationship occurred most frequently and the facilitative relationship least frequently. Furthermore, students’ and patients’ views on their relationships differed significantly. A number of background factors, contextual factors and consequences of the relationship were found to be associated with the type of relationship. In the student data, factors that predicted the type of relationship were age, current year of study and support received in the relationship with patient. The higher the student’s age, the more likely the relationship with the patient was facilitative. Fourth year studies and the support of a person other than a supervisor were significantly associated with an authoritative relationship. Among patients, several factors were found to predict the type of nursing student-patient relationships. Significant factors associated with a facilitative relationship were university-level education, several previous hospitalizations, admission to hospital for a medical problem, experience of caring for an ill family member and patient’s positive perception of atmosphere during collaboration and of student’s personal and professional growth. In patients, positive perceptions of student’s personal and professional attributes and patient’s improved health and a greater commitment to self-care, on the other hand, were significantly associated with an authoritative relationship, whereas positive perceptions of one’s own attributes as a patient were significantly associated with a mechanistic relationship. It is recommended that further research on the student-patient relationship and related factors should focus on questions of content, methodology and education.

Opinion Mining

In this thesis we study the field of opinion mining by giving a comprehensive review of the available research that has been done in this topic. Also using this available knowledge we present a case study of a multilevel opinion mining system for a student organization's sales management system. We describe the field of opinion mining by discussing its historical roots, its motivations and applications as well as the different scientific approaches that have been used to solve this challenging problem of mining opinions. To deal with this huge subfield of natural language processing, we first give an abstraction of the problem of opinion mining and describe the theoretical frameworks that are available for dealing with appraisal language. Then we discuss the relation between opinion mining and computational linguistics which is a crucial pre-processing step for the accuracy of the subsequent steps of opinion mining. The second part of our thesis deals with the semantics of opinions where we describe the different ways used to collect lists of opinion words as well as the methods and techniques available for extracting knowledge from opinions present in unstructured textual data. In the part about collecting lists of opinion words we describe manual, semi manual and automatic ways to do so and give a review of the available lists that are used as gold standards in opinion mining research. For the methods and techniques of opinion mining we divide the task into three levels that are the document, sentence and feature level. The techniques that are presented in the document and sentence level are divided into supervised and unsupervised approaches that are used to determine the subjectivity and polarity of texts and sentences at these levels of analysis. At the feature level we give a description of the techniques available for finding the opinion targets, the polarity of the opinions about these opinion targets and the opinion holders. Also at the feature level we discuss the various ways to summarize and visualize the results of this level of analysis. In the third part of our thesis we present a case study of a sales management system that uses free form text and that can benefit from an opinion mining system. Using the knowledge gathered in the review of this field we provide a theoretical multi level opinion mining system (MLOM) that can perform most of the tasks needed from an opinion mining system. Based on the previous research we give some hints that many of the laborious market research tasks that are done by the sales force, which uses this sales management system, can improve their insight about their partners and by that increase the quality of their sales services and their overall results.

Inter organizational learning within and by innovation networks

The thesis deals with the phenomenon of learning between organizations in innovation networks that develop new products, services or processes. Inter organizational learning is studied especially at the level of the network. The role of the network can be seen as twofold: either the network is a context for inter organizational learning, if the learner is something else than the network (organization, group, individual), or the network itself is the learner. Innovations are regarded as a primary source of competitiveness and renewal in organizations. Networking has become increasingly common particularly because of the possibility to extend the resource base of the organization through partnerships and to concentrate on core competencies. Especially in innovation activities, networks provide the possibility to answer the complex needs of the customers faster and to share the costs and risks of the development work. Networked innovation activities are often organized in practice as distributed virtual teams, either within one organization or as cross organizational co operation. The role of technology is considered in the research mainly as an enabling tool for collaboration and learning. Learning has been recognized as one important collaborative process in networks or as a motivation for networking. It is even more important in the innovation context as an enabler of renewal, since the essence of the innovation process is creating new knowledge, processes, products and services. The thesis aims at providing enhanced understanding about the inter organizational learning phenomenon in and by innovation networks, especially concentrating on the network level. The perspectives used in the research are the theoretical viewpoints and concepts, challenges, and solutions for learning. The methods used in the study are literature reviews and empirical research carried out with semi structured interviews analyzed with qualitative content analysis. The empirical research concentrates on two different areas, firstly on the theoretical approaches to learning that are relevant to innovation networks, secondly on learning in virtual innovation teams. As a result, the research identifies insights and implications for learning in innovation networks from several viewpoints on organizational learning. Using multiple perspectives allows drawing a many sided picture of the learning phenomenon that is valuable because of the versatility and complexity of situations and challenges of learning in the context of innovation and networks. The research results also show some of the challenges of learning and possible solutions for supporting especially network level learning.

Machine Learning and Clinical Text. Supporting Health Information Flow

Fluent health information flow is critical for clinical decision-making. However, a considerable part of this information is free-form text and inabilities to utilize it create risks to patient safety and cost-­effective hospital administration. Methods for automated processing of clinical text are emerging. The aim in this doctoral dissertation is to study machine learning and clinical text in order to support health information flow.First, by analyzing the content of authentic patient records, the aim is to specify clinical needs in order to guide the development of machine learning applications.The contributions are a model of the ideal information flow,a model of the problems and challenges in reality, and a road map for the technology development. Second, by developing applications for practical cases,the aim is to concretize ways to support health information flow. Altogether five machine learning applications for three practical cases are described: The first two applications are binary classification and regression related to the practical case of topic labeling and relevance ranking.The third and fourth application are supervised and unsupervised multi-class classification for the practical case of topic segmentation and labeling.These four applications are tested with Finnish intensive care patient records.The fifth application is multi-label classification for the practical task of diagnosis coding. It is tested with English radiology reports.The performance of all these applications is promising. Third, the aim is to study how the quality of machine learning applications can be reliably evaluated.The associations between performance evaluation measures and methods are addressed,and a new hold-out method is introduced.This method contributes not only to processing time but also to the evaluation diversity and quality. The main conclusion is that developing machine learning applications for text requires interdisciplinary, international collaboration. Practical cases are very different, and hence the development must begin from genuine user needs and domain expertise. The technological expertise must cover linguistics,machine learning, and information systems. Finally, the methods must be evaluated both statistically and through authentic user-feedback.

Towards adoption of eLearning technologies at Lappeenranta University of Technology: Understanding Teachers‟ Adoption of Blackboard Learning System

The electronic learning has become crucial in higher education with increased usage of learning management systems as a key source of integration on distance learning. The objective of this study is to understand how university teachers are influenced to use and adopt web-based learning management systems. Blackboard, as one of the systems used internationally by various universities is applied as a case. Semi-structured interviews were made with professors and lecturers who are using Blackboard at Lappeenranta University of Technology. The data collected were categorized under constructs adapted from Unified Theory of Acceptance and Use of Technology (UTAUT) and interpretation and discussion were based on reviewed literature. The findings suggest that adoption of learning management systems by LUT teachers is highly influenced by perceived usefulness, facilitating conditions and gained experience. The findings also suggest that easiness of using the system and social influence appear as medium influence of adoption for teachers at LUT.

Saksan opiskelijasta saksan opettajaksi. Aineenopettajaopiskelijan ammatillinen kasvu ohjatussa harjoitttelussa

Tutkimuksessani tarkastelin, miten ammatillinen kasvu ilmenee saksan opetusharjoittelijoiden näkemyksissä ja toiminnassa opettajan pedagogisiin opintoihin kuuluvan ohjatun harjoittelun aikana. Keräsin tutkimusaineiston lukuvuosina 2007–2010 kaikilta saksan opetusharjoittelijoilta, jotka suorittivat ohjatussa harjoittelussa vähintään 15 op Turun normaalikoulussa. Tutkimusaineistona käytin henkilökohtaisia harjoittelusuunnitelmia (HOPS) ja reflektiovihkoja, jotka ovat ohjatun harjoittelun normaaleja työvälineitä. Täydensin aineistoa ohjatun harjoittelun alussa toteutetulla kyselyllä ja harjoittelun päätteeksi tehdyllä puolistrukturoidulla teemahaastattelulla. Toimin tutkimuksen aikana Turun normaalikoulussa saksan opettajana ja aineryhmän harjoittelusta vastaavana opettajana. Tutkimuskysymykset tarkentuivat aineistolähtöisesti. Ensimmäinen tutkimuskysymys liittyi siihen, miten harjoittelijoiden ideaalit hyvästä vieraan kielen opetuksesta ja henkilökohtaiset tavoitteet toteutuivat ohjatun harjoittelun aikana. Alkukyselyn pohjalta ideaaleiksi nousivat vuorovaikutus ja monipuoliset työtavat, kulttuurin opettaminen, tavoitekielen käyttö luokkakielenä sekä selkeä kieliopin opetus. Tutkimuksessa kävi ilmi, että alkukyselyssä esiin tullut hyvän opettajan tai vieraan kielen opetuksen ideaali ei välttämättä näy opettajaksi opiskelevan HOPSeihin kirjatuissa henkilökohtaisissa tavoitteissa tai toteudu hänen harjoitustunneillaan. Parhaiten opetusharjoittelijat kokivat onnistuneensa vuorovaikutuksen luomisessa oppilaisiin ja opiskelijoihin sekä monipuolisten työtapojen käytössä. Eriyttäminen ja oppimaan oppimisen ohjaaminen koettiin hankalina. Suurin osa harjoittelijoista oli tyytyväisiä siihen, miten he onnistuivat tuomaan kulttuuria opetukseensa, kun taas tavoitekielen käyttö luokkakielenä ja kieliopin opetus koettiin haasteellisiksi. Toisessa tutkimuskysymyksessä tarkasteltiin, millainen ammatillinen näkemys vieraan kielen opiskelijalla on ohjatun harjoittelun jälkeen. Opetusharjoittelijat korostivat opettajan kasvatustyötä ja opettajien välistä yhteistyötä. Ajatus toimia saksan tai vieraan kielen opettajana oli vahvistunut pedagogisten opintojen aikana. Kolmas tutkimuskysymys kohdistui sen selvittämiseen, miten HOPS ja reflektiovihko toimivat harjoittelijoiden kasvun tukena. Harjoittelijoiden kirjallinen itsereflektio vaihteli syvällisistä pohdinnoista niukkiin merkintöihin. Suurimmassa osassa reflektiovihoista harjoittelijat olivat miettineet palautteissa esille tulleita asioita. HOPS ja reflektiovihko ovat tämän tutkimuksen perusteella toimivia harjoittelun ohjauksen välineitä, kunhan harjoittelijoita ohjataan niiden käytössä. Itsereflektion merkityksen puolesta puhuu se, että harjoittelijat, jotka olivat myös kirjallisesti pohtineet saksan käyttöä luokkakielenä, olivat muita tyytyväisempiä siihen, miten he kokivat onnistuneensa luokkakielen käytössä.

Evaluation of Machine Learning Classifiers for Mobile Network Intrusion Detection Systems

Mobile malwares are increasing with the growing number of Mobile users. Mobile malwares can perform several operations which lead to cybersecurity threats such as, stealing financial or personal information, installing malicious applications, sending premium SMS, creating backdoors, keylogging and crypto-ransomware attacks. Knowing the fact that there are many illegitimate Applications available on the App stores, most of the mobile users remain careless about the security of their Mobile devices and become the potential victim of these threats. Previous studies have shown that not every antivirus is capable of detecting all the threats; due to the fact that Mobile malwares use advance techniques to avoid detection. A Network-based IDS at the operator side will bring an extra layer of security to the subscribers and can detect many advanced threats by analyzing their traffic patterns. Machine Learning(ML) will provide the ability to these systems to detect unknown threats for which signatures are not yet known. This research is focused on the evaluation of Machine Learning classifiers in Network-based Intrusion detection systems for Mobile Networks. In this study, different techniques of Network-based intrusion detection with their advantages, disadvantages and state of the art in Hybrid solutions are discussed. Finally, a ML based NIDS is proposed which will work as a subsystem, to Network-based IDS deployed by Mobile Operators, that can help in detecting unknown threats and reducing false positives. In this research, several ML classifiers were implemented and evaluated. This study is focused on Android-based malwares, as Android is the most popular OS among users, hence most targeted by cyber criminals. Supervised ML algorithms based classifiers were built using the dataset which contained the labeled instances of relevant features. These features were extracted from the traffic generated by samples of several malware families and benign applications. These classifiers were able to detect malicious traffic patterns with the TPR upto 99.6% during Cross-validation test. Also, several experiments were conducted to detect unknown malware traffic and to detect false positives. These classifiers were able to detect unknown threats with the Accuracy of 97.5%. These classifiers could be integrated with current NIDS', which use signatures, statistical or knowledge-based techniques to detect malicious traffic. Technique to integrate the output from ML classifier with traditional NIDS is discussed and proposed for future work.

Fostering product innovation by learning through exporting in Eastern Europe

This study discusses the importance of learning through the process of exporting, and more specifically how such a process can enhance the product innovativeness of a company. The purpose of this study is to investigate the appropriate sources of learning and to suggest an interactive framework for how new knowledge from exporting markets can materialize itself into product innovation. The theoretical background of the study was constructed from academic literature, which is related to concepts of learning by exporting, along with sources for learning in the market and new product development. The empirical research in the form of a qualitative case study was based on four semi-structured interviews and secondary data from the case company official site. The interview data was collected between March and April 2015 from case company employees who directly work in the department of exporting and product development. The method of thematic analysis was used to categorize and interpret the collected data. What was conclusively discovered, was that the knowledge from an exporting market can be an incentive for product innovation, especially an incremental one. Foreign customers and competitors as important sources for new knowledge contribute to the innovative process. Foreign market competitors’ influence on product improvements was high only when the competitor was a market leader or held a colossal market share, while the customers’ influence is always high. Therefore, involving a foreign customer in the development of a new product is vital to a company that is interested in benefiting from what is learned through exporting. The interactive framework, which is based on the theoretical background and findings of the study, suggests that exporting companies can raise their product innovativeness by utilizing newly gained knowledge from exporting markets. Except for input, in the form of sources of learning, and product innovation as an output, the framework contains a process of knowledge transfer, the absorptive capacity of a firm and a new product development process. In addition, the framework and the findings enhance the understanding of the disputed relationship between an exporting experience and product innovation. However, future research is needed in order to fully understand all the elements of the framework, such as the absorptive capacity of a firm as well as more case companies to be processed in order to increase the generalization of the framework