A Sparse Bayesian Approach for Joint Feature Selection and Classifier Learning

Peer-reviewed

Opinion Mining

In this thesis we study the field of opinion mining by giving a comprehensive review of the available research that has been done in this topic. Also using this available knowledge we present a case study of a multilevel opinion mining system for a student organization's sales management system. We describe the field of opinion mining by discussing its historical roots, its motivations and applications as well as the different scientific approaches that have been used to solve this challenging problem of mining opinions. To deal with this huge subfield of natural language processing, we first give an abstraction of the problem of opinion mining and describe the theoretical frameworks that are available for dealing with appraisal language. Then we discuss the relation between opinion mining and computational linguistics which is a crucial pre-processing step for the accuracy of the subsequent steps of opinion mining. The second part of our thesis deals with the semantics of opinions where we describe the different ways used to collect lists of opinion words as well as the methods and techniques available for extracting knowledge from opinions present in unstructured textual data. In the part about collecting lists of opinion words we describe manual, semi manual and automatic ways to do so and give a review of the available lists that are used as gold standards in opinion mining research. For the methods and techniques of opinion mining we divide the task into three levels that are the document, sentence and feature level. The techniques that are presented in the document and sentence level are divided into supervised and unsupervised approaches that are used to determine the subjectivity and polarity of texts and sentences at these levels of analysis. At the feature level we give a description of the techniques available for finding the opinion targets, the polarity of the opinions about these opinion targets and the opinion holders. Also at the feature level we discuss the various ways to summarize and visualize the results of this level of analysis. In the third part of our thesis we present a case study of a sales management system that uses free form text and that can benefit from an opinion mining system. Using the knowledge gathered in the review of this field we provide a theoretical multi level opinion mining system (MLOM) that can perform most of the tasks needed from an opinion mining system. Based on the previous research we give some hints that many of the laborious market research tasks that are done by the sales force, which uses this sales management system, can improve their insight about their partners and by that increase the quality of their sales services and their overall results.

Design and evaluation of learning algorithms for dynamic resource management in virtual networks

Network virtualisation is considerably gaining attentionas a solution to ossification of the Internet. However, thesuccess of network virtualisation will depend in part on how efficientlythe virtual networks utilise substrate network resources.In this paper, we propose a machine learning-based approachto virtual network resource management. We propose to modelthe substrate network as a decentralised system and introducea learning algorithm in each substrate node and substrate link,providing self-organization capabilities. We propose a multiagentlearning algorithm that carries out the substrate network resourcemanagement in a coordinated and decentralised way. The taskof these agents is to use evaluative feedback to learn an optimalpolicy so as to dynamically allocate network resources to virtualnodes and links. The agents ensure that while the virtual networkshave the resources they need at any given time, only the requiredresources are reserved for this purpose. Simulations show thatour dynamic approach significantly improves the virtual networkacceptance ratio and the maximum number of accepted virtualnetwork requests at any time while ensuring that virtual networkquality of service requirements such as packet drop rate andvirtual link delay are not affected.

Opetuksen kehittäminen Lappeenrannan teknillisen yliopiston tuotantotalouden osastolla - työtä, leipää ja sirkushuveja

Työn tavoitteena oli koota Lappeenrannan teknillisen yliopiston tuotantotalouden osaston opetuksen kehittämisen historiatiedot ja henkilöstön mielipide vuosien 2000–2008 aikana tehdystä kehittämistyöstä organisaation jatkokehittämistä ja viestintää varten. Työn aihepiiri käsittelee organisaation kehittämistä (kehittymistä) yksilön ja organisaation oppimisen näkökulmasta. Historiatiedot on kerätty osaston henkilökunnalta ja saaduista dokumenteista. Henkilöstön palautteen kerääminen kehittämistyöstä toteutettiin laadullisena tutkimuksena suorittamalla 32 henkilökohtaista haastattelua. Tutkimuksen keskeisimmäksi tulokseksi on saatu onnistuneen kehittämisen malli, jossa yksilön ja organisaation kehittymiseen vaikuttavat vahvasti yksilötasolla tarve ja konkreettinen päämäärä. Koko organisaatiossa on lisäksi huomioitava muina osina systematiikka, yhteisöllisyys ja tietämyksen (osaamisen) hallinta.

Separation of Gas Bubbles from Circulation Waters and Pulp Suspensions in Open Channel Flow

The objective of this thesis was to study the removal of gases from paper mill circulation waters experimentally and to provide data for CFD modeling. Flow and bubble size measurements were carried out in a laboratory scale open gas separation channel. Particle Image Velocimetry (PIV) technique was used to measure the gas and liquid flow fields, while bubble size measurements were conducted using digital imaging technique with back light illumination. Samples of paper machine waters as well as a model solution were used for the experiments. The PIV results show that the gas bubbles near the feed position have the tendency to escape from the circulation channel at a faster rate than those bubbles which are further away from the feed position. This was due to an increased rate of bubble coalescence as a result of the relatively larger bubbles near the feed position. Moreover, a close similarity between the measured slip velocities of the paper mill waters and that of literature values was obtained. It was found that due to dilution of paper mill waters, the observed average bubble size was considerably large as compared to the average bubble sizes in real industrial pulp suspension and circulation waters. Among the studied solutions, the model solution has the highest average drag coefficient value due to its relatively high viscosity. The results were compared to a 2D steady sate CFD simulation model. A standard Euler-Euler k-ε turbulence model was used in the simulations. The channel free surface was modeled as a degassing boundary. From the drag models used in the simulations, the Grace drag model gave velocity fields closest to the experimental values. In general, the results obtained from experiments and CFD simulations are in good qualitative agreement.

High-speed solid-rotor induction machine – calculation program

The aim of this thesis is to utilize the technology developed at LUT and to provide an easy tool for high-speed solid-rotor induction machine preliminary design. Computer aided design tool MathCAD has been chosen as the environment for realizing the calculation program. Four versions of the design program have been made depending on the motor rotor type. The first rotor type is an axially slitted solid-rotor with steel end rings. The next one is an axially slitted solid-rotor with copper end rings. The third machine type is a solid rotor with deep, rectangular copper bars and end rings (squirrel cage). And the last one is a solid-rotor with round copper bars and end rings (squirrel cage). Each type of rotor has its own specialties but a general thread of design is common. This paper follows the structure of the calculating program and explains some features and formulas. The attention is concentrated on the difference between laminated and solid-rotor machine design principles. There is no deep analysis of the calculation ways are presented. References for all solution methods appearing during the design procedure are given for more detailed studying. This thesis pays respect to the latest innovations in solid-rotor machines theory. Rotor ends’ analytical calculation follows the latest knowledge in this field. Correction factor for adjusting the rotor impedance is implemented. The purpose of the created design program is to calculate the preliminary dimensions of the machine according to initial data. Obtained results are not recommended for exact machine development. Further more detailed design should be done in a finite element method application. Hence, this thesis is a practical tool for the prior evaluating of the high-speed machine with different solid-rotor types parameters.

Design and Control of Permanent Magnet Synchronous Machine for Hybrid Electric Vehicle

This master’s thesis mainly focuses on the design requirements of an Electric drive for Hybrid car application and its control strategy to achieve a wide speed range. It also emphasises how the control and performance requirements are transformed into its design variables. A parallel hybrid topology is considered where an IC engine and an electric drive share a common crank shaft. A permanent magnet synchronous machine (PMSM) is used as an electric drive machine. Performance requirements are converted into Machine design variables using the vector model of PMSM. Main dimensions of the machine are arrived using analytical approach and Finite Element Analysis (FEA) is used to verify the design and performance. Vector control algorithm was used to control the machine. The control algorithm was tested in a low power PMSM using an embedded controller. A prototype of 10 kW PMSM was built according to the design values. The prototype was tested in the laboratory using a high power converter. Tests were carried out to verify different operating modes. The results were in agreement with the calculations.

Learning Grasp Affordances from Vision

Monimutkaisissa ja muuttuvissa ympäristöissä työskentelevät robotit tarvitsevat kykyä manipuloida ja tarttua esineisiin. Tämä työ tutkii robottitarttumisen ja robottitartuntapis-teiden koneoppimisen aiempaa tutkimusta ja nykytilaa. Nykyaikaiset menetelmät käydään läpi, ja Le:n koneoppimiseen pohjautuva luokitin toteutetaan, koska se tarjoaa parhaan onnistumisprosentin tutkituista menetelmistä ja on muokattavissa sopivaksi käytettävissä olevalle robotille. Toteutettu menetelmä käyttää intensititeettikuvaan ja syvyyskuvaan po-hjautuvia ominaisuuksi luokitellakseen potentiaaliset tartuntapisteet. Tämän toteutuksen tulokset esitellään.

Adaptive Markov chain Monte Carlo and Bayesian filtering for state space models

This thesis is concerned with the state and parameter estimation in state space models. The estimation of states and parameters is an important task when mathematical modeling is applied to many different application areas such as the global positioning systems, target tracking, navigation, brain imaging, spread of infectious diseases, biological processes, telecommunications, audio signal processing, stochastic optimal control, machine learning, and physical systems. In Bayesian settings, the estimation of states or parameters amounts to computation of the posterior probability density function. Except for a very restricted number of models, it is impossible to compute this density function in a closed form. Hence, we need approximation methods. A state estimation problem involves estimating the states (latent variables) that are not directly observed in the output of the system. In this thesis, we use the Kalman filter, extended Kalman filter, Gauss–Hermite filters, and particle filters to estimate the states based on available measurements. Among these filters, particle filters are numerical methods for approximating the filtering distributions of non-linear non-Gaussian state space models via Monte Carlo. The performance of a particle filter heavily depends on the chosen importance distribution. For instance, inappropriate choice of the importance distribution can lead to the failure of convergence of the particle filter algorithm. In this thesis, we analyze the theoretical Lᵖ particle filter convergence with general importance distributions, where p ≥2 is an integer. A parameter estimation problem is considered with inferring the model parameters from measurements. For high-dimensional complex models, estimation of parameters can be done by Markov chain Monte Carlo (MCMC) methods. In its operation, the MCMC method requires the unnormalized posterior distribution of the parameters and a proposal distribution. In this thesis, we show how the posterior density function of the parameters of a state space model can be computed by filtering based methods, where the states are integrated out. This type of computation is then applied to estimate parameters of stochastic differential equations. Furthermore, we compute the partial derivatives of the log-posterior density function and use the hybrid Monte Carlo and scaled conjugate gradient methods to infer the parameters of stochastic differential equations. The computational efficiency of MCMC methods is highly depend on the chosen proposal distribution. A commonly used proposal distribution is Gaussian. In this kind of proposal, the covariance matrix must be well tuned. To tune it, adaptive MCMC methods can be used. In this thesis, we propose a new way of updating the covariance matrix using the variational Bayesian adaptive Kalman filter algorithm.

Teacher professional growth in an authentic learning environment

The last several decades have been marked by tremendous changes in education - technological, pedagogical, administrative, and social. These changes have led to considerable increments in the budgets devoted to professional development for teachers ~ with the express purpose of helping them accommodate their practices to the new realities oftheir classrooms. However, research has suggested that, in spite of the emphasis placed on encouraging sustained change in teaching practices, little has been accomplished. This begs the question of what ought to be done to not only reverse this outcome, but contribute to transformational change. The literature suggests some possibilities including: a) considering teachers as learners and applying what, is known about cognition and learning; b) modifying the location and nature ofprofessional development so that it is authentic, based in the classroom and focusing on tasks meaningful to the teacher; c) attending to the infrastructure underlying professional development; and d) ensuring opportunities for reflective practice. This dissertation looks at the impact of each ofthese variables through an analysis ofthe learning journeys of a group ofteachers engaged in a program called GrassRoots in one midsized school board in Ontario. Action research was conducted by the researcher in his role as consultant facilitating teacher professional growth around the use of Web sites as culminating performance tasks by students. Research focused on the pedagogical approach to the learning of the teachers involved and the infrastructure underlying their learning. Using grounded theory, a model for professional development was developed that can be used in the future to inform practices and, hopefully, lead to sustained transformational school change.

Comparison of classification ability of hyperball algorithms to neural network and k-nearest neighbour algorithms

The main focus of this thesis is to evaluate and compare Hyperbalilearning algorithm (HBL) to other learning algorithms. In this work HBL is compared to feed forward artificial neural networks using back propagation learning, K-nearest neighbor and 103 algorithms. In order to evaluate the similarity of these algorithms, we carried out three experiments using nine benchmark data sets from UCI machine learning repository. The first experiment compares HBL to other algorithms when sample size of dataset is changing. The second experiment compares HBL to other algorithms when dimensionality of data changes. The last experiment compares HBL to other algorithms according to the level of agreement to data target values. Our observations in general showed, considering classification accuracy as a measure, HBL is performing as good as most ANn variants. Additionally, we also deduced that HBL.:s classification accuracy outperforms 103's and K-nearest neighbour's for the selected data sets.

Exploring Reciprocity in International Service Learning Programs

Using aspects of grounded theory methodology, this study explored the perceptions and practical implementation of reciprocity in International Service Learning (ISL) Programs. Data were collected through interviews with nine ISL practitioners representing a variety of organizations offering international service learning programs. Findings suggest that multiple conceptualizations of ISL programs exist. ISL programs are interdisciplinary in nature and that using reciprocity as a guiding framework is problematic. Further attention is needed in relation to shifting the guiding framework of ISL programs from reciprocity to interdependence.

Feature Selection and Classification Using Age Layered Population Structure Genetic Programming

The curse of dimensionality is a major problem in the fields of machine learning, data mining and knowledge discovery. Exhaustive search for the most optimal subset of relevant features from a high dimensional dataset is NP hard. Sub–optimal population based stochastic algorithms such as GP and GA are good choices for searching through large search spaces, and are usually more feasible than exhaustive and deterministic search algorithms. On the other hand, population based stochastic algorithms often suffer from premature convergence on mediocre sub–optimal solutions. The Age Layered Population Structure (ALPS) is a novel metaheuristic for overcoming the problem of premature convergence in evolutionary algorithms, and for improving search in the fitness landscape. The ALPS paradigm uses an age–measure to control breeding and competition between individuals in the population. This thesis uses a modification of the ALPS GP strategy called Feature Selection ALPS (FSALPS) for feature subset selection and classification of varied supervised learning tasks. FSALPS uses a novel frequency count system to rank features in the GP population based on evolved feature frequencies. The ranked features are translated into probabilities, which are used to control evolutionary processes such as terminal–symbol selection for the construction of GP trees/sub-trees. The FSALPS metaheuristic continuously refines the feature subset selection process whiles simultaneously evolving efficient classifiers through a non–converging evolutionary process that favors selection of features with high discrimination of class labels. We investigated and compared the performance of canonical GP, ALPS and FSALPS on high–dimensional benchmark classification datasets, including a hyperspectral image. Using Tukey’s HSD ANOVA test at a 95% confidence interval, ALPS and FSALPS dominated canonical GP in evolving smaller but efficient trees with less bloat expressions. FSALPS significantly outperformed canonical GP and ALPS and some reported feature selection strategies in related literature on dimensionality reduction.

Feature Selection and Classification Using Age Layered Population Structure Genetic Programming

The curse of dimensionality is a major problem in the fields of machine learning, data mining and knowledge discovery. Exhaustive search for the most optimal subset of relevant features from a high dimensional dataset is NP hard. Sub–optimal population based stochastic algorithms such as GP and GA are good choices for searching through large search spaces, and are usually more feasible than exhaustive and determinis- tic search algorithms. On the other hand, population based stochastic algorithms often suffer from premature convergence on mediocre sub–optimal solutions. The Age Layered Population Structure (ALPS) is a novel meta–heuristic for overcoming the problem of premature convergence in evolutionary algorithms, and for improving search in the fitness landscape. The ALPS paradigm uses an age–measure to control breeding and competition between individuals in the population. This thesis uses a modification of the ALPS GP strategy called Feature Selection ALPS (FSALPS) for feature subset selection and classification of varied supervised learning tasks. FSALPS uses a novel frequency count system to rank features in the GP population based on evolved feature frequencies. The ranked features are translated into probabilities, which are used to control evolutionary processes such as terminal–symbol selection for the construction of GP trees/sub-trees. The FSALPS meta–heuristic continuously refines the feature subset selection process whiles simultaneously evolving efficient classifiers through a non–converging evolutionary process that favors selection of features with high discrimination of class labels. We investigated and compared the performance of canonical GP, ALPS and FSALPS on high–dimensional benchmark classification datasets, including a hyperspectral image. Using Tukey’s HSD ANOVA test at a 95% confidence interval, ALPS and FSALPS dominated canonical GP in evolving smaller but efficient trees with less bloat expressions. FSALPS significantly outperformed canonical GP and ALPS and some reported feature selection strategies in related literature on dimensionality reduction.