New kernel functions and learning methods for text and data mining

Recent advances in machine learning methods enable increasingly the automatic construction of various types of computer assisted methods that have been difficult or laborious to program by human experts. The tasks for which this kind of tools are needed arise in many areas, here especially in the fields of bioinformatics and natural language processing. The machine learning methods may not work satisfactorily if they are not appropriately tailored to the task in question. However, their learning performance can often be improved by taking advantage of deeper insight of the application domain or the learning problem at hand. This thesis considers developing kernel-based learning algorithms incorporating this kind of prior knowledge of the task in question in an advantageous way. Moreover, computationally efficient algorithms for training the learning machines for specific tasks are presented. In the context of kernel-based learning methods, the incorporation of prior knowledge is often done by designing appropriate kernel functions. Another well-known way is to develop cost functions that fit to the task under consideration. For disambiguation tasks in natural language, we develop kernel functions that take account of the positional information and the mutual similarities of words. It is shown that the use of this information significantly improves the disambiguation performance of the learning machine. Further, we design a new cost function that is better suitable for the task of information retrieval and for more general ranking problems than the cost functions designed for regression and classification. We also consider other applications of the kernel-based learning algorithms such as text categorization, and pattern recognition in differential display. We develop computationally efficient algorithms for training the considered learning machines with the proposed kernel functions. We also design a fast cross-validation algorithm for regularized least-squares type of learning algorithm. Further, an efficient version of the regularized least-squares algorithm that can be used together with the new cost function for preference learning and ranking tasks is proposed. In summary, we demonstrate that the incorporation of prior knowledge is possible and beneficial, and novel advanced kernels and cost functions can be used in algorithms efficiently.

Boundary identification in the domain of a parabolic partial differential equation

Bakgrunden och inspirationen till föreliggande studie är tidigare forskning i tillämpningar på randidentifiering i metallindustrin. Effektiv randidentifiering möjliggör mindre säkerhetsmarginaler och längre serviceintervall för apparaturen i industriella högtemperaturprocesser, utan ökad risk för materielhaverier. I idealfallet vore en metod för randidentifiering baserad på uppföljning av någon indirekt variabel som kan mätas rutinmässigt eller till en ringa kostnad. En dylik variabel för smältugnar är temperaturen i olika positioner i väggen. Denna kan utnyttjas som insignal till en randidentifieringsmetod för att övervaka ugnens väggtjocklek. Vi ger en bakgrund och motivering till valet av den geometriskt endimensionella dynamiska modellen för randidentifiering, som diskuteras i arbetets senare del, framom en flerdimensionell geometrisk beskrivning. I de aktuella industriella tillämpningarna är dynamiken samt fördelarna med en enkel modellstruktur viktigare än exakt geometrisk beskrivning. Lösningsmetoder för den s.k. sidledes värmeledningsekvationen har många saker gemensamt med randidentifiering. Därför studerar vi egenskaper hos lösningarna till denna ekvation, inverkan av mätfel och något som brukar kallas förorening av mätbrus, regularisering och allmännare följder av icke-välställdheten hos sidledes värmeledningsekvationen. Vi studerar en uppsättning av tre olika metoder för randidentifiering, av vilka de två första är utvecklade från en strikt matematisk och den tredje från en mera tillämpad utgångspunkt. Metoderna har olika egenskaper med specifika fördelar och nackdelar. De rent matematiskt baserade metoderna karakteriseras av god noggrannhet och låg numerisk kostnad, dock till priset av låg flexibilitet i formuleringen av den modellbeskrivande partiella differentialekvationen. Den tredje, mera tillämpade, metoden kännetecknas av en sämre noggrannhet förorsakad av en högre grad av icke-välställdhet hos den mera flexibla modellen. För denna gjordes även en ansats till feluppskattning, som senare kunde observeras överensstämma med praktiska beräkningar med metoden. Studien kan anses vara en god startpunkt och matematisk bas för utveckling av industriella tillämpningar av randidentifiering, speciellt mot hantering av olinjära och diskontinuerliga materialegenskaper och plötsliga förändringar orsakade av “nedfallande” väggmaterial. Med de behandlade metoderna förefaller det möjligt att uppnå en robust, snabb och tillräckligt noggrann metod av begränsad komplexitet för randidentifiering.

On the differential evolution algorithm and its appliation to training radial basis function networks

The parameter setting of a differential evolution algorithm must meet several requirements: efficiency, effectiveness, and reliability. Problems vary. The solution of a particular problem can be represented in different ways. An algorithm most efficient in dealing with a particular representation may be less efficient in dealing with other representations. The development of differential evolution-based methods contributes substantially to research on evolutionary computing and global optimization in general. The objective of this study is to investigatethe differential evolution algorithm, the intelligent adjustment of its controlparameters, and its application. In the thesis, the differential evolution algorithm is first examined using different parameter settings and test functions. Fuzzy control is then employed to make control parameters adaptive based on an optimization process and expert knowledge. The developed algorithms are applied to training radial basis function networks for function approximation with possible variables including centers, widths, and weights of basis functions and both having control parameters kept fixed and adjusted by fuzzy controller. After the influence of control variables on the performance of the differential evolution algorithm was explored, an adaptive version of the differential evolution algorithm was developed and the differential evolution-based radial basis function network training approaches were proposed. Experimental results showed that the performance of the differential evolution algorithm is sensitive to parameter setting, and the best setting was found to be problem dependent. The fuzzy adaptive differential evolution algorithm releases the user load of parameter setting and performs better than those using all fixedparameters. Differential evolution-based approaches are effective for training Gaussian radial basis function networks.

Conceptual Design of Embedded Weather Display

The objective of this thesis work is to describe the Conceptual Design process of an embedded electronic display device. The work presents the following sub processes: definition of device specifications, introduction to the technological alternatives for system components and their comparison, comparative photometric measurements of selected display panels, and the design and building of a functional concept prototype. This work focuses mainly on electronics design, albeit the mechanical issues and fields of the software architecture that significantly affect the decisions are also discussed when necessary. The VESA Flat Panel Display Measurement (FPDM) 2.0 Standard was applied to the appropriate extent into photometric measurements. The results were analyzed against the requirement standards of a customer-specific display development project. An Active Matrix LCD was selected as the display of concept prototype, but also the excellent visual characteristics of Active Matrix OLED technology were noted. Should the reliability of the OLED products be significantly improved in the future, utilizing such products in the described application must be reconsidered.

Differential Regulation of c-FLIP Isoforms Through Post-translational Modifications

Cells are constantly responding to signals from the surrounding tissues and the environment. To dispose of infected and potentially dangerous cells, to ensure the optimal execution of developmental processes and to maintain tissue homeostasis, a multicellular organism needs to tightly control both the number and the quality of its cells. Apoptosis is a form of active cellular self-destruction that enables an organism to regulate its cell number by deleting damaged or potentially dangerous cells. Apoptosis can be induced by death ligands, which bind to death receptors on the cell surface. Ligation of the receptors leads to the formation of an intracellular death inducing signaling complex (DISC). One of the DISC components is caspase-8, a protease that triggers the caspase cascade and is thereby a key initiator of programmed cell death. The activation of caspase-8 is controlled by the cellular FLICE-inhibitory proteins (c-FLIPs). Consequently, sensitivity towards receptor-mediated apoptosis is determined by the amount of c-FLIP, and the c-FLIP levels are actively regulated for example during erythroid differentiation of K562 erythroleukemia cells and by hyperthermia in Jurkat leukemia cells. The aim of my thesis was to investigate how c-FLIP is regulated during these processes. We found that c-FLIP isoforms are short-lived proteins, although c-FLIPS had an even shorter half-life than c-FLIPL. In both experimental models, increased death receptor sensitivity correlated with induced ubiquitylation and consequent proteasomal degradation of c-FLIP. Furthermore, we elucidated how phosphorylation regulates the biological functions and the turnover of c-FLIP, thereby contributing to death receptor sensitivity. We mapped the first phosphorylation sites on c-FLIP and dissected how their phosphorylation affects c-FLIP. Moreover, we demonstrated that phosphorylation of serine 193, a phosphorylated residue common to all c-FLIPs, is primarily mediated by the classical PKC. Furthermore, we discovered a novel connection between the phosphorylation and ubiquitylation of c-FLIP: phosphorylation of S193 protects c-FLIP from ubiquitylation. Surprisingly, although all c-FLIP isoforms are phosphorylated on this conserved residue, the biological outcome is different for the long and short isoforms, since S193 specifically prolongs the half-lives of the short c-FLIP isoforms, but not c-FLIPL. To summarize, we show that c-FLIP proteins are modified by ubiquitylation and phosphorylation, and that the biological outcomes of these modifications are isoform-specifically determined.

ContinuousMultimodal Global Optimization with Differential Evolution-Based Methods

Metaheuristic methods have become increasingly popular approaches in solving global optimization problems. From a practical viewpoint, it is often desirable to perform multimodal optimization which, enables the search of more than one optimal solution to the task at hand. Population-based metaheuristic methods offer a natural basis for multimodal optimization. The topic has received increasing interest especially in the evolutionary computation community. Several niching approaches have been suggested to allow multimodal optimization using evolutionary algorithms. Most global optimization approaches, including metaheuristics, contain global and local search phases. The requirement to locate several optima sets additional requirements for the design of algorithms to be effective in both respects in the context of multimodal optimization. In this thesis, several different multimodal optimization algorithms are studied in regard to how their implementation in the global and local search phases affect their performance in different problems. The study concentrates especially on variations of the Differential Evolution algorithm and their capabilities in multimodal optimization. To separate the global and local search search phases, three multimodal optimization algorithms are proposed, two of which hybridize the Differential Evolution with a local search method. As the theoretical background behind the operation of metaheuristics is not generally thoroughly understood, the research relies heavily on experimental studies in finding out the properties of different approaches. To achieve reliable experimental information, the experimental environment must be carefully chosen to contain appropriate and adequately varying problems. The available selection of multimodal test problems is, however, rather limited, and no general framework exists. As a part of this thesis, such a framework for generating tunable test functions for evaluating different methods of multimodal optimization experimentally is provided and used for testing the algorithms. The results demonstrate that an efficient local phase is essential for creating efficient multimodal optimization algorithms. Adding a suitable global phase has the potential to boost the performance significantly, but the weak local phase may invalidate the advantages gained from the global phase.

Evolution of Bioaffinity Reagents by Phage Display

Antibodies are natural binding proteins produced in vertebrates as a response to invading pathogens and foreign substances. Because of their capability for tight and specific binding, antibodies have found use as binding reagents in research and diagnostics. Properties of cloned recombinant antibodies can be further improved by means of in vitro evolution, combining mutagenesis with subsequent phage display selection. It is also possible to isolate entirely new antibodies from vast naïve or synthetic antibody libraries by phage display. In this study, library techniques and phage display selection were applied in order to optimise binding scaffolds and antigen recognition of antibodies, and to evolve new and improved bioaffinity reagents. Antibody libraries were generated by random and targeted mutagenesis. Expression and stability were mainly optimised by the random methods whereas targeted randomisation of the binding site residues was used for optimising the binding properties. Trinucleotide mutagenesis allowed design of defined randomisation patterns for a synthetic antibody library. Improved clones were selected by phage display. Capture by a specific anti- DHPS antibody was exploited in the selection of improved phage display of DHPS. Efficient selection for stability was established by combining phage display selection with denaturation under reducing conditions. Broad-specific binding of a generic anti-sulfonamide antibody was improved by selection with one of the weakest binding sulfonamides. In addition, p9 based phage display was studied in affinity selection from the synthetic library. A TIM barrel protein DHPS was engineered for efficient phage display by combining cysteinereplacement with random mutagenesis. The resulting clone allows use of phage display in further engineering of DHPS and possibly use as an alternative-binding scaffold. An anti-TSH scFv fragment, cloned from a monoclonal antibody, was engineered for improved stability to better suite an immunoassay. The improved scFv tolerates 8 – 9 °C higher temperature than the parental scFv and should have sufficient stability to be used in an immunoanalyser with incubation at 36 °C. The anti-TSH scFv fragment was compared with the corresponding Fab fragment and the parental monoclonal antibody as a capturing reagent in a rapid 5-min immunoassay for TSH. The scFv fragment provided some benefits over the conventionally used Mab in anayte-binding capacity and assay kinetics. However, the recombinant Fab fragment, which had similar kinetics to the scFv, provided a more sensitive and reliable assay than the scFv. Another cloned scFv fragment was engineered in order to improve broad-specific recognition of sulfonamides. The improved antibody detects different sulfonamides at concentrations below the maximum residue limit (100 μg/kg in EU and USA) and allows simultaneous screening of different sulfonamide drug residues. Finally, a synthetic antibody library was constructed and new antibodies were generated and affinity matured entirely in vitro. These results illuminate the possibilities of phage display and antibody engineering for generation and optimisation of binding reagents in vitro and indicate the potential of recombinant antibodies as affinity reagents in immunoassays.

Generalized Differential Evolution for Global Multi-Objective Optimization with Constraints

The objective of this thesis work is to develop and study the Differential Evolution Algorithm for multi-objective optimization with constraints. Differential Evolution is an evolutionary algorithm that has gained in popularity because of its simplicity and good observed performance. Multi-objective evolutionary algorithms have become popular since they are able to produce a set of compromise solutions during the search process to approximate the Pareto-optimal front. The starting point for this thesis was an idea how Differential Evolution, with simple changes, could be extended for optimization with multiple constraints and objectives. This approach is implemented, experimentally studied, and further developed in the work. Development and study concentrates on the multi-objective optimization aspect. The main outcomes of the work are versions of a method called Generalized Differential Evolution. The versions aim to improve the performance of the method in multi-objective optimization. A diversity preservation technique that is effective and efficient compared to previous diversity preservation techniques is developed. The thesis also studies the influence of control parameters of Differential Evolution in multi-objective optimization. Proposals for initial control parameter value selection are given. Overall, the work contributes to the diversity preservation of solutions in multi-objective optimization.

Ion selective electrodes for determination of low and ultra low concentrations of lead (II) in natural waters

Att övervaka förekomsten av giftiga komponenter i naturliga vattendrag är nödvändigt för människans välmående. Eftersom halten av föroreningar i naturens ekosystem bör hållas möjligast låg, pågår en ständig jakt efter kemiska analysmetoder med allt lägre detektionsgränser. I dagens läge görs miljöanalyser med dyr och sofistikerad instrumentering som kräver mycket underhåll. Jonselektiva elektroder har flera goda egenskaper som t.ex. bärbarhet, låg energiförbrukning, och dessutom är de relativt kostnadseffektiva. Att använda jonselektiva elektroder vid miljöanalyser är möjligt om deras känslighetsområde kan utvidgas genom att sänka deras detektionsgränser. För att sänka detektionsgränsen för Pb(II)-selektiva elektroder undersöktes olika typer av jonselektiva membran som baserades på polyakrylat-kopolymerer, PVC och PbS/Ag2S. Fast-fas elektroder med membran av PbS/Ag2S är i allmänhet enklare och mer robusta än konventionella elektroder vid spårämnesanalys av joniska föroreningar. Fast-fas elektrodernas detektionsgräns sänktes i detta arbete med en nyutvecklad galvanostatisk polariseringsmetod och de kunde sedan framgångsrikt användas för kvantitativa bestämningar av bly(II)-halter i miljöprov som hade samlats in i den finska skärgården nära tidigare industriområden. Analysresultaten som erhölls med jonselektiva elektroder bekräftades med andra analytiska metoder. Att sänka detektionsgränsen m.hj.a. den nyutvecklade polariseringsmetoden möjliggör bestämning av låga och ultra-låga blyhalter som inte kunde nås med klassisk potentiometri. Den verkliga fördelen med att använda dessa blyselektiva elektroder är möjligheten att utföra mätningar i obehandlade miljöprov trots närvaron av fasta partiklar vilket inte är möjligt att göra med andra analysmetoder. Jag väntar mig att den nyutvecklade polariseringsmetoden kommer att sätta en trend i spårämnesanalys med jonselektiva elektroder.

Differential Evolution approach and parameter estimation of chaotic

Parameter estimation still remains a challenge in many important applications. There is a need to develop methods that utilize achievements in modern computational systems with growing capabilities. Owing to this fact different kinds of Evolutionary Algorithms are becoming an especially perspective field of research. The main aim of this thesis is to explore theoretical aspects of a specific type of Evolutionary Algorithms class, the Differential Evolution (DE) method, and implement this algorithm as codes capable to solve a large range of problems. Matlab, a numerical computing environment provided by MathWorks inc., has been utilized for this purpose. Our implementation empirically demonstrates the benefits of a stochastic optimizers with respect to deterministic optimizers in case of stochastic and chaotic problems. Furthermore, the advanced features of Differential Evolution are discussed as well as taken into account in the Matlab realization. Test "toycase" examples are presented in order to show advantages and disadvantages caused by additional aspects involved in extensions of the basic algorithm. Another aim of this paper is to apply the DE approach to the parameter estimation problem of the system exhibiting chaotic behavior, where the well-known Lorenz system with specific set of parameter values is taken as an example. Finally, the DE approach for estimation of chaotic dynamics is compared to the Ensemble prediction and parameter estimation system (EPPES) approach which was recently proposed as a possible solution for similar problems.

Methods of genetic diversity creation and functional display for directed evolution experiments

Protein engineering aims to improve the properties of enzymes and affinity reagents by genetic changes. Typical engineered properties are affinity, specificity, stability, expression, and solubility. Because proteins are complex biomolecules, the effects of specific genetic changes are seldom predictable. Consequently, a popular strategy in protein engineering is to create a library of genetic variants of the target molecule, and render the population in a selection process to sort the variants by the desired property. This technique, called directed evolution, is a central tool for trimming protein-based products used in a wide range of applications from laundry detergents to anti-cancer drugs. New methods are continuously needed to generate larger gene repertoires and compatible selection platforms to shorten the development timeline for new biochemicals. In the first study of this thesis, primer extension mutagenesis was revisited to establish higher quality gene variant libraries in Escherichia coli cells. In the second study, recombination was explored as a method to expand the number of screenable enzyme variants. A selection platform was developed to improve antigen binding fragment (Fab) display on filamentous phages in the third article and, in the fourth study, novel design concepts were tested by two differentially randomized recombinant antibody libraries. Finally, in the last study, the performance of the same antibody repertoire was compared in phage display selections as a genetic fusion to different phage capsid proteins and in different antibody formats, Fab vs. single chain variable fragment (ScFv), in order to find out the most suitable display platform for the library at hand. As a result of the studies, a novel gene library construction method, termed selective rolling circle amplification (sRCA), was developed. The method increases mutagenesis frequency close to 100% in the final library and the number of transformants over 100-fold compared to traditional primer extension mutagenesis. In the second study, Cre/loxP recombination was found to be an appropriate tool to resolve the DNA concatemer resulting from error-prone RCA (epRCA) mutagenesis into monomeric circular DNA units for higher efficiency transformation into E. coli. Library selections against antigens of various size in the fourth study demonstrated that diversity placed closer to the antigen binding site of antibodies supports generation of antibodies against haptens and peptides, whereas diversity at more peripheral locations is better suited for targeting proteins. The conclusion from a comparison of the display formats was that truncated capsid protein three (p3Δ) of filamentous phage was superior to the full-length p3 and protein nine (p9) in obtaining a high number of uniquely specific clones. Especially for digoxigenin, a difficult hapten target, the antibody repertoire as ScFv-p3Δ provided the clones with the highest affinity for binding. This thesis on the construction, design, and selection of gene variant libraries contributes to the practical know-how in directed evolution and contains useful information for scientists in the field to support their undertakings.

The interplay of JNK and SCG10 during cortical development and in excitotoxic responses in brain

Initially identified as stress activated protein kinases (SAPKs), the c-Jun Nterminal kinases (JNKs) are currently accepted as potent regulators of various physiologically important cellular events. Named after their competence to phosphorylate transcription factor c-Jun in response to UVtreatment, JNKs play a key role in cell proliferation, cell death or cell migration. Interestingly, these functions are crucial for proper brain formation. The family consists of three JNK isoforms, JNK1, JNK2 and JNK3. Unlike brain specific JNK3 isoform, JNK1 and JNK2 are ubiquitously expressed. It is estimated that ten splice variants exist. However, the detailed cellular functions of these remain undetermined. In addition, physiological conditions keep the activities of JNK2 and JNK3 low in comparison with JNK1, whereas cellular stress raises the activity of these isoforms dramatically. Importantly, JNK1 activity is constitutively high in neurons, yet it does not stimulate cell death. This suggests a valuable role for JNK1 in brain development, but also as an important mediator of cell wellbeing. The aim of this thesis was to characterize the functional relationship between JNK1 and SCG10. We found that SCG10 is a bona fide target for JNK. By employing differential centrifugation we showed that SCG10 co-localized with active JNK, MKK7 and JIP1 in a fraction containing endosomes and Golgi vesicles. Investigation of JNK knockout tissues using phosphospecific antibodies recognizing JNK-specific phosphorylation sites on SCG10 (Ser 62/Ser 73) showed that phosphorylation of endogenous SCG10 was dramatically decreased in Jnk1-/- brains. Moreover, we found that JNK and SCG10 co-express during early embryonic days in brain regions that undergo extensive neuronal migration. Our study revealed that selective inhibition of JNK in the cytoplasm significantly increased both the frequency of exit from the multipolar stage and radial migration rate. However, as a consequence, it led to ill-defined cellular organization. Furthermore, we found that multipolar exit and radial migration in Jnk1 deficient mice can be connected to changes in phosphorylation state of SCG10. Also, the expression of a pseudo-phosphorylated mutant form of SCG10, mimicking the JNK1- phopshorylated form, brings migration rate back to normal in Jnk1 knockout mouse embryos. Furthermore, we investigated the role of SCG10 and JNK in regulation of Golgi apparatus (GA) biogenesis and whether pathological JNK action could be discernible by its deregulation. We found that SCG10 maintains GA integrity as with the absence of SCG10 neurons present more compact fragmented GA structure, as shown by the knockdown approach. Interestingly, neurons isolated from Jnk1-/- mice show similar characteristics. Block of ER to GA is believed to be involved in development of Parkinson's disease. Hence, by using a pharmacological approach (Brefeldin A treatment), we showed that GA recovery is delayed upon removal of the drug in Jnk1-/- neurons to an extent similar to the shRNA SCG10-treated cells. Finally, we investigated the role of the JNK1-SCG10 duo in the maintenance of GA biogenesis following excitotoxic insult. Although the GA underwent fragmentation in response to NMDA treatment, we observed a substantial delay in GA disintegration in neurons lacking either JNK1 or SCG10.