Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

First dice your dill – new methods and techniques in sample handling

This book is dedicated to celebrate the 60th birthday of Professor Rainer Huopalahti. Professor Rainer “Repe” Huopalahti has had, and in fact is still enjoying a distinguished career in the analysis of food and food related flavor compounds. One will find it hard to make any progress in this particular field without a valid and innovative sample handling technique and this is a field in which Professor Huopalahti has made great contributions. The title and the front cover of this book honors Professor Huopahti’s early steps in science. His PhD thesis which was published on 1985 is entitled “Composition and content of aroma compounds in the dill herb, Anethum graveolens L., affected by different factors”. At that time, the thesis introduced new technology being applied to sample handling and analysis of flavoring compounds of dill. Sample handling is an essential task that in just about every analysis. If one is working with minor compounds in a sample or trying to detect trace levels of the analytes, one of the aims of sample handling may be to increase the sensitivity of the analytical method. On the other hand, if one is working with a challenging matrix such as the kind found in biological samples, one of the aims is to increase the selectivity. However, quite often the aim is to increase both the selectivity and the sensitivity. This book provides good and representative examples about the necessity of valid sample handling and the role of the sample handling in the analytical method. The contributors of the book are leading Finnish scientists on the field of organic instrumental analytical chemistry. Some of them are also Repe’ s personal friends and former students from the University of Turku, Department of Biochemistry and Food Chemistry. Importantly, the authors all know Repe in one way or another and are well aware of his achievements on the field of analytical chemistry. The editorial team had a great time during the planning phase and during the “hard work editorial phase” of the book. For example, we came up with many ideas on how to publish the book. After many long discussions, we decided to have a limited edition as an “old school hard cover book” – and to acknowledge more modern ways of disseminating knowledge by publishing an internet version of the book on the webpages of the University of Turku. Downloading the book from the webpage for personal use is free of charge. We believe and hope that the book will be read with great interest by scientists working in the fascinating field of organic instrumental analytical chemistry. We decided to publish our book in English for two main reasons. First, we believe that in the near future, more and more teaching in Finnish Universities will be delivered in English. To facilitate this process and encourage students to develop good language skills, it was decided to be published the book in English. Secondly, we believe that the book will also interest scientists outside Finland – particularly in the other member states of the European Union. The editorial team thanks all the authors for their willingness to contribute to this book – and to adhere to the very strict schedule. We also want to thank the various individuals and enterprises who financially supported the book project. Without that support, it would not have been possible to publish the hardcover book.

Comparison measures of maps generated by geostatistical methods

This study uses several measures derived from the error matrix for comparing two thematic maps generated with the same sample set. The reference map was generated with all the sample elements and the map set as the model was generated without the two points detected as influential by the analysis of local influence diagnostics. The data analyzed refer to the wheat productivity in an agricultural area of 13.55 ha considering a sampling grid of 50 x 50 m comprising 50 georeferenced sample elements. The comparison measures derived from the error matrix indicated that despite some similarity on the maps, they are different. The difference between the estimated production by the reference map and the actual production was of 350 kilograms. The same difference calculated with the mode map was of 50 kilograms, indicating that the study of influential points is of fundamental importance to obtain a more reliable estimative and use of measures obtained from the error matrix is a good option to make comparisons between thematic maps.

A platform for anti-biofilm assays combining viability, biomass and matrix quantifications in susceptibility assessments of antimicrobials against Staphylococcus aureus biofilms

Bacteria can exist as planktonic, the lifestyle in which single cells exist in suspension, and as biofilms, which are surface-attached bacterial communities embedded in a selfproduced matrix. Most of the antibiotics and the methods for antimicrobial work have been developed for planktonic bacteria. However, the majority of the bacteria in natural habitats live as biofilms. Biofilms develop dauntingly fast high resistance towards conventional antibacterial treatments and thus, there is a great need to meet the demands of effective anti-biofilm therapy. In this thesis project it was attempted to fill the void of anti-biofilm screening methods by developing a platform of assays that evaluate the effect that screened compounds have on the total biomass, viability and the extracellular polysaccharide (EPS) layer of the biofilms. Additionally, a new method for studying biofilms and their interactions with compounds in a continuous flow system was developed using capillary electrochromatography (CEC). The screening platform was utilized with a screening campaign using a small library of cinchona alkaloids. The assays were optimized to be statistically robust enough for screening. The first assay, based on crystal violet staining, measures total biofilm biomass, and it was automated using a liquid handling workstation to decrease the manual workload and signal variation. The second assay, based on resazurin staining, measures viability of the biofilm, and it was thoroughly optimized for the strain used, but was then a very simple and fast method to be used for primary screening. The fluorescent resazurin probe is not toxic to the biofilms. In fact, it was also shown in this project that staining the biofilms with resazurin prior to staining with crystal violet had no effect on the latter and they can be used in sequence on the same screening plate. This sequential addition step was indeed a major improvement on the use of reagents and consumables and also shortened the work time. As a third assay in the platform a wheat germ agglutinin based assay was added to evaluate the effect a compound has on the EPS layer. Using this assay it was found that even if compounds might have clear effect on both biomass and viability, the EPS layer can be left untouched or even be increased. This is a clear implication of the importance of using several assays to be able to find “true hits” in a screening setting. In the pilot study of screening for antimicrobial and anti-biofilm effects using a cinchona alkaloid library, one compound was found to have antimicrobial effect against planktonic bacteria and prevent biofilm formation at low micromolar concentration. To eradicate biofilms, a higher concentration was needed. It was also shown that the chemical space occupied by the active compound was slightly different than the rest of the cinchona alkaloids as well as the rest of the compounds used for validatory screening during the optimization processes of the separate assays.

Immunohistochemical evaluation of fibrillar components of the extracellular matrix of transversalis fascia and anterior abdominal rectus sheath in men with inguinal hernia

OBJECTIVE: to evaluate the role of fibrillar extracellular matrix components in the pathogenesis of inguinal hernias. METHODS: samples of the transverse fascia and of the anterior sheath of the rectus abdominis muscle were collected from 40 men aged between 20 and 60 years with type II and IIIA Nyhus inguinal hernia and from 10 fresh male cadavers (controls) without hernia in the same age range. The staining technique was immunohistochemistry for collagen I, collagen III and elastic fibers; quantification of fibrillar components was performed with an image analysis processing software. RESULTS: no statistically significant differences were found in the amount of elastic fibers, collagen I and collagen III, and the ratio of collagen I / III among patients with inguinal hernia when compared with subjects without hernia. CONCLUSION: the amount of fibrillar extracellular matrix components did not change in patients with and without inguinal hernia.

Gene expression profile of ABC transporters and cytotoxic effect of ibuprofen and acetaminophen in an epithelial ovarian cancer cell line in vitro

PURPOSES: To determine the basic expression of ABC transporters in an epithelial ovarian cancer cell line, and to investigate whether low concentrations of acetaminophen and ibuprofen inhibited the growth of this cell line in vitro. METHODS: TOV-21 G cells were exposed to different concentrations of acetaminophen (1.5 to 15 μg/mL) and ibuprofen (2.0 to 20 μg/mL) for 24 to 48 hours. The cellular growth was assessed using a cell viability assay. Cellular morphology was determined by fluorescence microscopy. The gene expression profile of ABC transporters was determined by assessing a panel including 42 genes of the ABC transporter superfamily. RESULTS: We observed a significant decrease in TOV-21 G cell growth after exposure to 15 μg/mL of acetaminophen for 24 (p=0.02) and 48 hours (p=0.01), or to 20 μg/mL of ibuprofen for 48 hours (p=0.04). Assessing the morphology of TOV-21 G cells did not reveal evidence of extensive apoptosis. TOV-21 G cells had a reduced expression of the genes ABCA1, ABCC3, ABCC4, ABCD3, ABCD4 and ABCE1 within the ABC transporter superfamily. CONCLUSIONS: This study provides in vitro evidence of inhibitory effects of growth in therapeutic concentrations of acetaminophen and ibuprofen on TOV-21 G cells. Additionally, TOV-21 G cells presented a reduced expression of the ABCA1, ABCC3, ABCC4, ABCD3, ABCD4 and ABCE1 transporters.

Experimental poisoning by Senecio brasiliensis in calves: quantitative and semi-quantitative study on changes in the hepatic extracellular matrix and sinusoidal cells

Extracellular matrix plays an important role in chronic hepatic lesions and has been studied in experimental intoxication models. However in cattle, studies on chronic disease have focused on the hepatocellular damage and extracellular matrix (ECM) changes are usually overlooked. There are no specific studies on the hepatic ECM in either normal or chronically damaged bovine liver. Thus an experimental model of hepatic toxicity model using Senecio brasiliensis poisoned calves was designed. Senecio brasiliensis contains pyrrolizidine alkaloids which cause either acute or chronic progressive dose dependent liver damage. Five calves were orally fed with 0.38g of dry leaves of S. brasiliensis/kg/day for 24 days. Liver needle biopsy specimens were obtained every 15 days for 60 days. Clinical signs of digestive complications appeared at 3rd week. One calf died on 45th day and four were evaluated up to 60th day. Biopsy samples were processed for routine light microscopy, immuno-histochemistry and transmission electron microscopy. From 30th day on progressive liver damage characterized by hepatocellular ballooning, necrosis, apoptosis and megalocytosis, centrilobular, pericellular and portal fibrosis were seen by light microscopy. Quantitative and semi-quantitative measurements of hepatic ECM components were performed before and after the onset of lesions. Morphometric analysis of total collagen and elastic fiber system was conducted. Total collagen and I and III collagen types progressively increased in throughout the liver of affected calves. Changes in location, amount and disposition of the elastic fiber system were also observed. Then numbers of Kupffer cells were significantly increased at 30th day and total numbers of sinusoidal cells were significantly increased at 45th and 60th days. Liver damage was progressive and irreversible even after the exposure to the plant was discontinued. Severe fibrotic lesions occurred mainly in portal tracts, followed by veno-occlusive and pericellular fibrosis. Collagen types I and III s were present in every normal and damaged liver, with predominance of type I. In affected calves the increase of total collagen and elastic fibers system paralleled the number of total sinusoidal cells.

Electrical Machine Product Development; Transmission to Lean Approach

The requirements set by the market for electrical machines become increasingly demanding requiring more sophisticated technological solutions. Companies producing electrical ma-chines are challenged to develop machines that provide competitive edge for the customer for example through increased efficiency, reliability or some customer specific special requirement. The objective of this thesis is to derive a proposal for the first steps to transform the electrical machine product development process of a manufacturing company towards lean product development. The current product development process in the company is presented together with the processes of four other companies interviewed for the thesis. On the basis of current processes of the electrical machine industry and the related literature, a generalized electrical machine product development process is derived. The management isms and –tools utilized by the companies are analyzed. Adoption of lean Pull-Event –reviews, Oobeya –management and Knowledge based product development are suggested as the initial steps of implementing lean product development paradigm in the manufacturing company. Proposals for refining the cur-rent product development process and increasing the stakeholder involvement in the development projects are made. Lean product development is finding its way to Finnish electrical machine industry, but the results will be available only after the methods have been implemented and adopted by the companies. There is some enthusiasm about the benefits of lean approach and if executed successfully it will provide competitive edge for the Finnish electrical machine industry.

Introduction of the T-10 static line parachuting capability to the NH90 helicopter

The purpose of this study was to investigate the suitability of the Finnish Defence Forces’ NH90 helicopter for parachuting operations with the T-10 static line parachute system. The work was based on the Army Command’s need to compensate for the reduction in the outsourced flight hours for the military static line parachuting training. The aim of the research was to find out the procedures and limitations with which the NH90 IOC+ or FOC version helicopter could be used for static line parachutist training with the T-10B/MC1-1C parachutes. The research area was highly complicated and non-linear. Thus analytical methods could not be applied with sufficient confidence, even with present-day computing power. Therefore an empirical research method was selected, concentrating on flight testing supported with literature study and some calculated estimations. During three flights and 4.5 flight hours in Utti, Finland on 1720 September 2012, a total of 44 parachute drops were made. These consisted of 16 dummy drops and 28 paratrooper jumps. The test results showed that when equipped with the floor mounted PASI-1 anchor line, the deflector bar of the NHIndustries’ Parachuting Kit and Patria’s floor protection panels the Finnish NH90 variant could be safely used for T-10B/MC1-1C static line parachuting operations from the right cabin door at airspeed range of 5080 KIAS (90–150 km/h). The ceiling mounted anchor lines of the NHI’s Parachuting Kit were not usable with the T-10 system. This was due to the static lines’ unsafe behaviour in slipstream when connected to the cabin ceiling level. In conclusion, the NH90 helicopter can be used to meet the Army Command’s requirement for an additional platform for T-10 static line parachutist training. Material dropping, the effect of additional equipment and jumping from the rear ramp should be further studied.

Machine vision methods for process measurements in pulping

The papermaking industry has been continuously developing intelligent solutions to characterize the raw materials it uses, to control the manufacturing process in a robust way, and to guarantee the desired quality of the end product. Based on the much improved imaging techniques and image-based analysis methods, it has become possible to look inside the manufacturing pipeline and propose more effective alternatives to human expertise. This study is focused on the development of image analyses methods for the pulping process of papermaking. Pulping starts with wood disintegration and forming the fiber suspension that is subsequently bleached, mixed with additives and chemicals, and finally dried and shipped to the papermaking mills. At each stage of the process it is important to analyze the properties of the raw material to guarantee the product quality. In order to evaluate properties of fibers, the main component of the pulp suspension, a framework for fiber characterization based on microscopic images is proposed in this thesis as the first contribution. The framework allows computation of fiber length and curl index correlating well with the ground truth values. The bubble detection method, the second contribution, was developed in order to estimate the gas volume at the delignification stage of the pulping process based on high-resolution in-line imaging. The gas volume was estimated accurately and the solution enabled just-in-time process termination whereas the accurate estimation of bubble size categories still remained challenging. As the third contribution of the study, optical flow computation was studied and the methods were successfully applied to pulp flow velocity estimation based on double-exposed images. Finally, a framework for classifying dirt particles in dried pulp sheets, including the semisynthetic ground truth generation, feature selection, and performance comparison of the state-of-the-art classification techniques, was proposed as the fourth contribution. The framework was successfully tested on the semisynthetic and real-world pulp sheet images. These four contributions assist in developing an integrated factory-level vision-based process control.

Introduction of the T-10 static line parachuting capability to the NH90 helicopter

The purpose of this study was to investigate the suitability of the Finnish Defence Forces’ NH90 helicopter for parachuting operations with the T-10 static line parachute system. The work was based on the Army Command’s need to compensate for the reduction in the outsourced flight hours for the military static line parachuting training. The aim of the research was to find out the procedures and limitations with which the NH90 IOC+ or FOC version helicopter could be used for static line parachutist training with the T- 10B/MC1-1C parachutes. The research area was highly complicated and non-linear. Thus analytical methods could not be applied with sufficient confidence, even with present-day computing power. Therefore an empirical research method was selected, concentrating on flight testing supported with literature study and some calculated estimations. During three flights and 4.5 flight hours in Utti, Finland on 17−20 September 2012, a total of 44 parachute drops were made. These consisted of 16 dummy drops and 28 paratrooper jumps. The test results showed that when equipped with the floor mounted PASI-1 anchor line, the deflector bar of the NHIndustries’ Parachuting Kit and Patria’s floor protection panels the Finnish NH90 variant could be safely used for T-10B/MC1-1C static line parachuting operations from the right cabin door at airspeed range of 50−80 KIAS (∼90–150 km/h). The ceiling mounted anchor lines of the NHI’s Parachuting Kit were not usable with the T-10 system. This was due to the static lines’ unsafe behaviour in slipstream when connected to the cabin ceiling level. In conclusion, the NH90 helicopter can be used to meet the Army Command’s requirement for an additional platform for T-10 static line parachutist training. Material dropping, the effect of additional equipment and jumping from the rear ramp should be further studied.

Functional differences between two morphologically distinct cell subpopulations within a human colorectal carcinoma cell line

The LISP-I human colorectal adenocarcinoma cell line was isolated from a hepatic metastasis at the Ludwig Institute, São Paulo, SP, Brazil. The objective of the present study was to isolate morphologically different subpopulations within the LISP-I cell line, and characterize some of their behavioral aspects such as adhesion to and migration towards extracellular matrix components, expression of intercellular adhesion molecules and tumorigenicity in vitro. Once isolated, the subpopulations were submitted to adhesion and migration assays on laminin and fibronectin (crucial proteins to invasion and metastasis), as well as to anchorage-independent growth. Two morphologically different subpopulations were isolated: LISP-A10 and LISP-E11. LISP-A10 presents a differentiated epithelial pattern, and LISP-E11 is fibroblastoid, suggesting a poorly differentiated pattern. LISP-A10 expressed the two intercellular adhesion molecules tested, carcinoembryonic antigen (CEA) and desmoglein, while LISP-E11 expressed only low amounts of CEA. On the other hand, adhesion to laminin and fibronectin as well as migration towards these extracellular matrix proteins were higher in LISP-E11, as expected from its poorly differentiated phenotype. Both subpopulations showed anchorage-independent growth on a semi-solid substrate. These results raise the possibility that the heterogeneity found in the LISP-I cell line, which might have contributed to its ability to metastasize, was due to at least two different subpopulations herein identified.

Food packaging indicators and sensors

Various categories of food packaging indicators namely; VTT, Ageless Eye, Mocon, Åbo Akademi and Impak were selected and incorporated into food trays manufactured at LUT packaging laboratory. Each of these food packaging indicators was used to investigate (visually and qualitatively) the transmission of oxygen through the seal, and tray material, as well as to detect microbial activity within the content of the package. Applications of different methods of gas flushing, content variation and introduction of two distinct levels of oxygen scavengers were employed as treatments to evaluate the packaging performance of the food packaging indicators. Ease of handling of each food packaging indicator was also taken into considerations. Findings showed that for packages, which contained chicken product, the amount of oxygen in the package, measured immediately after the sealing operation on the first day gradually decreased to zero percent by the third day of the storage period. The oxygen level remained at this point throughout the duration of storage for the chicken packages. Besides, level of oxygen in the packages without product continued to increase with the storage time, at moderate rate of 0.1% for 100%N2 and 0.3% for 30%CO2/70%N2 empty packages. More carbon dioxide gas was recorded for packages flushed with 30%CO2/70%N2. Results also revealed that visual analysis of one of the color indicators for example Ageless Eye, conformed to the data derived from the luminescence food-packaging indicator. This shows that packaging operation of the packaging line was considerably stable, and efficient with negligible exception. However, it was found that most of the food packaging indicators investigated in this research study exhibited considerable packaging challenges, such as, reaction with the content of the package (Impak); over sensitivity (Åbo Akademi and Impak); ease of handling problem (Åbo Akademi); and ease of activation problem (VTT indicators). In this study, the strengths and limitations of different indicators were analyzed. This study demonstrates the applicability of various indicators in MAP using chicken package application.

TIMP-1 mediates the inhibitory effect of interleukin-6 on the proliferation of a hepatocarcinoma cell line in a STAT3-dependent manner

The tissue inhibitor of metalloproteinases (TIMP)-1 is a multifunctional protein which is not only an inhibitor of matrix metalloproteinases (MMPs) but also to have a possible "cytokine-like" action. Here, we first compared mRNA expression of TIMP-1 and MMP-9 in BEL-7402 (a hepatocellular carcinoma cell line), L-02 (a normal liver cell line) and QSG-7701 (a cell line derived from peripheral tissue of liver carcinoma) using real-time quantitative RT-PCR. By evaluating the variation of the MMP-9/TIMP-1 ratio as an index of reciprocal changes of the expression of the two genes, we observed that the MMP-9/TIMP-1 ratio was about 13- and 5-fold higher in BEL-7402 than in L-02 and QSG-7701, respectively. Significantly, overexpression of TIMP-1 decreased the MMP-9/TIMP-1 ratio in BEL-7402 and then inhibited the cell growth to 60% and reduced the migration to about 30%. Meanwhile, our data showed that interleukin-6 (IL-6) (100 ng/mL) could also inhibited the cell growth of BEL-7402. Further studies indicated that TIMP-1 mediated the inhibitory effect of IL-6 on BEL-7402 cell proliferation in a STAT3-dependent manner, which could further accelerate the expression of the cyclin-dependent kinase inhibitor p21. A dominant negative STAT3 mutant totally abolished IL-6-induced TIMP-1 expression and its biological functions. The present results demonstrate that TIMP-1 may be one of the mediators that regulate the inhibitory effect of IL-6 on BEL-7402 proliferation in which STAT3 signal transduction and p21 up-regulation also play important roles.

Primary mechanism of apoptosis induction in a leukemia cell line by fraction FA-2-b-ß prepared from the mushroom Agaricus blazei Murill

Agaricus blazei Murill is a native Brazilian mushroom which functions primarily as an anticancer substance in transplanted mouse tumors. However, the mechanism underlying this function of A. blazei Murill remains obscure. The present study was carried out to investigate the effect of fraction FA-2-b-ß, an RNA-protein complex isolated from A. blazei Murill, on human leukemia HL-60 cells in vitro. Typical apoptotic characteristics were determined by morphological methods using DNA agarose gel electrophoresis and flow cytometry. The growth suppressive effect of fraction FA-2-b-ß on HL-60 cells in vitro occurred in a dose- (5-80 µg/mL) and time-dependent (24-96 h) manner. The proliferation of HL-60 cells (1 x 10(5) cells/mL) treated with 40 µg/mL of fraction FA-2-b-ß for 24-96 h and with 5-80 µg/mL for 96 h resulted in inhibitory rates ranging from 8 to 54.5%, and from 4.9 to 86.3%, respectively. Both telomerase activity determined by TRAP-ELISA and mRNA expression of the caspase-3 gene detected by RT-PCR were increased in HL-60 cells during fraction FA-2-b-ß treatment. The rate of apoptosis correlated negatively with the decrease of telomerase activity (r = 0.926, P < 0.05), but correlated positively with caspase-3 mRNA expression (r = 0.926, P < 0.05). These data show that fraction FA-2-b-ß can induce HL-60 cell apoptosis and that the combined effect of down-regulation of telomerase activity and up-regulation of mRNA expression of the caspase-3 gene could be the primary mechanism of induction of apoptosis. These findings provide strong evidence that fraction FA-2-b-ß could be of interest for the clinical treatment of acute leukemia.