Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

Combustion properties of biomass residues rich in phosphorus

Tillgången på traditionella biobränslen är begränsad och därför behöver man ta fram nya, tidigare outnyttjade biobränslen för att möta de uppställda CO2 emissionsmålen av EU och det ständigt ökande energibehovet. Under de senare åren har intresset riktats mot termisk energiutvinning ur olika restfraktioner och avfall. Vid produktion av fordonsbränsle ur biomassa är den fasta restprodukten ofta den största procesströmmen i produktionsanläggningen. En riktig hantering av restprodukterna skulle göra produktionen mera lönsam och mer ekologiskt hållbar. Ett alternativ är att genom förbränning producera elektricitet och/eller värme eftersom dessa restprodukter anses som CO2-neutrala. Målsättningen med den här avhandlingen var att studera förbränningsegenskaperna hos några fasta restprodukter som uppstår vid framställning av förnybara fordonsbränslen. De fyra undersökta materialen är rapskaka, palmkärnskaka, torkad drank och stabiliserat rötslam. I studien används ett stort urval av undersökningsmetoder, från laboratorieskala till fullskalig förbränning, för att identifiera de huvudsakliga utmaningarna förknippade med förbränning av restprodukterna i pannor med fluidiserad bäddteknik. Med hjälp av detaljerad bränslekarakterisering kunde restprodukterna konstateras vara en värdefull källa för värme- och elproduktion. Den kemiska sammansättningen av restprodukterna varierar stort jämfört med mera traditionellt använda biobränslen. En gemensam faktor för alla de studerade restprodukterna är en hög fosforhalt. På grund av de låga fosforkoncentrationerna i de traditionella biobränslena har grundämnet hittills inte ansetts spela någon större roll i askkemin. Experimenten visade nu att fosfor inte mera kan försummas då man studerar kemin i förbränningsprocesser, då allt flera fosforrika bränslen tränger in på energimarknaden.

Simulation methods in the modelling of bioaffinity assays

Computational model-based simulation methods were developed for the modelling of bioaffinity assays. Bioaffinity-based methods are widely used to quantify a biological substance in biological research, development and in routine clinical in vitro diagnostics. Bioaffinity assays are based on the high affinity and structural specificity between the binding biomolecules. The simulation methods developed are based on the mechanistic assay model, which relies on the chemical reaction kinetics and describes the forming of a bound component as a function of time from the initial binding interaction. The simulation methods were focused on studying the behaviour and the reliability of bioaffinity assay and the possibilities the modelling methods of binding reaction kinetics provide, such as predicting assay results even before the binding reaction has reached equilibrium. For example, a rapid quantitative result from a clinical bioaffinity assay sample can be very significant, e.g. even the smallest elevation of a heart muscle marker reveals a cardiac injury. The simulation methods were used to identify critical error factors in rapid bioaffinity assays. A new kinetic calibration method was developed to calibrate a measurement system by kinetic measurement data utilizing only one standard concentration. A nodebased method was developed to model multi-component binding reactions, which have been a challenge to traditional numerical methods. The node-method was also used to model protein adsorption as an example of nonspecific binding of biomolecules. These methods have been compared with the experimental data from practice and can be utilized in in vitro diagnostics, drug discovery and in medical imaging.

Effects of low-level laser therapy on wound healing

OBJECTIVE: To gather and clarify the actual effects of low-level laser therapy on wound healing and its most effective ways of application in human and veterinary medicine.METHODS: We searched original articles published in journals between the years 2000 and 2011, in Spanish, English, French and Portuguese languages, belonging to the following databases: Lilacs, Medline, PubMed and Bireme; Tey should contain the methodological description of the experimental design and parameters used.RESULTS: doses ranging from 3 to 6 J/cm2 appear to be more effective and doses 10 above J/cm2 are associated with deleterious effects. The wavelengths ranging from 632.8 to 1000 nm remain as those that provide more satisfactory results in the wound healing process.CONCLUSION: Low-level laser can be safely applied to accelerate the resolution of cutaneous wounds, although this fact is closely related to the election of parameters such as dose, time of exposure and wavelength.

Identification and characterisation of a novel adhesin of Streptococcus suis and its use as a target of adhesion inhibition and bacterial detection

Streptococcus suis is an important pig pathogen but it is also zoonotic, i.e. capable of causing diseases in humans. Human S. suis infections are quite uncommon but potentially life-threatening and the pathogen is an emerging public health concern. This Gram-positive bacterium possesses a galabiose-specific (Galalpha1−4Gal) adhesion activity, which has been studied for over 20 years. P-fimbriated Escherichia coli−bacteria also possess a similar adhesin activity targeting the same disaccharide. The galabiose-specific adhesin of S. suis was identified by an affinity proteomics method. No function of the protein identified was formerly known and it was designated streptococcal adhesin P (SadP). The peptide sequence of SadP contains an LPXTG-motif and the protein was proven to be cell wall−anchored. SadP may be multimeric since in SDS-PAGE gel it formed a protein ladder starting from about 200 kDa. The identification was confirmed by producing knockout strains lacking functional adhesin, which had lost their ability to bind to galabiose. The adhesin gene was cloned in a bacterial expression host and properties of the recombinant adhesin were studied. The galabiose-binding properties of the recombinant protein were found to be consistent with previous results obtained studying whole bacterial cells. A live-bacteria application of surface plasmon resonance was set up, and various carbohydrate inhibitors of the galabiose-specific adhesins were studied with this assay. The potencies of the inhibitors were highly dependent on multivalency. Compared with P-fimbriated E. coli, lower concentrations of galabiose derivatives were needed to inhibit the adhesion of S. suis. Multivalent inhibitors of S. suis adhesion were found to be effective at low nanomolar concentrations. To specifically detect galabiose adhesin−expressing S. suis bacteria, a technique utilising magnetic glycoparticles and an ATP bioluminescence bacterial detection system was also developed. The identification and characterisation of the SadP adhesin give valuable information on the adhesion mechanisms of S. suis, and the results of this study may be helpful for the development of novel inhibitors and specific detection methods of this pathogen.

Formal development and quantitative verification of dependable systems

-

Tunable lumped-element bandpass filters for Cognitive Radio application

In this thesis the design of bandpass filters tunable at 400 MHz – 800 MHz was under research. Microwave filters are vital components which provide frequency selectivity in wide variety of electronic systems operating at high frequencies. Due to the occurrence of multi-frequency bands communication and diverse applications of wireless devices, requirement of tunable filters exists. The one of potential implementation of frequency-agile filters is frontends and spectrum sensors in Cognitive Radio (CR). The principle of CR is to detect and operate at a particular available spectrum without interfering with the primary user’s signals. This new method allows improving the efficiency of utilizing allocated spectrum such as TV band (400 MHz – 800 MHz). The focus of this work is development of sufficiently compact, low cost tunable filters with quite narrow bandwidth using currently available lumped-element components and PCB board technology. Filter design, different topologies and methods of tuning of bandpass filters are considered in this work. As a result, three types of topologies of bandpass filter were simulated and realised. They use digitally tunable capacitors (DTCs) for adjusting central frequency at TV "white space" spectrum. Measurements revealed that schematics presented in this work have proper output response and filters are successfully tuned by DTCs.

Comparison of methods for determination of nutrient digestibility of a dry kibble diet for ocelots

In this experiment, methods of total fecal collection (TFC) and internal markers (acid-insoluble ash - AIA, crude fiber - CF, and acid-detergent fiber - ADF) were compared for determination of the coefficients of apparent digestibility (CAD) for dry matter (DM), crude protein (CP), ether extract (EE), nitrogen-free extracts (NFE), and gross energy (GE) of commercial feline dry kibble for ocelots (Leopardus pardalis). Six adult animals, weighing 12.45±1.37 kg, gradually received experimental kibble in their usual diet until the beginning of the experiment and were submitted to an adaptation period ten days prior to the collection period. CAD obtained by TFC, AIA, CF, and ADF were, respectively, 73.7, 76.83, 62.01, and 46.03% for dry matter; 81.9, 84.8, 75.8, and 63.8% for crude protein; 85, 86.7, 78.5, and 69.1% for ether extract; 78.52, 79.55, 69.11, and 53.04% for nitrogen-free extracts; and 80.5, 82.2, 71.4, and 58.4% for gross energy. The AIA method showed to be efficient in determining coefficients of apparent digestibility and may contribute to investigations on the digestibility of diets for wild felines. In comparison to the items of ocelot's usual diet, the kibble used in this paper provided an adequate nutritional supply with reduced daily costs per animal.

The application of Process Management for reliability enhancement of the Taguchi method

The concept of Process Management has been used by managers and consultants that search for the improvement of both operational or managerial industrial processes. Its strength is in focusing on the external client and on the optimization of the internal process in order to fulfill their needs. By the time the needs of internal clients are being sought, a set of improvements takes place. The Taguchi method, because of its claim for knowledge share between design engineers and people engaged in the process, is a candidate for process management implementation. The objective of this paper is to propose that kind of application aiming for improvements related with reliability of results revealed by the robust design of Taguchi method.

Stochastic approximation methods in stochastic optimization

Stochastic approximation methods for stochastic optimization are considered. Reviewed the main methods of stochastic approximation: stochastic quasi-gradient algorithm, Kiefer-Wolfowitz algorithm and adaptive rules for them, simultaneous perturbation stochastic approximation (SPSA) algorithm. Suggested the model and the solution of the retailer's profit optimization problem and considered an application of the SQG-algorithm for the optimization problems with objective functions given in the form of ordinary differential equation.

Evaluation of methods for estimating energy performance of biogas production

This study focused on identifying various system boundaries and evaluating methods of estimating energy performance of biogas production. First, the output-input ratio method used for evaluating energy performance from the system boundaries was reviewed. Secondly, ways to assess the efficiency of biogas use and parasitic energy demand were investigated. Thirdly, an approach for comparing biogas production to other energy production methods was evaluated. Data from an existing biogas plant, located in Finland, was used for the evaluation of the methods. The results indicate that calculating and comparing the output-input ratios (Rpr1, Rpr2, Rut, Rpl and Rsy) can be used in evaluating the performance of biogas production system. In addition, the parasitic energy demand calculations (w) and the efficiency of utilizing produced biogas (η) provide detailed information on energy performance of the biogas plant. Furthermore, Rf and energy output in relation to total solid mass of feedstock (FO/TS) are useful in comparing biogas production with other energy recovery technologies. As a conclusion it is essential for the comparability of biogas plants that their energy performance would be calculated in a more consistent manner in the future.

Effect of dose and application site on quinclorac absorption by barnyardgrass biotypes

This work aimed to evaluate the uptake and translocation of quinclorac in function of application sites (shoot or roots) by Echinochloa crusgalli biotypes resistant and susceptible to this herbicide. The treatments consisted of quinclorac doses (0; 0.5; 1; 2; 4; 16 and 64 ppm), applied on the shoot or roots of seedlings of barnyardgrass biotypes. The experimental units consisted of plastic cups containing 250 cm³ of sand. The treatments were applied 10 days after emergence, when barnyardgrass plants reached a 2- to 3- leaf growth stage. The barnyardgrass biotypes were irrigated with nutritive solution weekly and maintained for 40 days after emergence, when length, fresh and dry matter of shoot and roots were evaluated. Variance analysis was carried out using the F test at 5% probability, and in case of significance, a non-linear regression analysis was also carried out using a three-parameter logistic model. In the susceptible biotype, quinclorac was more absorbed by the roots than by the shoot. Comparing dry mass production of the different plant parts of the susceptible biotype per application site, it was verified that quinclorac action is higher when applied to the plant roots. However, for the resistant biotype, it was not possible to determine the dose causing 50% reduction in dry mass accumulation (GR50) and in the resistance index (RI) between both biotypes, due to its high resistance to quinclorac (128 times the recommended dosage). The results showed that quinclorac resistance by the evaluated biotype is not due to differences in the absorption site, strongly suggesting that the resistance acquired by the biotype may result from alteration in the target site.

Effects of early compost application on no-till organic soybean

Weed control has always been an important issue in agriculture. With the advent of no-till systems, soil erosion was reduced but herbicide use was increased. Organic no-till systems try to adjust reduced erosion to the no use of herbicides. Nevertheless, this adjustment is limited by the cost of mechanical weed control. This cost may be reduced by improved cultural weed control with cover crops mulches. In this paper we report a study on the application of compost manure on an oats winter cover crop, preceding soybean, instead of on the soybean summer crop. Treatments comprised a control without compost manure, and compost manure doses of 4 and 8 Mg ha-1 applied either on oats in winter or soybean in summer, organized in a randomized block design, with five replications. In summer, plots were split into weed-controlled or not controlled subplots. The timing of application and the manure doses did not affect the oats biomass or the soybean performance. However, in summer, without water stress, the application of manure at 8 Mg ha-1 directly on soybean has reduced weed biomass in this crop.

Leaching of Atrazine in Commercial and Xerogel Formulations in Oxisol Using Bioassay and Chromatographic Methods

Mobility of atrazine in soil has contributed to the detection of levels above the legal limit in surface water and groundwater in Europe and the United States. The use of new formulations can reduce or minimize the impacts caused by the intensive use of this herbicide in Brazil, mainly in regions with higher agricultural intensification. The objective of this study was to compare the leaching of a commercial formulation of atrazine (WG) with a controlled release formulation (xerogel) using bioassay and chromatographic methods of analysis. The experiment was a split plot randomized block design with four replications, in a (2 x 6) + 1 arrangement. The main formulations of atrazine (WG and xerogel) were allocated in the plots, and the herbicide concentrations (0, 3200, 3600, 4200, 5400 and 8000 g ha-1), in the subplots. Leaching was determined comparatively by using bioassays with oat and chromatographic analysis. The results showed a greater concentration of the herbicide in the topsoil (0-4 cm) in the treatment with the xerogel formulation in comparison with the commercial formulation, which contradicts the results obtained with bioassays, probably because the amount of herbicide available for uptake by plants in the xerogel formulation is less than that available in the WG formulation.

Application Volumes and Sizes of Droplets for the Application of Diquat Herbicide in the Control of Eichhornia crassipes

The objective of this study was to evaluate the efficiency of application and the efficacy of control of water hyacinth (Eichhornia crassipes) with the use of the diquat herbicide done with two application volumes associated to three droplets classes. Three experiments were conducted; in the first, the application efficiency was evaluated; in the second, the control efficiency and; in the third, the droplet spectrum. They were conducted in a completely randomized design with five, four and six replicates, respectively. The treatments were arranged in a 2 x 3 factorial design, with two application volumes (100 and 200 L ha-1) and three droplets classes (fine, extremely coarse and ultra-coarse) to the first and second experiment and the third comprised two diquat concentrations in spray solution (2 and 4 g i.a. L-1) and three droplets classes (fine, extremely coarse and ultra-coarse). The application efficiency was determined by the coverage by droplets, spray deposition and active ingredient of the herbicide (diquat). The efficacy was measured by the control and the percentage of plants with regrowth at 50 days after application. The spectrum of droplets produced per each nozzle model used to obtain the droplets classes were analyzed. According to the parameters analyzed, using the droplets classes extremely thick and ultra thick can provide greater certainty in the application of diquat in the aquatic environment associated with the deposition of the active and sufficient coverage to control Eichhornia crassipes with both application volumes