Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

ESTIMATION OF CORK PRODUCTION USINGAERIAL IMAGERY1

ABSTRACT Inventory and prediction of cork harvest over time and space is important to forest managers who must plan and organize harvest logistics (transport, storage, etc.). Common field inventory methods including the stem density, diameter and height structure are costly and generally point (plot) based. Furthermore, the irregular horizontal structure of cork oak stands makes it difficult, if not impossible, to interpolate between points. We propose a new method to estimate cork production using digital multispectral aerial imagery. We study the spectral response of individual trees in visible and near infrared spectra and then correlate that response with cork production prior to harvest. We use ground measurements of individual trees production to evaluate the model’s predictive capacity. We propose 14 candidate variables to predict cork production based on crown size in combination with different NDVI index derivates. We use Akaike Information Criteria to choose the best among them. The best model is composed of combinations of different NDVI derivates that include red, green, and blue channels. The proposed model is 15% more accurate than a model that includes only a crown projection without any spectral information.

Estimation of the potential evapotranspiration by a simplified penman method

The numerous methods for calculating the potential or reference evapotranspiration (ETo or ETP) almost always do for a 24-hour period, including values of climatic parameters throughout the nocturnal period (daily averages). These results have a nil effect on transpiration, constituting the main evaporative demand process in cases of localized irrigation. The aim of the current manuscript was to come up with a model rather simplified for the calculation of diurnal daily ETo. It deals with an alternative approach based on the theoretical background of the Penman method without having to consider values of aerodynamic conductance of latent and sensible heat fluxes, as well as data of wind speed and relative humidity of the air. The comparison between the diurnal values of ETo measured in weighing lysimeters with elevated precision and estimated by either the Penman-Monteith method or the Simplified-Penman approach in study also points out a fairly consistent agreement among the potential demand calculation criteria. The Simplified-Penman approach was a feasible alternative to estimate ETo under the local meteorological conditions of two field trials. With the availability of the input data required, such a method could be employed in other climatic regions for scheduling irrigation.

Estimation of transpiration of the 'Valencia' orange young plant using thermal dissipation probe method

Most studies on measures of transpiration of plants, especially woody fruit, relies on methods of heat supply in the trunk. This study aimed to calibrate the Thermal Dissipation Probe Method (TDP) to estimate the transpiration, study the effects of natural thermal gradients and determine the relation between outside diameter and area of xylem in 'Valencia' orange young plants. TDP were installed in 40 orange plants of 15 months old, planted in boxes of 500 L, in a greenhouse. It was tested the correction of the natural thermal differences (DTN) for the estimation based on two unheated probes. The area of the conductive section was related to the outside diameter of the stem by means of polynomial regression. The equation for estimation of sap flow was calibrated having as standard lysimeter measures of a representative plant. The angular coefficient of the equation for estimating sap flow was adjusted by minimizing the absolute deviation between the sap flow and daily transpiration measured by lysimeter. Based on these results, it was concluded that the method of TDP, adjusting the original calibration and correction of the DTN, was effective in transpiration assessment.

Estimation of evaporation in the Banabuiú dam, in the state of Ceará, Brazil, by different combined methods, derived from the Penman equation

The state of Ceará, Brazil, has 75% of its area covered by Brazilian semiarid, with its peculiar features. In this state, the dams are constituted in water structure of strategic importance, ensuring, both in time and space, the development and supply of water to population. However, construction of reservoirs results in various impacts that should be carefully observed when deciding on their implementation. One of the impacts identified as negative is the increased evaporation, which constitutes a major component of water balance in reservoirs, especially in arid regions. Several methods for estimating evaporation have been proposed over time, many of them deriving from the Penman equation. This study evaluated six different methods for estimating evaporation in order to determine the most suitable for use in hydrological models for water balance in reservoirs in the state of Ceará. The tested methods were proposed by Penman, Kohler-Nordenson-Fox, Priestley-Taylor, deBruim-Keijman, Brutsaert-Stricker and deBruim. The methods presented good performance when tested for water balance during the dry season, and the Priestley-Taylor was the most appropriate, since the data from de simulated water balance with evaporation estimated by this method were the closest of the water balance data observed from measures of reservoir level and the elevation-volume curve provided by the Company of Management of Water Resources of the state of Ceará - COGERH.

Construction and calibration of TDR probes for volumetric water content estimation in a Distroferric Red Latosol

The determination of volumetric water content of soils is an important factor in irrigation management. Among the indirect methods for estimating, the time-domain reflectometry (TDR) technique has received a significant attention. Like any other technique, it has advantages and disadvantages, but its greatest disadvantage is the need of calibration and high cost of acquisition. The main goal of this study was to establish a calibration model for the TDR equipment, Trase System Model 6050X1, to estimate the volumetric water content in a Distroferric Red Latosol. The calibration was carried out in a laboratory with disturbed soil samples under study, packed in PVC columns of a volume of 0.0078m³. The TDR probes were handcrafted with three rods and 0.20m long. They were vertically installed in soil columns, with a total of five probes per column and sixteen columns. The weightings were carried out in a digital scale, while daily readings of dielectric constant were obtained in TDR equipment. The linear model θν = 0.0103 Ka + 0.1900 to estimate the studied volumetric water content showed an excellent coefficient of determination (0.93), enabling the use of probes in indirect estimation of soil moisture.

Estimation of alluvial recharge in the semiarid

In areas where there is irrigated agriculture, the recuperation of water reserves in alluvial aquifers may occur preferentially due to precipitation. Recharging can be evaluated from variation information of water depth measured in piezometers or observation wells. Thus, the aim of this research is to study the recharge in the alluvial aquifer formed by the Mimoso temporary stream in the semiarid region of Pernambuco (PE), Brazil, using the method of the fluctuation of the water level. This system is typical on the Brazilian Northeast semiarid region, using groundwater for domestic supply and for irrigation on small scale agriculture. Monthly potentiometric levels and rainfall data were used. The selected period for the study, from January 2002 to October 2009, involved extreme events of flooding and droughts as well as regular years, providing a better understanding of the behavior of the alluvial recharge. It was found that the system responds significantly to precipitation events. It was also observed that even with different soil textures in the study area, recharge factors were not significantly different. The study provided a better understanding of the behavior of aquifer recharge and its relationship with the soil and the rainfall events in the region.

Validation study of an overall radiation balance estimation method under Ponta Grossa (PR) Brazil weather conditions

Radiation balance is the fraction of incident solar radiation upon earth surface which is available to be used in several natural processes, such as biological metabolism, water loss by vegetated surfaces, variation of temperature in farming systems and organic decomposition. The present study aimed to assess and validate the performance of two estimation models for Rn in Ponta Grossa city, Paraná State, Brazil. To this end, during the period of 04/01/2008 to 04/30/2011, from radiometric data collected by an automatic weather station set at the Experimental Station, of the State University of Ponta Grossa. We performed a linear regression study by confrontation between measurements made through radiometric balance and Rn estimates obtained from Brunt classical method, and the proposed method. Both models showed excellent performance and were confirmed by the statistical parameters applied. However, the alternative method has the advantage of requiring only global solar radiation values, temperature, and relative humidity.

Estimation methods of reference evapotranspiration (ETo) for Uberlândia -MG

One approach to verify the adequacy of estimation methods of reference evapotranspiration is the comparison with the Penman-Monteith method, recommended by the United Nations of Food and Agriculture Organization - FAO, as the standard method for estimating ET0. This study aimed to compare methods for estimating ET0, Makkink (MK), Hargreaves (HG) and Solar Radiation (RS), with Penman-Monteith (PM). For this purpose, we used daily data of global solar radiation, air temperature, relative humidity and wind speed for the year 2010, obtained through the automatic meteorological station, with latitude 18° 91' 66" S, longitude 48° 25' 05" W and altitude of 869m, at the National Institute of Meteorology situated in the Campus of Federal University of Uberlandia - MG, Brazil. Analysis of results for the period were carried out in daily basis, using regression analysis and considering the linear model y = ax, where the dependent variable was the method of Penman-Monteith and the independent, the estimation of ET0 by evaluated methods. Methodology was used to check the influence of standard deviation of daily ET0 in comparison of methods. The evaluation indicated that methods of Solar Radiation and Penman-Monteith cannot be compared, yet the method of Hargreaves indicates the most efficient adjustment to estimate ETo.

Fuzzy rule-based system for evaluation of uncertainty transaction in cassava chain

A fuzzy ruled-based system was developed in this study and resulted in an index indicating the level of uncertainty related to commercial transactions between cassava growers and their dealers. The fuzzy system was developed based on Transaction Cost Economics approach. The fuzzy system was developed from input variables regarding information sharing between grower and dealer on “Demand/purchase Forecasting”, “Production Forecasting” and “Production Innovation”. The output variable is the level of uncertainty regarding the transaction between seller and buyer agent, which may serve as a system for detecting inefficiencies. Evidences from 27 cassava growers registered in the Regional Development Offices of Tupa and Assis, São Paulo, Brazil, and 48 of their dealers supported the development of the system. The mathematical model indicated that 55% of the growers present a Very High level of uncertainty, 33% present Medium or High. The others present Low or Very Low level of uncertainty. From the model, simulations of external interferences can be implemented in order to improve the degree of uncertainty and, thus, lower transaction costs.

Testing Futures Market Efficiency and Optimal Hedge Ratio Estimation: Evidence from Futures and Options on RTS

This study investigates futures market efficiency and optimal hedge ratio estimation. First, cointegration between spot and futures prices is studied using Johansen method, with two different model specifications. If prices are found cointegrated, restrictions on cointegrating vector and adjustment coefficients are imposed, to account for unbiasedness, weak exogeneity and prediction hypothesis. Second, optimal hedge ratios are estimated using static OLS, and time-varying DVEC and CCC models. In-sample and out-of-sample results for one, two and five period ahead are reported. The futures used in thesis are RTS index, EUR/RUB exchange rate and Brent oil, traded in Futures and options on RTS.(FORTS) For in-sample period, data points were acquired from start of trading of each futures contract, RTS index from August 2005, EUR/RUB exchange rate March 2009 and Brent oil October 2008, lasting till end of May 2011. Out-of-sample period covers start of June 2011, till end of December 2011. Our results indicate that all three asset pairs, spot and futures, are cointegrated. We found RTS index futures to be unbiased predictor of spot price, mixed evidence for exchange rate, and for Brent oil futures unbiasedness was not supported. Weak exogeneity results for all pairs indicated spot price to lead in price discovery process. Prediction hypothesis, unbiasedness and weak exogeneity of futures, was rejected for all asset pairs. Variance reduction results varied between assets, in-sample in range of 40-85 percent and out-of sample in range of 40-96 percent. Differences between models were found small, except for Brent oil in which OLS clearly dominated. Out-of-sample results indicated exceptionally high variance reduction for RTS index, approximately 95 percent.

Development of Cost Estimation with Feedback Information in EPCM Business

Target company of this study is a large machinery company, which is, inter alia, engaged in energy and pulp engineering, procurement and construction management (EPCM) supply business. The main objective of this study was to develop cost estimation of the target company by providing more accurate, reliable and up-to-date information through enterprise resource planning (ERP) system. Another objective was to find cost-effective methods to collect total cost of ownership information to support more informed supplier selection decision making. This study is primarily action-oriented, but also constructive, and it can be divided in two sections: theoretical literature review and empirical study on the abovementioned part of the target company’s business. Development of information collection is, in addition to literature review, based on nearly 30 qualitative interviews of employees at various organizational units, functions and levels at the target company. At the core of development was to make initial data more accurate, reliable and available, a necessary prerequisite for informed use of the information. Certain development suggestions and paths were presented in order to regain confidence in ERP system as information source by reorganizing work breakdown structure and by complementing mere cost information with quantitative, technical and scope information. Several methods to use the information ever more effectively were also discussed. While implementation of the development suggestions outreached the scope of this study, it was forwarded in test environment and interest groups.

Flexible multibody approach in bone strain estimation during physical activity:

Bone strain plays a major role as the activation signal for the bone (re)modeling process, which is vital for keeping bones healthy. Maintaining high bone mineral density reduces the chances of fracture in the event of an accident. Numerous studies have shown that bones can be strengthened with physical exercise. Several hypotheses have asserted that a stronger osteogenic (bone producing) effect results from dynamic exercise than from static exercise. These previous studies are based on short-term empirical research, which provide the motivation for justifying the experimental results with a solid mathematical background. The computer simulation techniques utilized in this work allow for non-invasive bone strain estimation during physical activity at any bone site within the human skeleton. All models presented in the study are threedimensional and actuated by muscle models to replicate the real conditions accurately. The objective of this work is to determine and present loading-induced bone strain values resulting from physical activity. It includes a comparison of strain resulting from four different gym exercises (knee flexion, knee extension, leg press, and squat) and walking, with the results reported for walking and jogging obtained from in-vivo measurements described in the literature. The objective is realized primarily by carrying out flexible multibody dynamics computer simulations. The dissertation combines the knowledge of finite element analysis and multibody simulations with experimental data and information available from medical field literature. Measured subject-specific motion data was coupled with forward dynamics simulation to provide natural skeletal movement. Bone geometries were defined using a reverse engineering approach based on medical imaging techniques. Both computed tomography and magnetic resonance imaging were utilized to explore modeling differences. The predicted tibia bone strains during walking show good agreement with invivo studies found in the literature. Strain measurements were not available for gym exercises; therefore, the strain results could not be validated. However, the values seem reasonable when compared to available walking and running invivo strain measurements. The results can be used for exercise equipment design aimed at strengthening the bones as well as the muscles during workout. Clinical applications in post fracture recovery exercising programs could also be the target. In addition, the methodology introduced in this study, can be applied to investigate the effect of weightlessness on astronauts, who often suffer bone loss after long time spent in the outer space.

Differential Evolution approach and parameter estimation of chaotic

Parameter estimation still remains a challenge in many important applications. There is a need to develop methods that utilize achievements in modern computational systems with growing capabilities. Owing to this fact different kinds of Evolutionary Algorithms are becoming an especially perspective field of research. The main aim of this thesis is to explore theoretical aspects of a specific type of Evolutionary Algorithms class, the Differential Evolution (DE) method, and implement this algorithm as codes capable to solve a large range of problems. Matlab, a numerical computing environment provided by MathWorks inc., has been utilized for this purpose. Our implementation empirically demonstrates the benefits of a stochastic optimizers with respect to deterministic optimizers in case of stochastic and chaotic problems. Furthermore, the advanced features of Differential Evolution are discussed as well as taken into account in the Matlab realization. Test "toycase" examples are presented in order to show advantages and disadvantages caused by additional aspects involved in extensions of the basic algorithm. Another aim of this paper is to apply the DE approach to the parameter estimation problem of the system exhibiting chaotic behavior, where the well-known Lorenz system with specific set of parameter values is taken as an example. Finally, the DE approach for estimation of chaotic dynamics is compared to the Ensemble prediction and parameter estimation system (EPPES) approach which was recently proposed as a possible solution for similar problems.