Kernel-Based Ranking. Methods for Learning and Performance Estimation

Machine learning provides tools for automated construction of predictive models in data intensive areas of engineering and science. The family of regularized kernel methods have in the recent years become one of the mainstream approaches to machine learning, due to a number of advantages the methods share. The approach provides theoretically well-founded solutions to the problems of under- and overfitting, allows learning from structured data, and has been empirically demonstrated to yield high predictive performance on a wide range of application domains. Historically, the problems of classification and regression have gained the majority of attention in the field. In this thesis we focus on another type of learning problem, that of learning to rank. In learning to rank, the aim is from a set of past observations to learn a ranking function that can order new objects according to how well they match some underlying criterion of goodness. As an important special case of the setting, we can recover the bipartite ranking problem, corresponding to maximizing the area under the ROC curve (AUC) in binary classification. Ranking applications appear in a large variety of settings, examples encountered in this thesis include document retrieval in web search, recommender systems, information extraction and automated parsing of natural language. We consider the pairwise approach to learning to rank, where ranking models are learned by minimizing the expected probability of ranking any two randomly drawn test examples incorrectly. The development of computationally efficient kernel methods, based on this approach, has in the past proven to be challenging. Moreover, it is not clear what techniques for estimating the predictive performance of learned models are the most reliable in the ranking setting, and how the techniques can be implemented efficiently. The contributions of this thesis are as follows. First, we develop RankRLS, a computationally efficient kernel method for learning to rank, that is based on minimizing a regularized pairwise least-squares loss. In addition to training methods, we introduce a variety of algorithms for tasks such as model selection, multi-output learning, and cross-validation, based on computational shortcuts from matrix algebra. Second, we improve the fastest known training method for the linear version of the RankSVM algorithm, which is one of the most well established methods for learning to rank. Third, we study the combination of the empirical kernel map and reduced set approximation, which allows the large-scale training of kernel machines using linear solvers, and propose computationally efficient solutions to cross-validation when using the approach. Next, we explore the problem of reliable cross-validation when using AUC as a performance criterion, through an extensive simulation study. We demonstrate that the proposed leave-pair-out cross-validation approach leads to more reliable performance estimation than commonly used alternative approaches. Finally, we present a case study on applying machine learning to information extraction from biomedical literature, which combines several of the approaches considered in the thesis. The thesis is divided into two parts. Part I provides the background for the research work and summarizes the most central results, Part II consists of the five original research articles that are the main contribution of this thesis.

Performance estimation for concretes made with recycled aggregates of construction and demolition waste of some Brazilian cities

The aim of this paper is to verify the influence of composition variability of recycled aggregates (RA) of construction and demolition wastes (CDW) on the performance of concretes. Performance was evaluated building mathematical models for compressive strength, modulus of elasticity and drying shrinkage. To obtain such models, an experimental program comprising 50 concrete mixtures was carried out. Specimens were casted, tested and results for compressive strength, modulus of elasticity and drying shrinkage were statistically analyzed. Models inputs are CDW composition observed at seven Brazilian cities. Results confirm that using RA from CDW for concrete building is quite feasible, independently of its composition, once compressive strength and modulus of elasticity still reached considerable values. We concluded the variability presented by recycled aggregates of CDW does not compromise their use for concrete building. However, this information must be used with caution, and experimental tests should always be performed to certify concrete properties.

Children's performance estimation in mathematics and science tests over a school year: A pilot study

The metacognitve ability to accurately estimate ones performance in a test, is assumed to be of central importance for initializing task-oriented effort. In addition activating adequate problem-solving strategies, and engaging in efficient error detection and correction. Although school children's' ability to estimate their own performance has been widely investigated, this was mostly done under highly-controlled, experimental set-ups including only one single test occasion. Method: The aim of this study was to investigate this metacognitive ability in the context of real achievement tests in mathematics. Developed and applied by a teacher of a 5th grade class over the course of a school year these tests allowed the exploration of the variability of performance estimation accuracy as a function of test difficulty. Results: Mean performance estimations were generally close to actual performance with somewhat less variability compared to test performance. When grouping the children into three achievement levels, results revealed higher accuracy of performance estimations in the high achievers compared to the low and average achievers. In order to explore the generalization of these findings, analyses were also conducted for the same children's tests in their science classes revealing a very similar pattern of results compared to the domain of mathematics. Discussion and Conclusion: By and large, the present study, in a natural environment, confirmed previous laboratory findings but also offered additional insights into the generalisation and the test dependency of students' performances estimations.

High spectral efficiency transmission emulation for non-linear transmission performance estimation for high order modulation formats

We demonstrate a simple method to experimentally evaluate nonlinear transmission performance of high order modulation formats using a low number of channels and channel-like ASE. We verify it's behaviour is consistent with the AWGN model of transmission.

Optical fibre limits:an approach using ASE channel estimation

In this talk we investigate the usage of spectrally shaped amplified spontaneous emission (ASE) in order to emulate highly dispersed wavelength division multiplexed (WDM) signals in an optical transmission system. Such a technique offers various simplifications to large scale WDM experiments. Not only does it offer a reduction in transmitter complexity, removing the need for multiple source lasers, it potentially reduces the test and measurement complexity by requiring only the centre channel of a WDM system to be measured in order to estimate WDM worst case performance. The use of ASE as a test and measurement tool is well established in optical communication systems and several measurement techniques will be discussed [1, 2]. One of the most prevalent uses of ASE is in the measurement of receiver sensitivity where ASE is introduced in order to degrade the optical signal to noise ratio (OSNR) and measure the resulting bit error rate (BER) at the receiver. From an analytical point of view noise has been used to emulate system performance, the Gaussian Noise model is used as an estimate of highly dispersed signals and has had consider- able interest [3]. The work to be presented here extends the use of ASE by using it as a metric to emulate highly dispersed WDM signals and in the process reduce WDM transmitter complexity and receiver measurement time in a lab environment. Results thus far have indicated [2] that such a transmitter configuration is consistent with an AWGN model for transmission, with modulation format complexity and nonlinearities playing a key role in estimating the performance of systems utilising the ASE channel emulation technique. We conclude this work by investigating techniques capable of characterising the nonlinear and damage limits of optical fibres and the resultant information capacity limits. REFERENCES McCarthy, M. E., N. Mac Suibhne, S. T. Le, P. Harper, and A. D. Ellis, “High spectral efficiency transmission emulation for non-linear transmission performance estimation for high order modulation formats," 2014 European Conference on IEEE Optical Communication (ECOC), 2014. 2. Ellis, A., N. Mac Suibhne, F. Gunning, and S. Sygletos, “Expressions for the nonlinear trans- mission performance of multi-mode optical fiber," Opt. Express, Vol. 21, 22834{22846, 2013. Vacondio, F., O. Rival, C. Simonneau, E. Grellier, A. Bononi, L. Lorcy, J. Antona, and S. Bigo, “On nonlinear distortions of highly dispersive optical coherent systems," Opt. Express, Vol. 20, 1022-1032, 2012.

A Model-Based Development and Verification Framework for Distributed System-on-Chip Architecture

The capabilities and thus, design complexity of VLSI-based embedded systems have increased tremendously in recent years, riding the wave of Moore’s law. The time-to-market requirements are also shrinking, imposing challenges to the designers, which in turn, seek to adopt new design methods to increase their productivity. As an answer to these new pressures, modern day systems have moved towards on-chip multiprocessing technologies. New architectures have emerged in on-chip multiprocessing in order to utilize the tremendous advances of fabrication technology. Platform-based design is a possible solution in addressing these challenges. The principle behind the approach is to separate the functionality of an application from the organization and communication architecture of hardware platform at several levels of abstraction. The existing design methodologies pertaining to platform-based design approach don’t provide full automation at every level of the design processes, and sometimes, the co-design of platform-based systems lead to sub-optimal systems. In addition, the design productivity gap in multiprocessor systems remain a key challenge due to existing design methodologies. This thesis addresses the aforementioned challenges and discusses the creation of a development framework for a platform-based system design, in the context of the SegBus platform - a distributed communication architecture. This research aims to provide automated procedures for platform design and application mapping. Structural verification support is also featured thus ensuring correct-by-design platforms. The solution is based on a model-based process. Both the platform and the application are modeled using the Unified Modeling Language. This thesis develops a Domain Specific Language to support platform modeling based on a corresponding UML profile. Object Constraint Language constraints are used to support structurally correct platform construction. An emulator is thus introduced to allow as much as possible accurate performance estimation of the solution, at high abstraction levels. VHDL code is automatically generated, in the form of “snippets” to be employed in the arbiter modules of the platform, as required by the application. The resulting framework is applied in building an actual design solution for an MP3 stereo audio decoder application.

The effects of estimating good vs. poor knowledge of results during acquisition of a spatial motor task

Recent studies have shown that providing learners Knowledge of Results (KR) after “good trials” rather than “poor trials” is superior for learning. The present study examined whether requiring participants to estimate their three best or three worst trials in a series of six trial blocks before receiving KR would prove superior to learning compared to not estimating their performance. Participants were required to push and release a slide along a confined pathway using their non-dominant hand to a target distance (133cm). The retention and transfer data suggest those participants who received KR after good trials demonstrated superior learning and performance estimations compared to those receiving KR after poor trials. The results of the present experiment offer an important theoretical extension in our understanding of the role of KR content and performance estimation on motor skill learning.

Efeitos genéticos aditivos diretos e maternos e heterozigóticos sobre os desempenhos pré e pós-desmama em uma população multirracial Aberdeen Angus × Nelore

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

What are the metacognitive costs of young children's overconfidence?

Children typically hold very optimistic views of their own skills but so far, only a few studies have investigated possible correlates of the ability to predict performance accurately. Therefore, this study examined the role of individual differences in performance estimation accuracy as a global metacognitive index for different monitoring and control skills (item-level judgments of learning [JOLs] and confidence judgments [CJs]), metacognitive control processes (allocation of study time and control of answers), and executive functions (cognitive flexibility, inhibition, working memory) in 6-year-olds (N=93). The three groups of under estimators, realists and over estimators differed significantly in their monitoring and control abilities: the under estimators outperformed the over estimators by showing a higher discrimination in CJs between correct and incorrect recognition. Also, the under estimators scored higher on the adequate control of incorrectly recognized items. Regarding the interplay of monitoring and control processes, under estimators spent more time studying items with low JOLs, and relied more systematically on their monitoring when controlling their recognition compared to over estimators. At the same time, the three groups did not differ significantly from each other in their executive functions. Overall, results indicate that differences in performance estimation accuracy are systematically related to other global and item-level metacognitive monitoring and control abilities in children as young as six years of age, while no meaningful association between performance estimation accuracy and executive functions was found.

Prediction of PM10 concentrations using Fuzzy c-Means and ANN

Salamanca has been considered among the most polluted cities in Mexico. The vehicular park, the industry and the emissions produced by agriculture, as well as orography and climatic characteristics have propitiated the increment in pollutant concentration of Particulate Matter less than 10 μg/m3 in diameter (PM10). In this work, a Multilayer Perceptron Neural Network has been used to make the prediction of an hour ahead of pollutant concentration. A database used to train the Neural Network corresponds to historical time series of meteorological variables (wind speed, wind direction, temperature and relative humidity) and air pollutant concentrations of PM10. Before the prediction, Fuzzy c-Means clustering algorithm have been implemented in order to find relationship among pollutant and meteorological variables. These relationship help us to get additional information that will be used for predicting. Our experiments with the proposed system show the importance of this set of meteorological variables on the prediction of PM10 pollutant concentrations and the neural network efficiency. The performance estimation is determined using the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The results shown that the information obtained in the clustering step allows a prediction of an hour ahead, with data from past 2 hours

ANN and Fuzzy c-Means applied to environmental pollution prediction

Salamanca, situated in center of Mexico is among the cities which suffer most from the air pollution in Mexico. The vehicular park and the industry, as well as orography and climatic characteristics have propitiated the increment in pollutant concentration of Sulphur Dioxide (SO2). In this work, a Multilayer Perceptron Neural Network has been used to make the prediction of an hour ahead of pollutant concentration. A database used to train the Neural Network corresponds to historical time series of meteorological variables and air pollutant concentrations of SO2. Before the prediction, Fuzzy c-Means and K-means clustering algorithms have been implemented in order to find relationship among pollutant and meteorological variables. Our experiments with the proposed system show the importance of this set of meteorological variables on the prediction of SO2 pollutant concentrations and the neural network efficiency. The performance estimation is determined using the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE). The results showed that the information obtained in the clustering step allows a prediction of an hour ahead, with data from past 2 hours.

Relationship between estimation and real motor performance in school-age children

The relationship between estimated and real motor competences was analyzed for several tasks. Participants were 303 children (160 boys and 143 girls), which had between 6 and 10 years of age (M=8.63, SD=1.16). None of the children presented developmental difficulties or learning disabilities, and all attended age-appropriate classes. Children were divided into three groups according to their age: group 1 (N= 102; age range: 6.48-8.01 years); group 2 (N= 101; age range: 8.02-9.22 years); and group 3 (N=100; age range: 9.24-10.93 years). Children were asked to predict their maximum distance for a locomotor, a manipulative, and a balance task, prior to performing those tasks. Children’s estimations were compared with their real performance to determine their accuracy. Children had, in general, a tendency to overestimate their performance (standing long jump: 56.11%, kicking: 63.37%, throwing: 73.60%, and Walking Backwards (WB) on a balance beam: 45.21%), and older children tended to be more accurate, except for the manipulative tasks. Furthermore, the relationship between estimation and real performance in children with different levels of motor coordination (Köperkoordinationstest für Kinder, KTK) was analyzed. The 75 children with the highest score comprised the Highest Motor Coordination (HMC) group, and the 78 children with the lowest score were placed in the Lowest Motor Coordination (LMC) group. There was a tendency for LMC and HMC children to overestimate their skills at all tasks, except for the HMC group at the WB task. Children with the HMC level tended to be more accurate when predicting their motor performance; however, differences in absolute percent error were only significant for the throwing and WB tasks. In conclusion, children display a tendency to overestimate their performance independently of their motor coordination level and task. This fact may be determinant to the development of their motor competences, since they are more likely to engage and persist in motor tasks, but it might also increase the occurrence of unintended injuries.

Performance de l'estimation du poids foetal par echographie pour les foetus < or = 2000 g et les macrosomes (> or = 4000 g) [Performance of ultrasound estimation of fetal weight in fetuses weighing < or = 2000 g and more than 4000 g].

Decreasing perinatal morbidity and mortality is one of the main goals of obstetrics. Prognosis of preterm births depends on gestational age and birthweight. Multidisciplinary management is discussed with the parents according to these two parameters. In other circumstances, a suspected macrosomy will influence the management of the last weeks of pregnancy. Induction of labor or Cesarean delivery will be considered to avoid shoulder dystocia, brachial plexus injury or perinatal asphyxia. Birthweight needs to be estimated with accuracy, and this article describes the efficiency of various ultrasound weight estimation formulae for small and large fetuses.

Accuracy of indirect estimation of power output from uphill performance in cycling.

PURPOSE: To use measurement by cycling power meters (Pmes) to evaluate the accuracy of commonly used models for estimating uphill cycling power (Pest). Experiments were designed to explore the influence of wind speed and steepness of climb on accuracy of Pest. The authors hypothesized that the random error in Pest would be largely influenced by the windy conditions, the bias would be diminished in steeper climbs, and windy conditions would induce larger bias in Pest. METHODS: Sixteen well-trained cyclists performed 15 uphill-cycling trials (range: length 1.3-6.3 km, slope 4.4-10.7%) in a random order. Trials included different riding position in a group (lead or follow) and different wind speeds. Pmes was quantified using a power meter, and Pest was calculated with a methodology used by journalists reporting on the Tour de France. RESULTS: Overall, the difference between Pmes and Pest was -0.95% (95%CI: -10.4%, +8.5%) for all trials and 0.24% (-6.1%, +6.6%) in conditions without wind (<2 m/s). The relationship between percent slope and the error between Pest and Pmes were considered trivial. CONCLUSIONS: Aerodynamic drag (affected by wind velocity and orientation, frontal area, drafting, and speed) is the most confounding factor. The mean estimated values are close to the power-output values measured by power meters, but the random error is between ±6% and ±10%. Moreover, at the power outputs (>400 W) produced by professional riders, this error is likely to be higher. This observation calls into question the validity of releasing individual values without reporting the range of random errors.

Use of individual race results in the estimation of genetic parameters of trotting performance for Finnhorse and Standardbred trotters

Selostus: Ravikilpailumenestysmittojen periytymisasteet ja toistumiskertoimet kilpailukohtaisten tulosten perusteella