Overall evaluation of the PETS2009 results

This paper presents the results of the crowd image analysis challenge, as part of the PETS 2009 workshop. The evaluation is carried out using a selection of the metrics available in the Video Analysis and Content Extraction (VACE) program and the CLassification of Events, Activities, and Relationships (CLEAR) consortium. The evaluation highlights the strengths of the authors’ systems in areas such as precision, accuracy and robustness.

PETS2009 and Winter-PETS 2009 results: a combined evaluation

This paper presents the results of the crowd image analysis challenge of the Winter PETS 2009 workshop. The evaluation is carried out using a selection of the metrics developed in the Video Analysis and Content Extraction (VACE) program and the CLassification of Events, Activities, and Relationships (CLEAR) consortium [13]. The evaluation highlights the detection and tracking performance of the authors’systems in areas such as precision, accuracy and robustness. The performance is also compared to the PETS 2009 submitted results.

PETS2010 and PETS2009 evaluation of results using individual ground truthed single views

This paper presents the results of the crowd image analysis challenge of the PETS2010 workshop. The evaluation was carried out using a selection of the metrics developed in the Video Analysis and Content Extraction (VACE) program and the CLassification of Events, Activities, and Relationships (CLEAR) consortium. The PETS 2010 evaluation was performed using new ground truthing create from each independant two dimensional view. In addition, the performance of the submissions to the PETS 2009 and Winter-PETS 2009 were evaluated and included in the results. The evaluation highlights the detection and tracking performance of the authors’ systems in areas such as precision, accuracy and robustness.

Evaluation of Earth system models using modern and palaeo-observations: the state-of-the-art

Earth system models are increasing in complexity and incorporating more processes than their predecessors, making them important tools for studying the global carbon cycle. However, their coupled behaviour has only recently been examined in any detail, and has yielded a very wide range of outcomes, with coupled climate-carbon cycle models that represent land-use change simulating total land carbon stores by 2100 that vary by as much as 600 Pg C given the same emissions scenario. This large uncertainty is associated with differences in how key processes are simulated in different models, and illustrates the necessity of determining which models are most realistic using rigorous model evaluation methodologies. Here we assess the state-of-the-art with respect to evaluation of Earth system models, with a particular emphasis on the simulation of the carbon cycle and associated biospheric processes. We examine some of the new advances and remaining uncertainties relating to (i) modern and palaeo data and (ii) metrics for evaluation, and discuss a range of strategies, such as the inclusion of pre-calibration, combined process- and system-level evaluation, and the use of emergent constraints, that can contribute towards the development of more robust evaluation schemes. An increasingly data-rich environment offers more opportunities for model evaluation, but it is also a challenge, as more knowledge about data uncertainties is required in order to determine robust evaluation methodologies that move the field of ESM evaluation from "beauty contest" toward the development of useful constraints on model behaviour.

Evaluation of biospheric components in Earth system models using modern and palaeo-observations: the state-of-the-art

Earth system models (ESMs) are increasing in complexity by incorporating more processes than their predecessors, making them potentially important tools for studying the evolution of climate and associated biogeochemical cycles. However, their coupled behaviour has only recently been examined in any detail, and has yielded a very wide range of outcomes. For example, coupled climate–carbon cycle models that represent land-use change simulate total land carbon stores at 2100 that vary by as much as 600 Pg C, given the same emissions scenario. This large uncertainty is associated with differences in how key processes are simulated in different models, and illustrates the necessity of determining which models are most realistic using rigorous methods of model evaluation. Here we assess the state-of-the-art in evaluation of ESMs, with a particular emphasis on the simulation of the carbon cycle and associated biospheric processes. We examine some of the new advances and remaining uncertainties relating to (i) modern and palaeodata and (ii) metrics for evaluation. We note that the practice of averaging results from many models is unreliable and no substitute for proper evaluation of individual models. We discuss a range of strategies, such as the inclusion of pre-calibration, combined process- and system-level evaluation, and the use of emergent constraints, that can contribute to the development of more robust evaluation schemes. An increasingly data-rich environment offers more opportunities for model evaluation, but also presents a challenge. Improved knowledge of data uncertainties is still necessary to move the field of ESM evaluation away from a "beauty contest" towards the development of useful constraints on model outcomes.

Performance evaluation of crowd image analysis using the PETS2009 dataset

This paper presents the PETS2009 outdoor crowd image analysis surveillance dataset and the performance evaluation of people counting, detection and tracking results using the dataset submitted to five IEEE Performance Evaluation of Tracking and Surveillance (PETS) workshops. The evaluation was carried out using well established metrics developed in the Video Analysis and Content Extraction (VACE) programme and the CLassification of Events, Activities, and Relationships (CLEAR) consortium. The comparative evaluation highlights the detection and tracking performance of the authors’ systems in areas such as precision, accuracy and robustness and provides a brief analysis of the metrics themselves to provide further insights into the performance of the authors’ systems.

Spatial evaluation of volcanic ash forecasts using satellite observations

The decision to close airspace in the event of a volcanic eruption is based on hazard maps of predicted ash extent. These are produced using output from volcanic ash transport and dispersion (VATD)models. In this paper an objectivemetric to evaluate the spatial accuracy of VATD simulations relative to satellite retrievals of volcanic ash is presented. The 5 metric is based on the fractions skill score (FSS). Thismeasure of skill provides more information than traditional point-bypoint metrics, such as success index and Pearson correlation coefficient, as it takes into the account spatial scale overwhich skill is being assessed. The FSS determines the scale overwhich a simulation has skill and can differentiate between a "near miss" and a forecast that is badly misplaced. The 10 idealised scenarios presented show that even simulations with considerable displacement errors have useful skill when evaluated over neighbourhood scales of 200–700km2. This method could be used to compare forecasts produced by different VATDs or using different model parameters, assess the impact of assimilating satellite retrieved ash data and evaluate VATD forecasts over a long time period.

Using metrics from complex networks to evaluate machine translation

Establishing metrics to assess machine translation (MT) systems automatically is now crucial owing to the widespread use of MT over the web. In this study we show that such evaluation can be done by modeling text as complex networks. Specifically, we extend our previous work by employing additional metrics of complex networks, whose results were used as input for machine learning methods and allowed MT texts of distinct qualities to be distinguished. Also shown is that the node-to-node mapping between source and target texts (English-Portuguese and Spanish-Portuguese pairs) can be improved by adding further hierarchical levels for the metrics out-degree, in-degree, hierarchical common degree, cluster coefficient, inter-ring degree, intra-ring degree and convergence ratio. The results presented here amount to a proof-of-principle that the possible capturing of a wider context with the hierarchical levels may be combined with machine learning methods to yield an approach for assessing the quality of MT systems. (C) 2010 Elsevier B.V. All rights reserved.

A Proposal for System Performance Evaluation for an ERP Knowledge Extraction Model

Much has been researched and discussed in the importance played by knowledge in organizations. We are witnessing the establishment of the knowledge economy, but this "new economy" brings in itself a whole complex system of metrics and evaluations, and cannot be dissociated from it. Due to its importance, the initiatives of knowledge management must be continually assessed on their progress in order to verify whether they are moving towards achieving the goals of success. Thus, good measurement practices should include not only how the organization quantifies its knowledge capital, but also how resources are allocated to supply their growth. Thinking about the aspects listed above, this paper presents an approach to a model for Knowledge extraction using an ERP system, suggesting the establishment of a set of indicators for assessing organizational performance. The objective is to evaluate the implementation of projects of knowledge management and thus observe the general development of the organization.

A multimetric index based on fish fauna for the evaluation of the biotic integrity of streams at a mesohabitat scale

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Microleakage evaluation of pit and fissure sealants done with different procedures, materials, and laser after invasive technique

This study evaluated the microleakage of pit and fissure sealants after different surface preparation (invasive technique and laser irradiation) and the use of different materials (fluoride resin-filled sealant, resin-modified glass ionomer cement and adhesive system). Eighty-four pre molars were used in this study, which were divided into seven groups. After the accomplishment of the different treatments, these were submitted to thermocycling process and assess for microleakage by examination under an epifluorescent microscope and scored zero to seven. Two specimens of each group were observed under scanning electron microscope (SEM). The results showed that laser irradiation did not lessen microleakage in pit and fissures when using a filled-resin sealant with fluoride or a resin-modified glass ionomer cement. The use of laser irradiation and adhesive system, followed by a resin-filled sealant with fluoride, showed the lowest microleakage scores in pit and fissures. Comparing this group to the resin-modified glass ionomer cement group, there was statistical significance. The use of a adhesive system decreased microleakage when using a fluoride resin-filled sealant with or without previous laser irradiation; although it was not statistically significant.

In vitro evaluation of marginal leakage in bonded restorations, with mechanical or chemical-mechanical (Carisolv) removal of carious tissue

This study carried out an in vitro evaluation and comparison of the occurrence of marginal leakage in bonded restorations using mechanical or chemical-mechanical (Carisolv) removal of carious tissue. For that purpose, 40 extracted decayed human molars were divided into 4 groups: GI (burs + Prime & Bond NT + TPH), GII (Carisolv + Prime & Bond NT + TPH), GIII (burs + SBMP + Z100) and GIV (Carisolv + SBMP + Z100). After accomplishment of the restorations and thermal cycling, the teeth were exposed to dye, sectioned and qualitatively evaluated. The results demonstrated that the system of removal of carious tissue did not influence the results of microleakage at any of the cavity margins. At dentinal margins, use of the Prime & Bond NT + TPH restorative system allowed the occurrence of less microleakage than the SBMP + Z100 system.

Electromyographic evaluation of the upper lip according to the breathing mode: A longitudinal study

The present study aimed at analyzing and comparing longitudinally the EMG (electromyographic activity) of the superior orbicularis oris muscle according to the breathing mode. The sample, 38 adolescents with Angle Class II Division 1 malocclusion with predominantly nose (PNB) or mouth (PMB) breathing, was evaluated at two different periods, with a two-year interval between them. For that purpose, a 16-channel electromyography machine was employed, which was properly calibrated in a PC equipped with an analogue-digital converter, with utilization of surface, passive and bipolar electrodes. The RMS data (root mean square) were collected at rest and in 12 movements and normalized according to time and amplitude, by the peak value of EMG, in order to allow comparisons between subjects and between periods. Comparison of the muscle function of PNB and PMB subjects at period 1 (P1), period 2 (P2) and the variation between periods (Δ) did not reveal statistically significant differences between groups (p < 0.05). However, longitudinal evaluation of the muscle function in PNB and PMB subjects demonstrated different evolutions in the percentage of required EMG for accomplishment of the movements investigated. It was possible to conclude that there are differences in the percentage of electric activity of the upper lip with the growth of the subjects according to the breathing mode.

Evaluation of gene expression SAP5, LIP9, and PLB2 of candida albicans biofilms after photodynamic inactivation

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Systematic Evaluation of Extraction Methods for Multiplatform-Based Metabotyping: Application to the Fasciola hepatica Metabolome

Combining data from multiple analytical platforms is essential for comprehensive study of the molecular phenotype (metabotype) of a given biological sample. The metabolite profiles generated are intrinsically dependent on the analytical platforms, each requiring optimization of instrumental parameters, separation conditions, and sample extraction to deliver maximal biological information. An in-depth evaluation of extraction protocols for characterizing the metabolome of the hepatobiliary fluke Fasciola hepatica, using ultra performance liquid chromatography and capillary electrophoresis coupled with mass spectroscopy is presented. The spectrometric methods were characterized by performance, and metrics of merit were established, including precision, mass accuracy, selectivity, sensitivity, and platform stability. Although a core group of molecules was common to all methods, each platform contributed a unique set, whereby 142 metabolites out of 14,724 features were identified. A mixture design revealed that the chloroform:methanol:water proportion of 15:59:26 was globally the best composition for metabolite extraction across UPLC-MS and CE-MS platforms accommodating different columns and ionization modes. Despite the general assumption of the necessity of platform-adapted protocols for achieving effective metabotype characterization, we show that an appropriately designed single extraction procedure is able to fit the requirements of all technologies. This may constitute a paradigm shift in developing efficient protocols for high-throughput metabolite profiling with more-general analytical applicability.