Semantic evaluation at large scale (SEALS)

This paper describes the main goals and outcomes of the EU-funded Framework 7 project entitled Semantic Evaluation at Large Scale (SEALS). The growth and success of the Semantic Web is built upon a wide range of Semantic technologies from ontology engineering tools through to semantic web service discovery and semantic search. The evaluation of such technologies ? and, indeed, assessments of their mutual compatibility ? is critical for their sustained improvement and adoption. The SEALS project is creating an open and sustainable platform on which all aspects of an evaluation can be hosted and executed and has been designed to accommodate most technology types. It is envisaged that the platform will become the de facto repository of test datasets and will allow anyone to organise, execute and store the results of technology evaluations free of charge and without corporate bias. The demonstration will show how individual tools can be prepared for evaluation, uploaded to the platform, evaluated according to some criteria and the subsequent results viewed. In addition, the demonstration will show the flexibility and power of the SEALS Platform for evaluation organisers by highlighting some of the key technologies used.

The cultural adaptation and validation of the ""Burn Specific Health Scale-Revised"" (BSHS-R): Version for Brazilian burn victims

Background: The Burns Specific Health Scale-Revised (BSHS-R) is of easy application, can be self-administered, and it is considered a good scale to evaluate various important life aspects of burn victims. Objectives: To translate and culturally adapt the BSHS-R into the Brazilian-Portuguese language and to evaluate the internal consistency and convergent validity of the translated BSHS-R. Methods: The cultural adaptation of the BSHS-R included translation and back-translation, discussions with professionals and patients to ensure conceptual equivalence, semantic evaluation, and pre-test of the instrument. The Final Brazilian-Portuguese Version (FBPV) of the BSHS-R was tested on a group of 115 burn patients for internal consistency and validity of construct (using the Rosenberg Self-Esteem Scale (RSES) and the Beck Depression Inventory (BDI)). Results: All values of Cronbach`s alpha were greater than. 8, demonstrating that the internal consistency of the FBPV was very high. Self-esteem was highly correlated with affect and body image (r = .59, p < .001), and with interpersonal relationships (T = .51, p < .001). Correlations between the domains of the FBPV and the BDI were all negative but larger in magnitude than the correlations with RSES. Depression was highly correlated with affect and body image (r = -77, p < .001), and with interpersonal relationships (r = -67, p < .001). Conclusions: The results showed that the adapted version of the BSHS-R into Brazilian-Portuguese fulfills the validity and reliability criteria required from an instrument of health status assessment for burn patients. (C) 2008 Elsevier Ltd and ISBI. All rights reserved.

Cross-cultural adaptation and psychometric properties of the Brazilian-Portuguese version of the Duke Anticoagulation Satisfaction Scale

Aim. The aim of this study was to evaluate the internal reliability and validity of the BrazilianPortuguese version of Duke Anticoagulation Satisfaction Scale (DASS) among cardiovascular patients. Background. Oral anticoagulation is widely used to prevent and treat thromboembolic events in several conditions, especially in cardiovascular diseases; however, this therapy can induce dissatisfaction and reduce the quality of life. Design. Methodological and cross-sectional research design. Methods. The cultural adaptation of the DASS included the translation and back-translation, discussions with healthcare professionals and patients to ensure conceptual equivalence, semantic evaluation and instrument pretest. The BrazilianPortuguese version of the DASS was tested among subjects followed in a university hospital anticoagulation outpatient clinic. The psychometric properties were assessed by construct validity (convergent, known groups and dimensionality) and internal consistency/reliability (Cronbachs alpha). Results. A total of 180 subjects under oral anticoagulation formed the baseline validation population. DASS total score and SF-36 domain correlations were moderate for General health (r = -0.47, p < 0.01), Vitality (r = -0.44, p < 0.01) and Mental health (r = -0.42, p < 0.01) (convergent). Age and length on oral anticoagulation therapy (in years) were weakly correlated with total DASS score and most of the subscales, except Limitation (r = -0.375, p < 0.01) (Known groups). The Cronbachs alpha coefficient was 0.79 for the total scale, and it ranged from 0.76 (hassles and burdens)0.46 (psychological impact) among the domains, confirming the internal consistency reliability. Conclusions. The BrazilianPortuguese version of the DASS has shown levels of reliability and validity comparable with the original English version. Relevance to clinical practice. Healthcare practitioners and researchers need internationally validated measurement tools to compare outcomes of interventions in clinical management and research tools in oral anticoagulation therapy.

DAEDALUS at SemEval-2014 Task 9: Comparing approaches for sentiment analysis in twitter

This paper describes our participation at SemEval- 2014 sentiment analysis task, in both contextual and message polarity classification. Our idea was to com- pare two different techniques for sentiment analysis. First, a machine learning classifier specifically built for the task using the provided training corpus. On the other hand, a lexicon-based approach using natural language processing techniques, developed for a ge- neric sentiment analysis task with no adaptation to the provided training corpus. Results, though far from the best runs, prove that the generic model is more robust as it achieves a more balanced evaluation for message polarity along the different test sets.

Semantic web adoption: online tools for web evaluation and metadata extraction

This work briefly analyses the difficulties to adopt the Semantic Web, and in particular proposes systems to know the present level of migration to the different technologies that make up the Semantic Web. It focuses on the presentation and description of two tools, DigiDocSpider and DigiDocMetaEdit, designed with the aim of verifYing, evaluating, and promoting its implementation.

The state of semantic technology today - overview of the first SEALS evaluation campaigns

This paper describes the first five SEALS Evaluation Campaigns over the semantic technologies covered by the SEALS project (ontology engineering tools, ontology reasoning tools, ontology matching tools, semantic search tools, and semantic web service tools). It presents the evaluations and test data used in these campaigns and the tools that participated in them along with a comparative analysis of their results. It also presents some lessons learnt after the execution of the evaluation campaigns and draws some final conclusions.

Infrastructure and Workflow for the Formal Evaluation of Semantic Search Technologies

This paper describes an infrastructure for the automated evaluation of semantic technologies and, in particular, semantic search technologies. For this purpose, we present an evaluation framework which follows a service-oriented approach for evaluating semantic technologies and uses the Business Process Execution Language (BPEL) to define evaluation workflows that can be executed by process engines. This framework supports a variety of evaluations, from different semantic areas, including search, and is extendible to new evaluations. We show how BPEL addresses this diversity as well as how it is used to solve specific challenges such as heterogeneity, error handling and reuse

Semantic Interoperability: Perspectives and Technology Evaluation

Interoperability between semantic technologies is a must because they need to be in communication to interchange ontologies and use them in the distributed and open environment of the SemanticWeb. However, such interoperability is not straightforward due to the high heterogeneity in such technologies. This chapter describes the problem of semantic technology interoperability from two different perspectives. First, from a theoretical perspective by presenting an overview of the different factors that affect interoperability and, second, from a practical perspective by reusing evaluation methods and applying them to six current semantic technologies in order to assess their interoperability.

SemQuaRE - An extension of the SQuaRE quality model for the evaluation of semantic technologies

To correctly evaluate semantic technologies and to obtain results that can be easily integrated, we need to put evaluations under the scope of a unique software quality model. This paper presents SemQuaRE, a quality model for semantic technologies. SemQuaRE is based on the SQuaRE standard and describes a set of quality characteristics specific to semantic technologies and the quality measures that can be used for their measurement. It also provides detailed formulas for the calculation of such measures. The paper shows that SemQuaRE is complete with respect to current evaluation trends and that it has been successfully applied in practice.

Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping

The Leximancer system is a relatively new method for transforming lexical co-occurrence information from natural language into semantic patterns in an unsupervised manner. It employs two stages of co-occurrence information extraction-semantic and relational-using a different algorithm for each stage. The algorithms used are statistical, but they employ nonlinear dynamics and machine learning. This article is an attempt to validate the output of Leximancer, using a set of evaluation criteria taken from content analysis that are appropriate for knowledge discovery tasks.

Cross ontology query answering on the semantic web:an initial evaluation

PowerAqua is a Question Answering system, which takes as input a natural language query and is able to return answers drawn from relevant semantic resources found anywhere on the Semantic Web. In this paper we provide two novel contributions: First, we detail a new component of the system, the Triple Similarity Service, which is able to match queries effectively to triples found in different ontologies on the Semantic Web. Second, we provide a first evaluation of the system, which in addition to providing data about PowerAqua's competence, also gives us important insights into the issues related to using the Semantic Web as the target answer set in Question Answering. In particular, we show that, despite the problems related to the noisy and incomplete conceptualizations, which can be found on the Semantic Web, good results can already be obtained.

Ecological interface design for pasteurizer II: A process description of semantic mapping

Ecological interface design (EID) is proving to be a promising approach to the design of interfaces for complex dynamic systems. Although the principles of EID and examples of its effective use are widely available, few readily available examples exist of how the individual displays that constitute an ecological interface are developed. This paper presents the semantic mapping process within EID in the context of prior theoretical work in this area. The semantic mapping process that was used in developing an ecological interface for the Pasteurizer II microworld is outlined, and the results of an evaluation of the ecological interface against a more conventional interface are briefly presented. Subjective reports indicate features of the ecological interface that made it particularly valuable for participants. Finally, we outline the steps of an analytic process for using EID. The findings presented here can be applied in the design of ecological interfaces or of configural displays for dynamic processes.

Episodic and semantic memory in children with mesial temporal sclerosis

The aim of this study was to analyze semantic and episodic memory deficits in children with mesial temporal sclerosis (MTS) and their correlation with clinical epilepsy variables. For this purpose, 19 consecutive children and adolescents with MTS (8 to 16 years old) were evaluated and their performance on five episodic memory tests (short- and long-term memory and learning) and four semantic memory tests was compared with that of 28 healthy volunteers. Patients performed worse on tests of immediate and delayed verbal episodic memory, visual episodic memory, verbal and visual learning, mental scanning for semantic clues, object naming, word definition, and repetition of sentences. Clinical variables such as early age at seizure onset, severity of epilepsy, and polytherapy impaired distinct types of memory. These data confirm that children with MTS have episodic memory deficits and add new information on semantic memory. The data also demonstrate that clinical variables contribute differently to episodic and semantic memory performance. (C) 2011 Elsevier Inc. All rights reserved.

Fostering e-participation sustainability through a BPM-driven semantic model

According to a recent Eurobarometer survey (2014), 68% of Europeans tend not to trust national governments. As the increasing alienation of citizens from politics endangers democracy and welfare, governments, practitioners and researchers look for innovative means to engage citizens in policy matters. One of the measures intended to overcome the so-called democratic deficit is the promotion of civic participation. Digital media proliferation offers a set of novel characteristics related to interactivity, ubiquitous connectivity, social networking and inclusiveness that enable new forms of societal-wide collaboration with a potential impact on leveraging participative democracy. Following this trend, e-Participation is an emerging research area that consists in the use of Information and Communication Technologies to mediate and transform the relations among citizens and governments towards increasing citizens’ participation in public decision-making. However, despite the widespread efforts to implement e-Participation through research programs, new technologies and projects, exhaustive studies on the achieved outcomes reveal that it has not yet been successfully incorporated in institutional politics. Given the problems underlying e-Participation implementation, the present research suggested that, rather than project-oriented efforts, the cornerstone for successfully implementing e-Participation in public institutions as a sustainable added-value activity is a systematic organisational planning, embodying the principles of open-governance and open-engagement. It further suggested that BPM, as a management discipline, can act as a catalyst to enable the desired transformations towards value creation throughout the policy-making cycle, including political, organisational and, ultimately, citizen value. Following these findings, the primary objective of this research was to provide an instrumental model to foster e-Participation sustainability across Government and Public Administration towards a participatory, inclusive, collaborative and deliberative democracy. The developed artefact, consisting in an e-Participation Organisational Semantic Model (ePOSM) underpinned by a BPM-steered approach, introduces this vision. This approach to e-Participation was modelled through a semi-formal lightweight ontology stack structured in four sub-ontologies, namely e-Participation Strategy, Organisational Units, Functions and Roles. The ePOSM facilitates e-Participation sustainability by: (1) Promoting a common and cross-functional understanding of the concepts underlying e-Participation implementation and of their articulation that bridges the gap between technical and non-technical users; (2) Providing an organisational model which allows a centralised and consistent roll-out of strategy-driven e-Participation initiatives, supported by operational units dedicated to the execution of transformation projects and participatory processes; (3) Providing a standardised organisational structure, goals, functions and roles related to e-Participation processes that enhances process-level interoperability among government agencies; (4) Providing a representation usable in software development for business processes’ automation, which allows advanced querying using a reasoner or inference engine to retrieve concrete and specific information about the e-Participation processes in place. An evaluation of the achieved outcomes, as well a comparative analysis with existent models, suggested that this innovative approach tackling the organisational planning dimension can constitute a stepping stone to harness e-Participation value.

Analysis and evaluation of techniques for the extraction of classes in the ontology learning process

This paper analyzes and evaluates, in the context of Ontology learning, some techniques to identify and extract candidate terms to classes of a taxonomy. Besides, this work points out some inconsistencies that may be occurring in the preprocessing of text corpus, and proposes techniques to obtain good terms candidate to classes of a taxonomy.