Semantic scheduling of virtualized infrastructures for scientific workflows

Virtualized Infrastructures are a promising way for providing flexible and dynamic computing solutions for resourceconsuming tasks. Scientific Workflows are one of these kind of tasks, as they need a large amount of computational resources during certain periods of time. To provide the best infrastructure configuration for a workflow it is necessary to explore as many providers as possible taking into account different criteria like Quality of Service, pricing, response time, network latency, etc. Moreover, each one of these new resources must be tuned to provide the tools and dependencies required by each of the steps of the workflow. Working with different infrastructure providers, either public or private using their own concepts and terms, and with a set of heterogeneous applications requires a framework for integrating all the information about these elements. This work proposes semantic technologies for describing and integrating all the information about the different components of the overall system and a set of policies created by the user. Based on this information a scheduling process will be performed to generate an infrastructure configuration defining the set of virtual machines that must be run and the tools that must be deployed on them.

OPPL-Galaxy, a Galaxy tool for enhancing ontology exploitation as part of bioinformatics workflows

Biomedical ontologies are key elements for building up the Life Sciences Semantic Web. Reusing and building biomedical ontologies requires flexible and versatile tools to manipulate them efficiently, in particular for enriching their axiomatic content. The Ontology Pre Processor Language (OPPL) is an OWL-based language for automating the changes to be performed in an ontology. OPPL augments the ontologists’ toolbox by providing a more efficient, and less error-prone, mechanism for enriching a biomedical ontology than that obtained by a manual treatment. Results We present OPPL-Galaxy, a wrapper for using OPPL within Galaxy. The functionality delivered by OPPL (i.e. automated ontology manipulation) can be combined with the tools and workflows devised within the Galaxy framework, resulting in an enhancement of OPPL. Use cases are provided in order to demonstrate OPPL-Galaxy’s capability for enriching, modifying and querying biomedical ontologies. Conclusions Coupling OPPL-Galaxy with other bioinformatics tools of the Galaxy framework results in a system that is more than the sum of its parts. OPPL-Galaxy opens a new dimension of analyses and exploitation of biomedical ontologies, including automated reasoning, paving the way towards advanced biological data analyses.

Common motifs in scientific workflows: An empirical analysis

While workflow technology has gained momentum in the last decade as a means for specifying and enacting computational experiments in modern science, reusing and repurposing existing workflows to build new scientific experiments is still a daunting task. This is partly due to the difficulty that scientists experience when attempting to understand existing workflows, which contain several data preparation and adaptation steps in addition to the scientifically significant analysis steps. One way to tackle the understandability problem is through providing abstractions that give a high-level view of activities undertaken within workflows. As a first step towards abstractions, we report in this paper on the results of a manual analysis performed over a set of real-world scientific workflows from Taverna and Wings systems. Our analysis has resulted in a set of scientific workflow motifs that outline i) the kinds of data intensive activities that are observed in workflows (data oriented motifs), and ii) the different manners in which activities are implemented within workflows (workflow oriented motifs). These motifs can be useful to inform workflow designers on the good and bad practices for workflow development, to inform the design of automated tools for the generation of workflow abstractions, etc.

Common motifs in scientific workflows: An empirical analysis

Workflow technology continues to play an important role as a means for specifying and enacting computational experiments in modern science. Reusing and re-purposing workflows allow scientists to do new experiments faster, since the workflows capture useful expertise from others. As workflow libraries grow, scientists face the challenge of finding workflows appropriate for their task, understanding what each workflow does, and reusing relevant portions of a given workflow.We believe that workflows would be easier to understand and reuse if high-level views (abstractions) of their activities were available in workflow libraries. As a first step towards obtaining these abstractions, we report in this paper on the results of a manual analysis performed over a set of real-world scientific workflows from Taverna, Wings, Galaxy and Vistrails. Our analysis has resulted in a set of scientific workflow motifs that outline (i) the kinds of data-intensive activities that are observed in workflows (Data-Operation motifs), and (ii) the different manners in which activities are implemented within workflows (Workflow-Oriented motifs). These motifs are helpful to identify the functionality of the steps in a given workflow, to develop best practices for workflow design, and to develop approaches for automated generation of workflow abstractions.

Creación, diseño y análisis de un nuevo Sistema de codificación de imagen y vídeo basado en saltos semánticos en el dominio del espacio

En este proyecto se exponen, por un lado, los fundamentos de un nuevo sistema de codificación de imagen. Este sistema, llamado Logarithmical Hoping Encoding (LHE) codifica cada píxel de la imagen utilizando saltos logarítmicos en el dominio del espacio, es decir, trabajando con los valores de luminancia y crominancia de los píxeles, sin necesidad de pasar al dominio de la frecuencia. Además, se realiza el análisis de dicho sistema y se ofrecen resultados comparativos con formatos de compresión actuales, tales como JPEG. Por otro lado, se presentan las primeras ideas para el desarrollo de un sistema que permita comprimir vídeo utilizando la tecnología LHE. Así mismo, se muestran los primeros resultados obtenidos y las conclusiones derivadas de los mismos.

Parallel workflows to personalize clinical guidelines recommendations: application to gestational diabetes mellitus

The MobiGuide system provides patients with personalized decision support tools, based on computerized clinical guidelines, in a mobile environment. The generic capabilities of the system will be demonstrated applied to the clinical domain of Gestational Diabetes (GD). This paper presents a methodology to identify personalized recommendations, obtained from the analysis of the GD guideline. We added a conceptual parallel part to the formalization of the GD guideline called "parallel workflow" that allows considering patient?s personal context and preferences. As a result of analysing the GD guideline and eliciting medical knowledge, we identified three different types of personalized advices (therapy, measurements and upcoming events) that will be implemented to perform patients? guiding at home, supported by the MobiGuide system. These results will be essential to determine the distribution of functionalities between mobile and server decision support capabilities.

Leveraging Semantics to Improve Reproducibility in Scientific Workflows

Reproducibility of published results is a cornerstone in scientific publishing and progress. Therefore, the scientific community has been encouraging authors and editors to publish their contributions in a verifiable and understandable way. Efforts such as the Reproducibility Initiative [1], or the Reproducibility Projects on Biology [2] and Psychology [3] domains, have been defining standards and patterns to assess whether an experimental result is reproducible.

Structuring research methods and data with the research object model: genomics workflows as a case study

Background: One of the main challenges for biomedical research lies in the computer-assisted integrative study of large and increasingly complex combinations of data in order to understand molecular mechanisms. The preservation of the materials and methods of such computational experiments with clear annotations is essential for understanding an experiment, and this is increasingly recognized in the bioinformatics community. Our assumption is that offering means of digital, structured aggregation and annotation of the objects of an experiment will provide necessary meta-data for a scientist to understand and recreate the results of an experiment. To support this we explored a model for the semantic description of a workflow-centric Research Object (RO), where an RO is defined as a resource that aggregates other resources, e.g., datasets, software, spreadsheets, text, etc. We applied this model to a case study where we analysed human metabolite variation by workflows. Results: We present the application of the workflow-centric RO model for our bioinformatics case study. Three workflows were produced following recently defined Best Practices for workflow design. By modelling the experiment as an RO, we were able to automatically query the experiment and answer questions such as “which particular data was input to a particular workflow to test a particular hypothesis?”, and “which particular conclusions were drawn from a particular workflow?”. Conclusions: Applying a workflow-centric RO model to aggregate and annotate the resources used in a bioinformatics experiment, allowed us to retrieve the conclusions of the experiment in the context of the driving hypothesis, the executed workflows and their input data. The RO model is an extendable reference model that can be used by other systems as well.

FragFlow: automated fragment detection in scientific workflows

Scientific workflows provide the means to define, execute and reproduce computational experiments. However, reusing existing workflows still poses challenges for workflow designers. Workflows are often too large and too specific to reuse in their entirety, so reuse is more likely to happen for fragments of workflows. These fragments may be identified manually by users as sub-workflows, or detected automatically. In this paper we present the FragFlow approach, which detects workflow fragments automatically by analyzing existing workflow corpora with graph mining algorithms. FragFlow detects the most common workflow fragments, links them to the original workflows and visualizes them. We evaluate our approach by comparing FragFlow results against user-defined sub-workflows from three different corpora of the LONI Pipeline system. Based on this evaluation, we discuss how automated workflow fragment detection could facilitate workflow reuse.

A semantic-based approach to attain reproducibility of computational environments in scientific workflows: a case study

Reproducible research in scientific workflows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and final results, improves understanding, and permits replaying a workflow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We define a process for documenting the workflow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation using a real workflow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predefined virtual machine image on both computing platforms.

Producción de recursos educativos abiertos con componentes sociales y semánticos basados con técnicas de ingeniería del conocimiento

Los Recursos Educativos Abiertos (REA) y los Cursos Educativos Abiertos (OCW) son utilizados como apoyo para los procesos de enseñanza aprendizaje; el carácter de abierto de estos recursos contribuye a la difusión de conocimiento y facilita el acceso a la información. Existe una gran cantidad de universidades e instituciones de educación superior que se han unido al movimiento abierto, poniendo a disposición los OCW que sus docentes realizan para los estudiantes formales, sin embargo se ha detectado que no existe un proceso estándar en la producción de OCW ya que cada universidad lo realiza con modelos propios de acuerdo a las normativas institucionales. Por lo cual en este trabajo de tesis doctoral se propone un modelo de producción de REA y OCW, denominado REACS que contempla el uso de un modelo de diseño instruccional que permite realizar un proceso sistemático de actividades que contribuyen al aprendizaje; además de la utilización de herramientas sociales y herramientas semánticas que aportan al trabajo colaborativo e identificación de los recursos por su significado, lo cual aporta a la inteligencia colectiva. REACS fue comparado con procesos de producción similares de las universidades relevantes del movimiento OCW, además de ser implementado en un caso de estudio con tres fases en la creación de OCW para una institución de educación superior. Con esta validación se pudo comprobar que REACS aportaba a incrementar el número de estudiantes que aprueban un curso y disminuye el tiempo de producción y publicación de un OCW. ABSTRACT Open Educational Resources (OER) and Open Course Ware (OCW) are used as support for teaching and learning processes; the open characteristic of these resources contributes to the diffusion of knowledge and facilitates the access to information. There are an important number of universities and institutions of higher education have joined to the open movement, making available the OCW´s for formal students. However, it has been found that there is not a standard process for the production of OCW and each university develop these with their own models according to their institutional regulations. Therefore, this doctoral thesis proposes REACS, a production model of OER and OCW that contemplates the use of an instructional design model that allows a systematic process of activities that contribute to learning. REACS includes the use of social tools and semantic tools that provide collaborative and identification of resources based in their meaning, contributing with the collective intelligence. REACS was compared with similar production processes belonging to relevant universities in the OCW movement. Additionally OCW were produced using REACS in a study case developed in three stages. With these validations, it was found that REACS contributed to increasing the student approved ratio and the OCW production and publication times were reduced.

Enhanced reproducibility of SADI web service workflows with Galaxy and Docke

Background: Semantic Web technologies have been widely applied in the life sciences, for example by data providers such as OpenLifeData and through web services frameworks such as SADI. The recently reported OpenLifeData2SADI project offers access to the vast OpenLifeData data store through SADI services. Findings: This article describes how to merge data retrieved from OpenLifeData2SADI with other SADI services using the Galaxy bioinformatics analysis platform, thus making this semantic data more amenable to complex analyses. This is demonstrated using a working example, which is made distributable and reproducible through a Docker image that includes SADI tools, along with the data and workflows that constitute the demonstration. Conclusions: The combination of Galaxy and Docker offers a solution for faithfully reproducing and sharing complex data retrieval and analysis workflows based on the SADI Semantic web service design patterns.

COMUNICAÇÃO EM LINHA E RUÍDOS SEMÂNTICOS NA RECUPERAÇÃO DE INFORMAÇÕES EM PESQUISAS CIENTÍFICAS

O pesquisador científico necessita de informações precisas, em tempo hábil para conclusão de seus trabalhos. Com o advento da INTERNET, o processo de comunicação em linha, homem x máquina, mediado pelos mecanismos de busca, tornou-se, simultaneamente, um auxílio e uma dificuldade no processo de recuperação de informações. O pesquisador teve que adaptar-se ao modo de operar da INTERNET e incluiu conhecimentos de diferenças idiomáticas, de terminologia, além de utilizar instrumentos que lhe forneçam parâmetros para obter maior pertinência e relevância nos dados. O uso de agentes inteligentes para melhoria de resultados e a diminuição de ruídos semânticos têm sido apontados como soluções para aumento da precisão no resultado das buscas. O estudo de casos exploratório realizado analisa a pesquisa em linha a partir da teoria da informação e propõe duas formas de otimizar o processo comunicacional com vistas à pertinência e relevância dos dados obtidos: a primeira sugere a aplicação de algoritmos que utilizem o vocabulário controlado como mediador do processo de comunicação utilizando-se dos descritores para recuperação em linha. , e a segunda ressalta a importância dos agentes inteligentes no processo de comunicação homem-máquina.(AU)

COMUNICAÇÃO EM LINHA E RUÍDOS SEMÂNTICOS NA RECUPERAÇÃO DE INFORMAÇÕES EM PESQUISAS CIENTÍFICAS

O pesquisador científico necessita de informações precisas, em tempo hábil para conclusão de seus trabalhos. Com o advento da INTERNET, o processo de comunicação em linha, homem x máquina, mediado pelos mecanismos de busca, tornou-se, simultaneamente, um auxílio e uma dificuldade no processo de recuperação de informações. O pesquisador teve que adaptar-se ao modo de operar da INTERNET e incluiu conhecimentos de diferenças idiomáticas, de terminologia, além de utilizar instrumentos que lhe forneçam parâmetros para obter maior pertinência e relevância nos dados. O uso de agentes inteligentes para melhoria de resultados e a diminuição de ruídos semânticos têm sido apontados como soluções para aumento da precisão no resultado das buscas. O estudo de casos exploratório realizado analisa a pesquisa em linha a partir da teoria da informação e propõe duas formas de otimizar o processo comunicacional com vistas à pertinência e relevância dos dados obtidos: a primeira sugere a aplicação de algoritmos que utilizem o vocabulário controlado como mediador do processo de comunicação utilizando-se dos descritores para recuperação em linha. , e a segunda ressalta a importância dos agentes inteligentes no processo de comunicação homem-máquina.(AU)

Informações de suporte ao escalonamento de workflows científicos para a execução em plataformas de computação em nuvem

A ciência tem feito uso frequente de recursos computacionais para execução de experimentos e processos científicos, que podem ser modelados como workflows que manipulam grandes volumes de dados e executam ações como seleção, análise e visualização desses dados segundo um procedimento determinado. Workflows científicos têm sido usados por cientistas de várias áreas, como astronomia e bioinformática, e tendem a ser computacionalmente intensivos e fortemente voltados à manipulação de grandes volumes de dados, o que requer o uso de plataformas de execução de alto desempenho como grades ou nuvens de computadores. Para execução dos workflows nesse tipo de plataforma é necessário o mapeamento dos recursos computacionais disponíveis para as atividades do workflow, processo conhecido como escalonamento. Plataformas de computação em nuvem têm se mostrado um alternativa viável para a execução de workflows científicos, mas o escalonamento nesse tipo de plataforma geralmente deve considerar restrições específicas como orçamento limitado ou o tipo de recurso computacional a ser utilizado na execução. Nesse contexto, informações como a duração estimada da execução ou limites de tempo e de custo (chamadas aqui de informações de suporte ao escalonamento) são importantes para garantir que o escalonamento seja eficiente e a execução ocorra de forma a atingir os resultados esperados. Este trabalho identifica as informações de suporte que podem ser adicionadas aos modelos de workflows científicos para amparar o escalonamento e a execução eficiente em plataformas de computação em nuvem. É proposta uma classificação dessas informações, e seu uso nos principais Sistemas Gerenciadores de Workflows Científicos (SGWC) é analisado. Para avaliar o impacto do uso das informações no escalonamento foram realizados experimentos utilizando modelos de workflows científicos com diferentes informações de suporte, escalonados com algoritmos que foram adaptados para considerar as informações inseridas. Nos experimentos realizados, observou-se uma redução no custo financeiro de execução do workflow em nuvem de até 59% e redução no makespan chegando a 8,6% se comparados à execução dos mesmos workflows sendo escalonados sem nenhuma informação de suporte disponível.