Extending Science Gateway Frameworks to Support Big Data Applications in the Cloud

Cloud computing offers massive scalability and elasticity required by many scien-tific and commercial applications. Combining the computational and data handling capabilities of clouds with parallel processing also has the potential to tackle Big Data problems efficiently. Science gateway frameworks and workflow systems enable application developers to implement complex applications and make these available for end-users via simple graphical user interfaces. The integration of such frameworks with Big Data processing tools on the cloud opens new oppor-tunities for application developers. This paper investigates how workflow sys-tems and science gateways can be extended with Big Data processing capabilities. A generic approach based on infrastructure aware workflows is suggested and a proof of concept is implemented based on the WS-PGRADE/gUSE science gateway framework and its integration with the Hadoop parallel data processing solution based on the MapReduce paradigm in the cloud. The provided analysis demonstrates that the methods described to integrate Big Data processing with workflows and science gateways work well in different cloud infrastructures and application scenarios, and can be used to create massively parallel applications for scientific analysis of Big Data.

From science gateways to commercial solutions

The European CloudSME project that incorporated 24 European SMEs, besides five academic partners, has finished its funded phase in March 2016. This presentation will provide a summary of the results of the project, and will analyze the challenges and differences when developing “SME Gateways”, when compared to “Science Gateways”. CloudSME started in 2013 with the aim to develop a cloud-based simulation platform for manufacturing and engineering SMEs. The project was based around industry use-cases, five of which were incorporated in the project from the start, and seven additional ones that were added as an outcome of an open call in January 2015. CloudSME utilized science gateway related technologies, such as the commercial CloudBroker Platform and the WS-PGRADE/gUSE Gateway Framework that were developed in the preceding SCI-BUS project. As most important outcome, the project successfully implemented 12 industry quality demonstrators that showcase how SMEs in the manufacturing and engineering sector can utilize cloud-based simulation services. Some of these solutions are already market-ready and currently being rolled out by the software vendor companies. Some others require further fine-tuning and the implementation of commercial interfaces before being put into the market. The CloudSME use-cases came from a very wide application spectrum. The project implemented, for example, an open marketplace for micro-breweries to optimize their production and distribution processes, an insole design validation service to be used by podiatrists and shoe manufacturers, a generic stock management solution for manufacturing SMEs, and also several “classical” high-performance computing case-studies, such as fluid dynamics simulations for model helicopter design, and dual-fuel internal combustion engine simulation. As the project generated significant impact and interest in the manufacturing sector, 10 CloudSME stakeholders established a follow-up company called CloudSME UG for the future commercialization of the results. Besides the success stories, this talk would also like to highlight the difficulties when transferring the outcomes of an academic research project to real commercial applications. The different mindset and approach of academic and industry partners presented a real challenge for the CloudSME project, with some interesting and valuable lessons learnt. The academic way of supporting SMEs did not always work well with the rather different working practices and culture of many participants. Also, the quality of support regarding operational solutions required by the SMEs is well beyond the typical support services academic institutions are prepared for. Finally, a clear lack of trust in academic solutions when compared to commercial solutions was also imminent. The talk will highlight some of these challenges underpinned by the implementation of the CloudSME use-cases.

Special Section Guest Editorial: Image/Video Quality and System Performance

Rapid developments in display technologies, digital printing, imaging sensors, image processing and image transmission are providing new possibilities for creating and conveying visual content. In an age in which images and video are ubiquitous and where mobile, satellite, and three-dimensional (3-D) imaging have become ordinary experiences, quantification of the performance of modern imaging systems requires appropriate approaches. At the end of the imaging chain, a human observer must decide whether images and video are of a satisfactory visual quality. Hence the measurement and modeling of perceived image quality is of crucial importance, not only in visual arts and commercial applications but also in scientific and entertainment environments. Advances in our understanding of the human visual system offer new possibilities for creating visually superior imaging systems and promise more accurate modeling of image quality. As a result, there is a profusion of new research on imaging performance and perceived quality.

Commercial Use of WS-PGRADE/gUSE

Although originally an academic and research product, the WS-PGRADE/gUSE framework is increasingly applied by commercial institutions too. Within the SCI-BUS project, several commercial gateways have been developed by various companies. WS-PGRADE/gUSE is also intensively used within another European research project, CloudSME (Cloud-based Simulation Platform for Manufacturing and Engineering). This chapter provides an overview and de-scribes in detail some commercial WS-PGRADE/gUSE based gateway implemen-tations. Two representative case studies from the SCI-BUS project, the Build and Test portal and the eDOX Archiver Gateway are introduced. An overview of WS-PGRADE/gUSE based gateways for running simulation applications in the cloud within the CloudSME project is also provided.