Real user evaluation of spoken dialogue systems using Amazon Mechanical Turk

This paper describes a framework for evaluation of spoken dialogue systems. Typically, evaluation of dialogue systems is performed in a controlled test environment with carefully selected and instructed users. However, this approach is very demanding. An alternative is to recruit a large group of users who evaluate the dialogue systems in a remote setting under virtually no supervision. Crowdsourcing technology, for example Amazon Mechanical Turk (AMT), provides an efficient way of recruiting subjects. This paper describes an evaluation framework for spoken dialogue systems using AMT users and compares the obtained results with a recent trial in which the systems were tested by locally recruited users. The results suggest that the use of crowdsourcing technology is feasible and it can provide reliable results. Copyright © 2011 ISCA.

A framework for clarifying the meaning of Triple Bottom-Line, Integrated, and Sustainability Assessment

Terms such as Integrated Assessment and Sustainability Assessment are used to label 'new' approaches to impact assessment that are designed to direct planning and decision-making towards sustainable development (SD). Established assessment techniques, such as EIA and SEA, are also widely promoted as SD 'tools'. This paper presents the findings of a literature review undertaken to identify the features that are typically promoted for improving the SD-directedness of assessments. A framework is developed which reconciles the broad range of emerging approaches and tackles the inconsistent use of terminology. The framework comprises a three-dimensional space defined by the following axes: the comprehensiveness of the SD coverage; the degree of 'integration' of the techniques and themes; and the extent to which a strategic perspective is adopted. By applying the framework, assessment approaches can be positioned relative to one another, enabling comparison on the basis of substance rather than semantics. © 2007 Elsevier Inc. All rights reserved.

An approach to scenario analysis of the sustainability of an industrial sector applied to clothing and textiles in the UK

Companies aiming to be 'sustainability leaders' in their sector and governments wanting to support their ambitions need a means to assess the changes required to make a significant difference in the impact of their whole sector. Previous work on scenario analysis/scenario planning demonstrates extensive developments and applications, but as yet few attempts to integrate the 'triple bottom line' concerns of sustainability into scenario planning exercises. This paper, therefore, presents a methodology for scenario analysis of large change to an entire sector. The approach includes calculation of a 'triple bottom line graphic equaliser' to allow exploration and evaluation of the trade-offs between economic, environmental and social impacts. The methodology is applied to the UK's clothing and textiles sector, and results from the study of the sector are summarised. In reflecting on the specific study, some suggestions are made about future application of a similar methodology, including a template of candidate solutions that may lead to significant reduction in impacts. © 2007 Elsevier Ltd. All rights reserved.

Towards a process framework for assessing the potential value of technologies

Current technology valuation literature predominantly focuses on explaining the merits and implications of specific tools, but little research is available that takes a contextual process perspective. The aim of this paper is to further develop an integrative process framework that supports the structuring of the valuation process and the more systematic choice of valuation techniques for new technologies. The paper starts by reviewing key concepts and issues that surround the assessment of technology investments and the evidence of what companies use. Many factors need to be brought into the appraisal process, reflecting technological and market conditions. While there is usually a desire to reduce the assessment to a financial value, it is also widely appreciated that there is long term strategic value in securing a technological lead, which is difficult, or even inappropriate, to assess in purely financial terms. The multiple factors involved in the evaluation activity are identified with respect to the changing nature of the appraisal process as the technology matures and the implications for associated tools. The result of the literature review is a process framework which provides a conceptual basis for integrating valuation techniques. This framework is then populated with the results of industrial case studies on technology valuation to allow conclusions on its applicability to be drawn. © 2011 IEEE.

Towards the development of an integrated design framework using the Blackboard model

Current research into the process of engineering design is extending the use of computers towards the acquisition, representation and application of design process knowledge in addition to the existing storage and manipulation of product-based models of design objects. This is a difficult task because the design of mechanical systems is a complex, often unpredictable process involving ill-structured problem solving skills and large amounts of knowledge, some which may be of an incomplete and subjective nature. Design problems require the integration of a variety of modes of working such as numerical, graphical, algorithmic or heuristic and demand products through synthesis, analysis and evaluation activities.

This report presents the results of a feasibility study into the blackboard approach and discusses the development of an initial prototype system that will enable an alphanumeric design dialogue between a designer and an expert to be analysed in a formal way, thus providing real-life protocol data on which to base the blackboard message structures.

A conceptual framework for future-proofing the energy performance of buildings

This paper presents a review undertaken to understand the concept of 'future-proofing' the energy performance of buildings. The long lifecycles of the building stock, the impacts of climate change and the requirements for low carbon development underline the need for long-term thinking from the early design stages. 'Future-proofing' is an emerging research agenda with currently no widely accepted definition amongst scholars and building professionals. In this paper, it refers to design processes that accommodate explicitly full lifecycle perspectives and energy trends and drivers by at least 2050, when selecting energy efficient measures and low carbon technologies. A knowledge map is introduced, which explores the key axes (or attributes) for achieving a 'future-proofed' energy design; namely, coverage of sustainability issues, lifecycle thinking, and accommodating risks and uncertainties that affect the energy consumption. It is concluded that further research is needed so that established building energy assessment methods are refined to better incorporate future-proofing. The study follows an interdisciplinary approach and is targeted at design teams with aspirations to achieve resilient and flexible low-energy buildings over the long-term. © 2012 Elsevier Ltd.

Tomorrow's cities - a framework for building resilience

Urbanisation is the great driving force of the twenty-first century. Cities are associated with both productivity and creativity, and the benefits offered by closely connected and high density living and working contribute to sustainability. At the same time, cities need extensive infrastructure – like water, power, sanitation and transportation systems – to operate effectively. Cities therefore comprise multiple components, forming both static and dynamic systems that are interconnected directly and indirectly on a number of levels, all forming the backdrop for the interaction of people and processes. Bringing together large numbers of people and complex products in rich interactions can lead to vulnerability from hazards, threats and even trends, whether natural hazards, epidemics, political upheaval, demographic changes, economic instability and/or mechanical failures; The key to countering vulnerability is the identification of critical systems and clear understanding of their interactions and dependencies. Critical systems can be assessed methodically to determine the implications of their failure and their interconnectivities with other systems to identify options. The overriding need is to support resilience – defined here as the degree to which a system or systems can continue to function effectively in a changing environment. Cities need to recognise the significance of devising adaptation strategies and processes to address a multitude of uncertainties relating to climate, economy, growth and demography. In this paper we put forward a framework to support cities in understanding the hazards, threats and trends that can make them vulnerable to unexpected changes and unpredictable shocks. The framework draws on an asset model of the city, in which components that contribute to resilience include social capital, economic assets, manufactured assets, and governance. The paper reviews the field, and draws together an overarching framework intended to help cities plan a robust trajectory towards increased resilience through flexibility, resourcefulness and responsiveness. It presents some brief case studies demonstrating the applicability of the proposed framework to a wide variety of circumstances.

Preliminary evaluation of a nuclear scenario involving innovative gas cooled reactors

In order to guarantee a sustainable supply of future energy demand without compromising the environment, some actions for a substantial reduction of CO 2 emissions are nowadays deeply analysed. One of them is the improvement of the nuclear energy use. In this framework, innovative gas-cooled reactors (both thermal and fast) seem to be very attractive from the electricity production point of view and for the potential industrial use along the high temperature processes (e.g., H 2 production by steam reforming or I-S process). This work focuses on a preliminary (and conservative) evaluation of possible advantages that a symbiotic cycle (EPR-PBMR-GCFR) could entail, with special regard to the reduction of the HLW inventory and the optimization of the exploitation of the fuel resources. The comparison between the symbiotic cycle chosen and the reference one (once-through scenario, i.e., EPR-SNF directly disposed) shows a reduction of the time needed to reach a fixed reference level from ∼170000 years to ∼1550 years (comparable with typical human times and for this reason more acceptable by the public opinion). In addition, this cycle enables to have a more efficient use of resources involved: the total electric energy produced becomes equal to ∼630 TWh/year (instead of only ∼530 TWh/year using only EPR) without consuming additional raw materials. © 2009 Barbara Vezzoni et al.

Tomorrow's cities: a framework to assess urban resilience

Urbanisation is one of the great driving forces of the twenty-first century. Cities generate both productivity and creativity, and the benefits offered by high-density living and working contribute to sustainability. Cities comprise multiple components, forming both static and dynamic systems that are interconnected directly and indirectly on a number of levels. Bringing together large numbers of people within a complex system can lead to vulnerability from a wide range of hazards, threats and trends. The key to reducing this vulnerability is the identification of critical systems and determination of the implications of their failure and their interconnectivities with other systems. One emerging approach to these challenges focuses on building resilience – defined here as the degree to which a system can continue to function effectively in a changing environment. This paper puts forward a framework designed to help engineers, planners and designers to support cities in understanding the hazards, threats and trends that can make them vulnerable, and identify focus areas for building resilience into the systems, which allow it to function and prosper. Four case studies of cities whose resilience was tested by recent extreme weather events are presented, seeking to demonstrate the application of the proposed framework.

Sustainability: A new imperative in contaminated land remediation

Land is not only a critical component of the earth's life support system, but also a precious resource and an important factor of production in economic systems. However, historical industrial operations have resulted in large areas of contaminated land that are only slowly being remediated. In recent years, sustainability has drawn increasing attention in the environmental remediation field. In Europe, there has been a movement towards sustainable land management; and in the US, there is an urge for green remediation. Based on a questionnaire survey and a review of existing theories and empirical evidence, this paper suggests the expanding emphasis on sustainable remediation is driven by three general factors: (1) increased recognition of secondary environmental impacts (e.g., life-cycle greenhouse gas emissions, air pollution, energy consumption, and waste production) from remediation operations, (2) stakeholders' demand for economically sustainable brownfield remediation and "green" practices, and (3) institutional pressures (e.g., social norm and public policy) that promote sustainable practices (e.g., renewable energy, green building, and waste recycling). This paper further argues that the rise of the "sustainable remediation" concept represents a critical intervention point from where the remediation field will be reshaped and new norms and standards will be established for practitioners to follow in future years. This paper presents a holistic view of sustainability considerations in remediation, and an integrated framework for sustainability assessment and decision making. The paper concludes that "sustainability" is becoming a new imperative in the environmental remediation field, with important implications for regulators, liability owners, consultants, contractors, and technology vendors. © 2014 Elsevier Ltd.