Crowdsourcing interactions : using crowdsourcing for evaluating interactive information retrieval systems

In the field of information retrieval (IR), researchers and practitioners are often faced with a demand for valid approaches to evaluate the performance of retrieval systems. The Cranfield experiment paradigm has been dominant for the in-vitro evaluation of IR systems. Alternative to this paradigm, laboratory-based user studies have been widely used to evaluate interactive information retrieval (IIR) systems, and at the same time investigate users’ information searching behaviours. Major drawbacks of laboratory-based user studies for evaluating IIR systems include the high monetary and temporal costs involved in setting up and running those experiments, the lack of heterogeneity amongst the user population and the limited scale of the experiments, which usually involve a relatively restricted set of users. In this paper, we propose an alternative experimental methodology to laboratory-based user studies. Our novel experimental methodology uses a crowdsourcing platform as a means of engaging study participants. Through crowdsourcing, our experimental methodology can capture user interactions and searching behaviours at a lower cost, with more data, and within a shorter period than traditional laboratory-based user studies, and therefore can be used to assess the performances of IIR systems. In this article, we show the characteristic differences of our approach with respect to traditional IIR experimental and evaluation procedures. We also perform a use case study comparing crowdsourcing-based evaluation with laboratory-based evaluation of IIR systems, which can serve as a tutorial for setting up crowdsourcing-based IIR evaluations.

Geometric sensitivity of patient-specific finite element models of the spine to variability in user-selected anatomical landmarks

Software to create individualised finite element (FE) models of the osseoligamentous spine using pre-operative computed tomography (CT) data-sets for spinal surgery patients has recently been developed. This study presents a geometric sensitivity analysis of this software to assess the effect of intra-observer variability in user-selected anatomical landmarks. User-selected landmarks on the osseous anatomy were defined from CT data-sets for three scoliosis patients and these landmarks were used to reconstruct patient-specific anatomy of the spine and ribcage using parametric descriptions. The intra-observer errors in landmark co-ordinates for these anatomical landmarks were calculated. FE models of the spine and ribcage were created using the reconstructed anatomy for each patient and these models were analysed for a loadcase simulating clinical flexibility assessment. The intra-observer error in the anatomical measurements was low in comparison to the initial dimensions, with the exception of the angular measurements for disc wedge and zygapophyseal joint (z-joint) orientation and disc height. This variability suggested that CT resolution may influence such angular measurements, particularly for small anatomical features, such as the z-joints, and may also affect disc height. The results of the FE analysis showed low variation in the model predictions for spinal curvature with the mean intra-observer variability substantially less than the accepted error in clinical measurement. These findings demonstrate that intra-observer variability in landmark point selection has minimal effect on the subsequent FE predictions for a clinical loadcase.

Reframe : QUT's evaluation framework survey business rules

Reframe is changing our approach to the evaluation of courses, units, teaching and student experience at QUT. We are moving away from a single survey tool to a richer, more holistic and customisable approach. This approach will help our academics design and deliver high-quality learning experiences, and review the impact of their teaching practice on student learning. Through it, we will also be able to provide more timely access to specialised support and meet external reporting requirements.

Protocols : Reframe : QUT's evaluation framework

Reframe is changing our approach to the evaluation of courses, units, teaching and student experience at QUT. We are moving away from a single survey tool to a richer, more holistic and customisable approach. These protocols allows academic staff and administrators access to the ways in which the policy is enacted through process.

From impact assessment to user-friendly risk management decision-making

In earlier cultures and societies, hazards and risks to human health were dealt with by methods derived from myth, metaphor and ritual. In modem society however, notions of hazard and risk have been transformed from the level of a folk discourse to that of an expert centred concept (Plough & Krimsky, 1987). With the professionalization of risk and hazard analysis came a preferred framework for decision making based on a range of 'technical' methodologies (Giere, 1991 ). This is especially true for decision processes relating to risk assessment and management, and impact assessment. Such approaches however, often entail narrow technical-based theoretical assumptions about human behaviour and the natural world, and the· methods used. They therefore carry 'in-built' error factors that contribute considerable uncertainty to the results.

Semantic querying of business process models

Determining similarity between business process models has recently gained interest in the business process management community. So far similarity was addressed separately either at semantic or structural aspect of process models. Also, most of the contributions that measure similarity of process models assume an ideal case when process models are enriched with semantics - a description of meaning of process model elements. However, in real life this results in a heavy human effort consuming pre-processing phase which is often not feasible. In this paper we propose an automated approach for querying a business process model repository for structurally and semantically relevant models. Similar to the search on the Internet, a user formulates a BPMN-Q query and as a result receives a list of process models ordered by relevance to the query. We provide a business process model search engine implementation for evaluation of the proposed approach.

Causal behavioural profiles : efficient computation, applications, and evaluation

Analysis of behavioural consistency is an important aspect of software engineering. In process and service management, consistency verification of behavioural models has manifold applications. For instance, a business process model used as system specification and a corresponding workflow model used as implementation have to be consistent. Another example would be the analysis to what degree a process log of executed business operations is consistent with the corresponding normative process model. Typically, existing notions of behaviour equivalence, such as bisimulation and trace equivalence, are applied as consistency notions. Still, these notions are exponential in computation and yield a Boolean result. In many cases, however, a quantification of behavioural deviation is needed along with concepts to isolate the source of deviation. In this article, we propose causal behavioural profiles as the basis for a consistency notion. These profiles capture essential behavioural information, such as order, exclusiveness, and causality between pairs of activities of a process model. Consistency based on these profiles is weaker than trace equivalence, but can be computed efficiently for a broad class of models. In this article, we introduce techniques for the computation of causal behavioural profiles using structural decomposition techniques for sound free-choice workflow systems if unstructured net fragments are acyclic or can be traced back to S- or T-nets. We also elaborate on the findings of applying our technique to three industry model collections.

Histories of user-generated content : between formal and informal media economies

Debates about user-generated content (UGC) often depend on a contrast with its normative opposite, the professionally produced content that is supported and sustained by commercial media businesses or public organisations. UGC is seen to appear within or in opposition to professional media, often as a disruptive, creative, change-making force. Our suggestion is to position UGC not in opposition to professional or "producer media", or in hybridised forms of subjective combination with it (the so-called "pro-sumer" or "pro-am" system), but in relation to different criteria, namely the formal and informal elements in media industries. In this article, we set out a framework for the comparative and historical analysis of UGC systems and their relations with other formal and informal media activity, illustrated with examples ranging from games to talkback radio. We also consider the policy implications that emerge from a historicised reading of UGC as a recurring dynamic within media industries, rather than a manifestation of consumer agency specific to digital cultures.

Evaluation of a pilot-scale oil extraction from microalgae for biodiesel production

Biodiesel derived from microalgae is one of a suite of potential solutions to meet the increasing demand for a renewable, carbon-neutral energy source. However, there are numerous challenges that must be addressed before algae biodiesel can become commercially viable. These challenges include the economic feasibility of harvesting and dewatering the biomass and the extraction of lipids and their conversion into biodiesel. Therefore, it is essential to find a suitable extraction process given these processes presently contribute significantly to the total production costs which, at this stage, inhibit the ability of biodiesel to compete financially with petroleum diesel. This study focuses on pilot-scale (100 kg dried microalgae) solvent extraction of lipids from microalgae and subsequent transesterification to biodiesel. Three different solvents (hexane, isopropanol (IPA) and hexane + IPA (1:1)) were used with two different extraction methods (static and Soxhlet) at bench-scale to find the most suitable solvent extraction process for the pilot-scale. The Soxhlet method extracted only 4.2% more lipid compared to the static method. However, the fatty acid profiles of different extraction methods with different solvents are similar, suggesting that none of the solvents or extraction processes were biased for extraction of particular fatty acids. Considering the cost and availability of the solvents, hexane was chosen for pilot-scale extraction using static extraction. At pilot-scale the lipid yield was found to be 20.3% of total biomass which is 2.5% less than from bench scale. Extracted fatty acids were dominated by polyunsaturated fatty acids (PUFAs) (68.94±0.17%) including 47.7±0.43 and 17.86±0.42% being docosahexaenoic acid (DHA) (C22:6) and docosapentaenoic acid (DPA) (C22:5, ω-3), respectively. These high amounts of long chain poly unsaturated fatty acids are unique to some marine microalgae and protists and vary with environmental conditions, culture age and nutrient status, as well as with cultivation process. Calculated physical and chemical properties of density, viscosity of transesterified fatty acid methyl esters (FAMEs) were within the limits of the biodiesel standard specifications as per ASTM D6751-2012 and EN 14214. The calculated cetane number was, however, significantly lower (17.8~18.6) compared to ASTM D6751-2012 or EN 14214-specified minimal requirements. We conclude that the obtained microalgal biodiesel would likely only be suitable for blending with petroleum diesel to a maximum of 5 to 20%.

Evaluation of BMP-2 gene-activated muscle grafts for cranial defect repair

Large, osseous, segmental defects heal poorly. Muscle has a propensity to form bone when exposed to an osteogenic stimulus such as that provided by transfer and expression of cDNA encoding bone morphogenetic protein-2 (BMP-2). The present study evaluated the ability of genetically modified, autologous muscle to heal large cranial defects in rats. Autologous grafts (8 mm � 2 mm) were punched from the biceps femoris muscle and transduced intraoperatively with recombinant adenovirus vector containing human BMP-2 or green fluorescent protein cDNA. While the muscle biopsies were incubating with the vector, a central parietal 8 mm defect was surgically created in the calvarium of the same animal. The gene-activated muscle graft was then implanted into the cranial defect. After 8 weeks, crania were examined radiographically, histologically, and by micro-computed tomography and dual energy X-ray absorptiometry. Although none of the defects were completely healed in this time, muscle grafts expressing BMP-2 deposited more than twice as much new bone as controls. Histology confirmed the anatomical integrity of the newly formed bone, which was comparable in thickness and mineral density to the original cranial bone. This study confirms the in vivo osteogenic properties of genetically modified muscle and suggests novel strategies for healing bone. � 2011 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 30:1095–1102, 2012

Ability of recombinant human bone morphogenetic protein 2 to enhance bone healing in the presence of tobramycin : evaluation in a rat segmental defect model

Objective To determine whether locally applied tobramycin influences the ability of recombinant human bone morphogenetic protein 2 (rhBMP-2) to heal a segmental defect in the rat femur. Methods The influence of tobramycin on the osteogenic differentiation of mesenchymal stem cells was first evaluated in vitro. For the subsequent, in vivo experiments, a 5-mm segmental defect was created in the right femur of each of 25 Sprague-Dawley rats and stabilized with an external fixator and four Kirschner wires. Rats were divided in four groups: empty control, tobramycin (11 mg)/absorbable collagen sponge, rhBMP-2 (11 μg)/absorbable collagen sponge, and rhBMP-2/absorbable collagen sponge with tobramycin. Bone healing was monitored by radiography at 3 and 8 weeks. Animals were euthanized at 8 weeks and the properties of the defect were compared with the intact contralateral femur. Bone formation in the defect region was assessed by dual-energy x-ray absorptiometry, microcomputed tomography, histology, and mechanical testing. Results Tobramycin exerted a dose-dependent inhibition of alkaline phosphatase induction and calcium deposition by mesenchymal stem cells cultured under osteogenic conditions. The inhibition was reversed in the presence of 500 ng/mL of rhBMP-2. Segmental defects in the rat femora failed to heal in the absence of rhBMP-2. Tobramycin exerted no inhibitory effects on the ability of rhBMP-2 to heal these defects and increased the bone area of the defects treated with rhBMP-2. Data obtained from all other parameters of healing, including dual-energy x-ray absorptiometry, microcomputed tomography, histology, and mechanical testing, were unaffected by tobramycin. Conclusions Although our in vitro results suggested that tobramycin inhibits the osteogenic differentiation of mesenchymal stem cells, this could be overcome by rhBMP-2. Tobramycin did not impair the ability of rhBMP-2 to heal critical-sized femoral defects in rats. Indeed, bone area was increased by nearly 20% in the rhBMP-2 group treated with tobramycin. This study shows that locally applied tobramycin can be used in conjunction with rhBMP-2 to enhance bone formation at fracture sites.

Evaluating the use of a very large-scale presentation and collaboration framework

In this paper we describe the use and evaluation of CubIT, a multi-user, very large-scale presentation and collaboration framework. CubIT is installed at the Queensland University of Technology’s (QUT) Cube facility. The “Cube” is an interactive visualisation facility made up of five very large-scale interactive multi-panel wall displays, each consisting of up to twelve 55-inch multi-touch screens (48 screens in total) and massive projected display screens situated above the display panels. The paper outlines the unique design challenges, features, use and evaluation of CubIT. The system was built to make the Cube facility accessible to QUT’s academic and student population. CubIT enables users to easily upload and share their own media content, and allows multiple users to simultaneously interact with the Cube’s wall displays. The features of CubIT are implemented via three user interfaces, a multi-touch interface working on the wall displays, a mobile phone and tablet application and a web-based content management system. The evaluation reveals issues around the public use and functional scope of the system.

Evaluation of robust autonomy and implications on UAS certification and design

As the number of potential applications of Unmanned Aircraft Systems (UAS) grows in civilian operations and national security, National Airworthiness Authorities are under increasing pressure to provide a path for certification and allow UAS integration into the national airspace. The success of this integration depends on developments in improved UAS reliability and safety, regulations for certification, and technologies for operational performance and safety assessment. This paper focusses on the latter and describes the use of a framework for evaluating robust autonomy of UAS, namely, the autonomous system’s ability to either continue operation in the presence of faults or safely shut down. The paper draws parallels between the proposed evaluation framework and the evaluation of pilots during the licensing process. It also discusses how the data from the proposed evaluation can be uses as an aid for decision making in certification and UAS designs.

Tools for Mobile Multimedia Programming and Development

Mobile devices are rapidly developing into the primary technology for users to work, socialize, and play in a variety of settings and contexts. Their pervasiveness has provided researchers with the means to investigate innovative solutions to ever more complex user demands. Tools for Mobile Multimedia Programming and Development investigates the use of mobile platforms for research projects, focusing on the development, testing, and evaluation of prototypes rather than final products, which enables researchers to better understand the needs of users through image processing, object recognition, sensor integration, and user interactions. This book benefits researchers and professionals in multiple disciplines who utilize such techniques in the creation of prototypes for mobile devices and applications. This book is part of the Advances in Wireless Technologies and Telecommunication series collection.

On evaluation of robust autonomy for uninhabited vehicles and systems

This paper discusses a method to quantify robust autonomy of Uninhabited Vehicles and Systems (UVS) in aerospace, marine, or land applications. Based on mission-vehicle specific performance criteria, we define an system utility function that can be evaluated using simulation scenarios for an envelope of environmental conditions. The results of these evaluations are used to compute a figure of merit or measure for operational efectiveness (MOE). The procedure is then augmented to consider faults and the performance of mechanisms to handle these faulty operational modes. This leads to a measure of robust autonomy (MRA). The objective of the proposed figures of merit is to assist in decision making about vehicle performance and reliability at both vehicle development stage (using simulation models) and at certification stage (using hardware-in-the-loop testing). Performance indices based on dynamic and geometric tasks associated with vehicle manoeuvring problems are proposed, and an example of a two- dimensional y scenario is provided to illustrate the use of the proposed figures of merit.