Systematic evaluation of GoSoapBox in tertiary education : a student response system for improving learning experiences and outcomes

There is currently a wide range of research into the recent introduction of student response systems in higher education and tertiary settings (Banks 2006; Kay and Le Sange, 2009; Beatty and Gerace 2009; Lantz 2010; Sprague and Dahl 2009). However, most of this pedagogical literature has generated ‘how to’ approaches regarding the use of ‘clickers’, keypads, and similar response technologies. There are currently no systematic reviews on the effectiveness of ‘GoSoapBox’ – a more recent, and increasingly popular student response system – for its capacity to enhance critical thinking, and achieve sustained learning outcomes. With rapid developments in teaching and learning technologies across all undergraduate disciplines, there is a need to obtain comprehensive, evidence-based advice on these types of technologies, their uses, and overall efficacy. This paper addresses this current gap in knowledge. Our teaching team, in an undergraduate Sociology and Public Health unit at the Queensland University of Technology (QUT), introduced GoSoapBox as a mechanism for discussing controversial topics, such as sexuality, gender, economics, religion, and politics during lectures, and to take opinion polls on social and cultural issues affecting human health. We also used this new teaching technology to allow students to interact with each other during class – both on both social and academic topics – and to generate discussions and debates during lectures. The paper reports on a data-driven study into how this interactive online tool worked to improve engagement and the quality of academic work produced by students. This paper will firstly, cover the recent literature reviewing student response systems in tertiary settings. Secondly, it will outline the theoretical framework used to generate this pedagogical research. In keeping with the social and collaborative features of Web 2.0 technologies, Bandura’s Social Learning Theory (SLT) will be applied here to investigate the effectiveness of GoSoapBox as an online tool for improving learning experiences and the quality of academic output by students. Bandura has emphasised the Internet as a tool for ‘self-controlled learning’ (Bandura 2001), as it provides the education sector with an opportunity to reconceptualise the relationship between learning and thinking (Glassman & Kang 2011). Thirdly, we describe the methods used to implement the use of GoSoapBox in our lectures and tutorials, and which aspects of the technology we drew on for learning purposes, as well as the methods for obtaining feedback from the students about the effectiveness or otherwise of this tool. Fourthly, we report cover findings from an examination of all student/staff activity on GoSoapBox as well as reports from students about the benefits and limitations of it as a learning aid. We then display a theoretical model that is produced via an iterative analytical process between SLT and our data analysis for use by academics and teachers across the undergraduate curriculum. The model has implications for all teachers considering the use of student response systems to improve the learning experiences of their students. Finally, we consider some of the negative aspects of GoSoapBox as a learning aid.

Datasets for the evaluation of multi-sensor perception in natural environments with challenging conditions

This paper presents large, accurately calibrated and time-synchronised datasets, gathered outdoors in controlled environmental conditions, using an unmanned ground vehicle (UGV), equipped with a wide variety of sensors. It discusses how the data collection process was designed, the conditions in which these datasets have been gathered, and some possible outcomes of their exploitation, in particular for the evaluation of performance of sensors and perception algorithms for UGVs.

Critical reflections on the use of participatory methodologies to build evaluation capacities in international development organisations

There has been a greater focus on strengthening evaluation capacity building (ECB) within development organisations in recent years. This can be attributed in part to the growing appreciation of the value of participatory and collaborative forms of evaluation. Evaluation is increasingly seen as an ongoing learning process and an important means of strengthening capacity and improving organisational performance (Horton et al., 2003:7). While there are many benefits of using participatory methodologies in ECB projects, our experiences and a review of the literature in this area highlight the many challenges, issues and contradictions that can affect the success of such ECB efforts. We discuss these issues, drawing on our learnings from the ongoing participatory action research (PAR) project 'Assessing Communication for Social Change’ (AC4SC). This four year project, which began in 2007, is a collaboration between communication and development academics and evaluation specialists from two Australian universities and communication for development practitioners and monitoring and evaluation (M&E) staff in the NGO Equal Access Nepal (EAN). The aim is to develop, implement, and evaluate a participatory methodology for assessing the social change impacts of community radio programs produced by EAN. It builds on previous projects that used ethnographic action research (EAR) methodology (Tacchi et al., 2007).

Protocols : Reframe : QUT's evaluation framework

Reframe is changing our approach to the evaluation of courses, units, teaching and student experience at QUT. We are moving away from a single survey tool to a richer, more holistic and customisable approach. These protocols allows academic staff and administrators access to the ways in which the policy is enacted through process.

Generating natural language texts from business process models

Process Modeling is a widely used concept for understanding, documenting and also redesigning the operations of organizations. The validation and usage of process models is however affected by the fact that only business analysts fully understand them in detail. This is in particular a problem because they are typically not domain experts. In this paper, we investigate in how far the concept of verbalization can be adapted from object-role modeling to process models. To this end, we define an approach which automatically transforms BPMN process models into natural language texts and combines different techniques from linguistics and graph decomposition in a flexible and accurate manner. The evaluation of the technique is based on a prototypical implementation and involves a test set of 53 BPMN process models showing that natural language texts can be generated in a reliable fashion.

Structuring acyclic process models

This article studies the problem of transforming a process model with an arbitrary topology into an equivalent well-structured process model. While this problem has received significant attention, there is still no full characterization of the class of unstructured process models that can be transformed into well-structured ones, nor an automated method for structuring any process model that belongs to this class. This article fills this gap in the context of acyclic process models. The article defines a necessary and sufficient condition for an unstructured acyclic process model to have an equivalent well-structured process model under fully concurrent bisimulation, as well as a complete structuring method. The method has been implemented as a tool that takes process models captured in the BPMN and EPC notations as input. The article also reports on an empirical evaluation of the structuring method using a repository of process models from commercial practice.

Semantic querying of business process models

Determining similarity between business process models has recently gained interest in the business process management community. So far similarity was addressed separately either at semantic or structural aspect of process models. Also, most of the contributions that measure similarity of process models assume an ideal case when process models are enriched with semantics - a description of meaning of process model elements. However, in real life this results in a heavy human effort consuming pre-processing phase which is often not feasible. In this paper we propose an automated approach for querying a business process model repository for structurally and semantically relevant models. Similar to the search on the Internet, a user formulates a BPMN-Q query and as a result receives a list of process models ordered by relevance to the query. We provide a business process model search engine implementation for evaluation of the proposed approach.

On application of structural decomposition for process model abstraction

Real world business process models may consist of hundreds of elements and have sophisticated structure. Although there are tasks where such models are valuable and appreciated, in general complexity has a negative influence on model comprehension and analysis. Thus, means for managing the complexity of process models are needed. One approach is abstraction of business process models-creation of a process model which preserves the main features of the initial elaborate process model, but leaves out insignificant details. In this paper we study the structural aspects of process model abstraction and introduce an abstraction approach based on process structure trees (PST). The developed approach assures that the abstracted process model preserves the ordering constraints of the initial model. It surpasses pattern-based process model abstraction approaches, allowing to handle graph-structured process models of arbitrary structure. We also provide an evaluation of the proposed approach.

Causal behavioural profiles : efficient computation, applications, and evaluation

Analysis of behavioural consistency is an important aspect of software engineering. In process and service management, consistency verification of behavioural models has manifold applications. For instance, a business process model used as system specification and a corresponding workflow model used as implementation have to be consistent. Another example would be the analysis to what degree a process log of executed business operations is consistent with the corresponding normative process model. Typically, existing notions of behaviour equivalence, such as bisimulation and trace equivalence, are applied as consistency notions. Still, these notions are exponential in computation and yield a Boolean result. In many cases, however, a quantification of behavioural deviation is needed along with concepts to isolate the source of deviation. In this article, we propose causal behavioural profiles as the basis for a consistency notion. These profiles capture essential behavioural information, such as order, exclusiveness, and causality between pairs of activities of a process model. Consistency based on these profiles is weaker than trace equivalence, but can be computed efficiently for a broad class of models. In this article, we introduce techniques for the computation of causal behavioural profiles using structural decomposition techniques for sound free-choice workflow systems if unstructured net fragments are acyclic or can be traced back to S- or T-nets. We also elaborate on the findings of applying our technique to three industry model collections.

Evaluation of a pilot-scale oil extraction from microalgae for biodiesel production

Biodiesel derived from microalgae is one of a suite of potential solutions to meet the increasing demand for a renewable, carbon-neutral energy source. However, there are numerous challenges that must be addressed before algae biodiesel can become commercially viable. These challenges include the economic feasibility of harvesting and dewatering the biomass and the extraction of lipids and their conversion into biodiesel. Therefore, it is essential to find a suitable extraction process given these processes presently contribute significantly to the total production costs which, at this stage, inhibit the ability of biodiesel to compete financially with petroleum diesel. This study focuses on pilot-scale (100 kg dried microalgae) solvent extraction of lipids from microalgae and subsequent transesterification to biodiesel. Three different solvents (hexane, isopropanol (IPA) and hexane + IPA (1:1)) were used with two different extraction methods (static and Soxhlet) at bench-scale to find the most suitable solvent extraction process for the pilot-scale. The Soxhlet method extracted only 4.2% more lipid compared to the static method. However, the fatty acid profiles of different extraction methods with different solvents are similar, suggesting that none of the solvents or extraction processes were biased for extraction of particular fatty acids. Considering the cost and availability of the solvents, hexane was chosen for pilot-scale extraction using static extraction. At pilot-scale the lipid yield was found to be 20.3% of total biomass which is 2.5% less than from bench scale. Extracted fatty acids were dominated by polyunsaturated fatty acids (PUFAs) (68.94±0.17%) including 47.7±0.43 and 17.86±0.42% being docosahexaenoic acid (DHA) (C22:6) and docosapentaenoic acid (DPA) (C22:5, ω-3), respectively. These high amounts of long chain poly unsaturated fatty acids are unique to some marine microalgae and protists and vary with environmental conditions, culture age and nutrient status, as well as with cultivation process. Calculated physical and chemical properties of density, viscosity of transesterified fatty acid methyl esters (FAMEs) were within the limits of the biodiesel standard specifications as per ASTM D6751-2012 and EN 14214. The calculated cetane number was, however, significantly lower (17.8~18.6) compared to ASTM D6751-2012 or EN 14214-specified minimal requirements. We conclude that the obtained microalgal biodiesel would likely only be suitable for blending with petroleum diesel to a maximum of 5 to 20%.

Evaluation of robust autonomy and implications on UAS certification and design

As the number of potential applications of Unmanned Aircraft Systems (UAS) grows in civilian operations and national security, National Airworthiness Authorities are under increasing pressure to provide a path for certification and allow UAS integration into the national airspace. The success of this integration depends on developments in improved UAS reliability and safety, regulations for certification, and technologies for operational performance and safety assessment. This paper focusses on the latter and describes the use of a framework for evaluating robust autonomy of UAS, namely, the autonomous system’s ability to either continue operation in the presence of faults or safely shut down. The paper draws parallels between the proposed evaluation framework and the evaluation of pilots during the licensing process. It also discusses how the data from the proposed evaluation can be uses as an aid for decision making in certification and UAS designs.

An outcome evaluation of the ‘Skipper’ designated driver program

The general aim of designated driver programs is to reduce the level of drink driving by encouraging potential drink drivers to travel with a driver who has abstained from (or at least limited) consuming alcohol. Designated driver programs appear to be quite widespread around the world, however a limited number have been subject to rigorous evaluation. This paper reports results from an outcome evaluation of a designated driver program called ‘Skipper’, which was trialled in a provincial city in Queensland, Australia. The outcome evaluation included surveys three weeks prior to (baseline), four months following (1st follow-up), and 16 months following (2nd follow-up) the commencement of the trial in both the ‘intervention area’ (baseline, n = 202; 1st follow-up, n = 211; 2nd follow-up, n = 200) and a ‘comparison area’(baseline, n = 203; 1st follow-up, n = 199; 2nd follow-up, n = 201); and a comparison of random breath testing and crash data before and after the trial. The survey results indicate that awareness of the program in the intervention area was quite high four months following its introduction and that this was maintained at 16 months. The results also suggest that the ‘Skipper’ program and the related publicity had positive impacts on behaviour with an increase in the proportion of people participating in designated driver as a passenger. It is less clear, however, whether the ‘Skipper’ program impacted on other behaviours of interest, such as drink driving or involvement in alcohol-related crashes. Suggestions for further research and program improvement are discussed as well as limitations of the research.

Transcript and metabolite profiling for the evaluation of tobacco tree and poplar as feedstock for the bio-based industry

The global demand for food, feed, energy and water poses extraordinary challenges for future generations. It is evident that robust platforms for the exploration of renewable resources are necessary to overcome these challenges. Within the multinational framework MultiBioPro we are developing biorefinery pipelines to maximize the use of plant biomass. More specifically, we use poplar and tobacco tree (Nicotiana glauca) as target crop species for improving saccharification, isoprenoid, long chain hydrocarbon contents, fiber quality, and suberin and lignin contents. The methods used to obtain these outputs include GC-MS, LC-MS and RNA sequencing platforms. The metabolite pipelines are well established tools to generate these types of data, but also have the limitations in that only well characterized metabolites can be used. The deep sequencing will allow us to include all transcripts present during the developmental stages of the tobacco tree leaf, but has to be mapped back to the sequence of Nicotiana tabacum. With these set-ups, we aim at a basic understanding for underlying processes and at establishing an industrial framework to exploit the outcomes. In a more long term perspective, we believe that data generated here will provide means for a sustainable biorefinery process using poplar and tobacco tree as raw material. To date the basal level of metabolites in the samples have been analyzed and the protocols utilized are provided in this article.

Evaluation of a community-based intervention to promote physical activity in youth : lessons from active winners

Purpose To test the effects of a community-based physical activity intervention designed to increase physical activity and to conduct an extensive process evaluation of the intervention. Design Quasi-experimental. Setting Two rural communities in South Carolina. One community received the intervention, and the other served as the comparison. Subjects Public school students who were in fifth grade at the start of the study (558 at baseline) were eligible to participate. A total of 436 students participated over the course of the study. Intervention The intervention included after-school and summer physical activity programs and home, school, and community components designed to increase physical activity in youth. The intervention took place over an 18-month period. Measures. Students reported after-school physical activity at three data collection points (prior to, during, and following the intervention) using the Previous Day Physical Activity Recall (PDPAR). They also completed a questionnaire designed to measure hypothesized psychosocial and environmental determinants of physical activity behavior The process evaluation used meeting records, documentation of program activities, interviews, focus groups, and heart rate monitoring to evaluate the planning and implementation of the intervention. Results There were no significant differences in the physical activity variables and few significant differences in the psychosocial variables between the intervention and comparison groups. The process evaluation indicated that the after-school and summer physical activity component of the intervention was implemented as planned, but because of resource and time limitations, the home, school, and community components were not implemented as planned. Conclusions The intervention did not have a significant effect on physical activity in the target population of children in the intervention community. This outcome is similar to that reported in other studies of community-based physical activity intervention.

Supporting process model validation through natural language generation

The design and development of process-aware information systems is often supported by specifying requirements as business process models. Although this approach is generally accepted as an effective strategy, it remains a fundamental challenge to adequately validate these models given the diverging skill set of domain experts and system analysts. As domain experts often do not feel confident in judging the correctness and completeness of process models that system analysts create, the validation often has to regress to a discourse using natural language. In order to support such a discourse appropriately, so-called verbalization techniques have been defined for different types of conceptual models. However, there is currently no sophisticated technique available that is capable of generating natural-looking text from process models. In this paper, we address this research gap and propose a technique for generating natural language texts from business process models. A comparison with manually created process descriptions demonstrates that the generated texts are superior in terms of completeness, structure, and linguistic complexity. An evaluation with users further demonstrates that the texts are very understandable and effectively allow the reader to infer the process model semantics. Hence, the generated texts represent a useful input for process model validation.