Understanding evaluation of learning support in mathematics and statistics

With rapid and continuing growth of learning support initiatives in mathematics and statistics found in many parts of the world, and with the likelihood that this trend will continue, there is a need to ensure that robust and coherent measures are in place to evaluate the effectiveness of these initiatives. The nature of learning support brings challenges for measurement and analysis of its effects. After briefly reviewing the purpose, rationale for, and extent of current provision, this article provides a framework for those working in learning support to think about how their efforts can be evaluated. It provides references and specific examples of how workers in this field are collecting, analysing and reporting their findings. The framework is used to structure evaluation in terms of usage of facilities, resources and services provided, and also in terms of improvements in performance of the students and staff who engage with them. Very recent developments have started to address the effects of learning support on the development of deeper approaches to learning, the affective domain and the development of communities of practice of both learners and teachers. This article intends to be a stimulus to those who work in mathematics and statistics support to gather even richer, more valuable, forms of data. It provides a 'toolkit' for those interested in evaluation of learning support and closes by referring to an on-line resource being developed to archive the growing body of evidence. © 2011 Taylor & Francis.

Practical decision support system evaluation for consultant selection

The widespread development of Decision Support System (DSS) in construction indicate that the evaluation of software become more important than before. However, it is identified that most research in construction discipline did not attempt to assess its usability. Therefore, little is known about the approach on how to properly evaluate a DSS for specific problem. In this paper, we present a practical framework that can be guidance for DSS evaluation. It focuses on how to evaluate software that is dedicatedly designed for consultant selection problem. The framework features two main components i.e. Sub-system Validation and Face Validation. Two case studies of consultant selection at Malaysian Department of Irrigation and Drainage were integrated in this framework. Some inter-disciplinary area such as Software Engineering, Human Computer Interaction (HCI) and Construction Project Management underpinned the discussion of the paper. It is anticipated that this work can foster better DSS development and quality decision making that accurately meet the client’s expectation and needs

Technology transfer evaluation in the high technology industry : an interdisciplinary perspective

The underlying objective of this study was to develop a novel approach to evaluate the potential for commercialisation of a new technology. More specifically, this study examined the 'ex-ante'. evaluation of the technology transfer process. For this purpose, a technology originating from the high technology sector was used. The technology relates to the application of software for the detection of weak signals from space, which is an established method of signal processing in the field of radio astronomy. This technology has the potential to be used in commercial and industrial areas other than astronomy, such as detecting water leakages in pipes. Its applicability to detecting water leakage was chosen owing to several problems with detection in the industry as well as the impact it can have on saving water in the environment. This study, therefore, will demonstrate the importance of interdisciplinary technology transfer. The study employed both technical and business evaluation methods including laboratory experiments and the Delphi technique to address the research questions. There are several findings from this study. Firstly, scientific experiments were conducted and these resulted in a proof of concept stage of the chosen technology. Secondly, validation as well as refinement of criteria from literature that can be used for „ex-ante. evaluation of technology transfer has been undertaken. Additionally, after testing the chosen technology.s overall transfer potential using the modified set of criteria, it was found that the technology is still in its early stages and will require further development for it to be commercialised. Furthermore, a final evaluation framework was developed encompassing all the criteria found to be important. This framework can help in assessing the overall readiness of the technology for transfer as well as in recommending a viable mechanism for commercialisation. On the whole, the commercial potential of the chosen technology was tested through expert opinion, thereby focusing on the impact of a new technology and the feasibility of alternate applications and potential future applications.

Patterns-based evaluation of open source BPM systems: The cases of jBPM, OpenWFE, and Enhydra Shark

In keeping with the proliferation of free software development initiatives and the increased interest in the business process management domain, many open source workflow and business process management systems have appeared during the last few years and are now under active development. This upsurge gives rise to two important questions: What are the capabilities of these systems? and How do they compare to each other and to their closed source counterparts? In other words: What is the state-of-the-art in the area?. To gain an insight into these questions, we have conducted an in-depth analysis of three of the major open source workflow management systems – jBPM, OpenWFE, and Enhydra Shark, the results of which are reported here. This analysis is based on the workflow patterns framework and provides a continuation of the series of evaluations performed using the same framework on closed source systems, business process modelling languages, and web-service composition standards. The results from evaluations of the three open source systems are compared with each other and also with the results from evaluations of three representative closed source systems: Staffware, WebSphere MQ, and Oracle BPEL PM. The overall conclusion is that open source systems are targeted more toward developers rather than business analysts. They generally provide less support for the patterns than closed source systems, particularly with respect to the resource perspective, i.e. the various ways in which work is distributed amongst business users and managed through to completion.

Understanding IT business value creation and evaluation in least developed economies

Information Technology (IT) is an important resource that can facilitate growth and development in both the developed and developing economies. The forces of globalisation increase the digital divide between the developed and developing economies is increasing. The least developed economies (LDEs) are the most vulnerable within this environment. Intense competition for IT resources means that LDEs need a deeper understanding of how to source and evaluate their IT-related efforts. This effort puts LDEs in a better position to source funding from various stakeholders and promote localized investment in IT. This study presents a complementary approach to securing better IT-related business value in organizations in the LDEs. It further evaluates how IT and the complementaries need to managed within the LDEs. Analysis of data collected from five LDEs show that organizations that invest in IT and related complementaries are able to better their business processes. The data also suggest that improved business processes lead to overall business processes improvements. The above is only possible if organizations adopt IT and make related changes in the complementary resources within the established culture and localizing the required changes.

Evaluation of design-builder qualifications through the analysis of Requests for Qualifications (RFQs)

The two-phrase best-value process has been widely used by public agencies for Design and Build (DB) procurement, with a key issue in the first phase of pre-qualification being the determination of evaluation criteria. This study identified a set of general qualification criteria for design-builders and compares their relative importance by a thorough content analysis of 97 Requests for Qualification (RFQ) for public DB projects advertised between 2000 and 2011 in various regions of the USA. The thirty-nine qualification criteria found are summarized and classified into eight categories comprising: experience; project understanding and approach; organizational structure and capacity; past performance record; professional qualifications; responsiveness to RFQs, office location and familiarity with local environment; and legal status in descending order of their relative importance. A comparative analysis of different types of projects shows that the relative weightings of the qualification criteria vary according to different characteristics of the DB projects involved.

Scheduling Trains with Priorities : a no-wait blocking parallel-machine job-shop scheduling model

The paper investigates train scheduling problems when prioritised trains and non-prioritised trains are simultaneously traversed in a single-line rail network. In this case, no-wait conditions arise because the prioritised trains such as express passenger trains should traverse continuously without any interruption. In comparison, non-prioritised trains such as freight trains are allowed to enter the next section immediately if possible or to remain in a section until the next section on the routing becomes available, which is thought of as a relaxation of no-wait conditions. With thorough analysis of the structural properties of the No-Wait Blocking Parallel-Machine Job-Shop-Scheduling (NWBPMJSS) problem that is originated in this research, an innovative generic constructive algorithm (called NWBPMJSS_Liu-Kozan) is proposed to construct the feasible train timetable in terms of a given order of trains. In particular, the proposed NWBPMJSS_Liu-Kozan constructive algorithm comprises several recursively-used sub-algorithms (i.e. Best-Starting-Time-Determination Procedure, Blocking-Time-Determination Procedure, Conflict-Checking Procedure, Conflict-Eliminating Procedure, Tune-up Procedure and Fine-tune Procedure) to guarantee feasibility by satisfying the blocking, no-wait, deadlock-free and conflict-free constraints. A two-stage hybrid heuristic algorithm (NWBPMJSS_Liu-Kozan-BIH) is developed by combining the NWBPMJSS_Liu-Kozan constructive algorithm and the Best-Insertion-Heuristic (BIH) algorithm to find the preferable train schedule in an efficient and economical way. Extensive computational experiments show that the proposed methodology is promising because it can be applied as a standard and fundamental toolbox for identifying, analysing, modelling and solving real-world scheduling problems.

A hybrid shifting bottleneck procedure algorithm for the parallel-machine job-shop scheduling problem

In practice, parallel-machine job-shop scheduling (PMJSS) is very useful in the development of standard modelling approaches and generic solution techniques for many real-world scheduling problems. In this paper, based on the analysis of structural properties in an extended disjunctive graph model, a hybrid shifting bottleneck procedure (HSBP) algorithm combined with Tabu Search metaheuristic algorithm is developed to deal with the PMJSS problem. The original-version SBP algorithm for the job-shop scheduling (JSS) has been significantly improved to solve the PMJSS problem with four novelties: i) a topological-sequence algorithm is proposed to decompose the PMJSS problem into a set of single-machine scheduling (SMS) and/or parallel-machine scheduling (PMS) subproblems; ii) a modified Carlier algorithm based on the proposed lemmas and the proofs is developed to solve the SMS subproblem; iii) the Jackson rule is extended to solve the PMS subproblem; iv) a Tabu Search metaheuristic algorithm is embedded under the framework of SBP to optimise the JSS and PMJSS cases. The computational experiments show that the proposed HSBP is very efficient in solving the JSS and PMJSS problems.

A process evaluation of a comprehensive school based injury prevention program

A process evaluation enables understanding of critical issues that can inform the improved, ongoing implementation of an intervention program. This study describes the process evaluation of a comprehensive, multi-level injury prevention program for adolescents. The program targets change in injury associated with violence, transport and alcohol risks and incorporates two primary elements: an 8-week, teacher delivered attitude and behaviour change curriculum for Grade 8 students; and a professional development program for teachers on school level methods of protection, focusing on strategies to increase students’ connectedness to school.

How effective is the new child restraint legislation in Queensland? A mixed methods evaluation

Research Interests: Are parents complying with the legislation? Is this the same for urban, regional and rural parents? Indigenous parents? What difficulties do parents experience in complying? Do parents understand why the legislation was put in place? Have there been negative consequences for other organisations or sectors of the community?

An evaluation framework for assessing the impact of public bicycle share schemes

INTRODUCTION • Public bicycle share schemes have emerged as a method of increasing rates of bicycle riding. • The overwhelming majority of schemes have begun since 2005, taking advantage of various tracking and payment technologies making short term rental practical and affordable. • Very little research has been undertaken to determine their potentially broad impact on transport behaviour and consequently, it is difficult to understand the performance of these schemes in terms of reduced emissions and congestion, as well as possible increases in physical activity.

"Silly hours" : facilitating more effective part-time work arrangements through job design

Part-time employment presents a conundrum in that it facilitates work-life priorities, while also, compared to equivalent full-time roles, attracting penalties such as diminished career prospects and lower commensurate remuneration. Recently, some promising theoretical developments in the job/work design literature suggest that consideration of work design may redress some of the penalties associated with part-time work. Adopting the framework of the Elaborated Model of Work Design by Parker and colleagues (2001), we examined this possibility through interviews with part-time professional service employees and their supervisors. The findings revealed that in organizations characterised by cultural norms of extended working hours and a singular-focused commitment to work, part-time roles were often inadequately re-designed when adapted from full-time arrangements. The findings also demonstrated that certain work design characteristics (e.g. predictability of work-flow, interdependencies with co-workers) render some roles more suitable for part-time arrangements than others. The research provides insights into gaps between policy objectives and outcomes associated with part-time work, challenges assumptions about the limitations of part-time roles, and suggests re-design strategies for more effective part-time arrangements.

When non-significance maybe significant: lessons learned from a study into the development, implementation and evaluation of a risk assessment tool for fleet settings

Introduction This study reports on the development of a self report assessment tool to increase the efficacy of crash prediction within Australian Fleet settings Over last 20 years an array of measures have been produced (Driver anger scale, Driving Skill Inventory, Manchester Driver Behaviour Questionnaire, Driver Attitude Questionnaire, Driver Stress Inventory, Safety Climate Questionnaire) While these tools are useful, research has demonstrated limited ability to accurately identify individuals most likely to be involved in a crash. Reasons cited include; - Crashes are relatively rare - Other competing factors may influence crash event - Ongoing questions regarding the validity of self report measures (common method variance etc) - Lack of contemporary issues relating to fleet driving performance

Evaluation of the 2010 child restraint legislation in Queensland

Occupant injury comprises the largest proportion of child road crash trauma in most highly motorised countries. In Australia, road crashes are the primary cause of death for children aged 1-14 years and are among the top three causes of serious injury to this age group. For this reason considerable research attention has been focused on understanding the contributing factors and the most effective ways of improving children’s safety as car passengers. Australia has been particularly active in this area, with well regarded work being conducted on levels of use of dedicated child restraints, restraint crash performance in laboratory conditions, examination of real world restraint crash performance (case review), and studies of psychosocial factors influencing perceptions about restraints and their use (Brown & Bilston, 2006; Brown, McCaskill, Henderson & Bilston, 2006; Edwards, Anderson & Hutchinson, 2006; Lennon, 2005, 2007). New legislation for the restraint of children as vehicle passengers was enacted in Queensland in March 2010. This new legislation recognises the importance of dedicated restraint use for children up to at least age 7 years and the protective benefits of rear seating position in the event of a crash. As part of improving children’s safety and addressing key priority areas, the Queensland Injury Prevention Council (QIPC) and Department of Transport and Main Roads (TMR) commissioned the Centre for Accident Research and Road Safety, Queensland (CARRS-Q) to evaluate the impact of the new legislation. Although at the time of commencing the research the legislation had only been in force for 14 months, it was deemed critical to review its effectiveness in guiding parental choices and compliance in order to inform the design and focus of further supporting initiatives and interventions. Specifically, the research sought clear evidence of exactly what impact, if any, the legislation has had on compliance levels and what difficulties (if any) parents/carers experience in relation to interpreting as well as complying with the requirements of the new law. Knowledge about these barriers or difficulties will allow any future changes or improvements to the legislation to address such barriers and thus improve its effectiveness. Moreover, better information about how the legislation has affected parents will provide a basis to plan non-legislative comprehensive multi-strategy interventions such as community, educational or behavioural interventions with parents/carers and other stakeholder groups. In addition, it will allow identification of the most effective aspects of the legislation and those areas in need of extra attention to improve effectiveness/compliance and thus better protect children travelling in cars and improve their health and safety. This report presents the findings from the four components of the research: the literature review; observational study; intercept interviews and focus group with parents; and the interviews with key stakeholders.

Performance evaluation of eight contemporary passive solar homes in subtropical Australia

Subtropical south-east Queensland’s expanding population is expected to lead to a demand for an additional 754,000 dwellings by 2031. A legacy of poor housing design, minimal building regulations, an absence of building performance evaluation and various social and market factors has lead to a high and growing penetration of, and reliance on, air conditioners to provide comfort in this relatively benign climate. This reliance impacts on policy goals to adapt to and mitigate against global warming, electricity infrastructure investment and household resilience. Based on the concept of bioclimatic design, this field study scrutinizes eight non-air conditioned homes to develop a deeper understanding of the role of contemporary passive solar architecture in the delivery of thermally comfortable and resilient homes in the subtropics. These homes were found to provide inhabitants with an acceptable level of thermal comfort (18-28oC) for 77 – 97% of the year. Family expectations and experiences of comfort, and the various design strategies utilized were compared against the measured performance outcomes. This comparison revealed issues that limited quantification and implementation of design intent and highlighted factors that constrained system optimisation.