Formative evaluation for improving collaborative planning : a case study at the regional scale

In an earlier paper (Cameron & Johnson 2004) we introduced the idea of formative evaluation (or evaluation for development), the purpose of which is to provide information for improving planning programs and activities. This type of evaluation differs from the two other types: outcome evaluation which aims to judge the success or otherwise of a program; and evaluation for knowledge which seeks to contribute to theoretical work on planning processes and activities. In the earlier paper we also outlined the first stage of formative evaluation in the SEQ 2021 regional planning exercise showing how the process of planning for community engagement was modified in light of the evaluation findings. This current paper details the second stage of formative evaluation in which the collaborative planning component of SEQ 2021 was evaluated, as such it further demonstrates how formative evaluation can be used to improve planning programs. The evaluation findings also provide insights into strategies for more effective collaborative planning. We begin with an overview of collaborative approaches to regional planning, including the SEQ 2021 regional planning program. We then outline formal and informal evaluations of various collaborative regional planning exercises, including the predecessor of SEQ 2021 - SEQ 2001. This sets the scene for discussion of the approach used to evaluate the collaborative component of SEQ 2021. After outlining the main findings from the evaluation and the ways these findings were used to refine the collaborative planning process we conclude with a series of recommendations, relevant not only to SEQ 2021 but to other collaborative planning exercises

Lane filtering and situation awareness in motorcyclists : an on-road proof of concept study

In Victoria, as in other jurisdictions, there is very little research on the potential risks and benefits of lane filtering by motorcyclists, particularly from a road safety perspective. This on-road proof of concept study aimed to investigate whether and how lane filtering influences motorcycle rider situation awareness at intersections and to address factors that need to be considered for the design of a larger study in this area. Situation awareness refers to road users’ understanding of ‘what is going on’ around them and is a critical commodity for safe performance. Twenty-five experienced motorcyclists rode their own instrumented motorcycle around an urban test route in Melbourne whilst providing verbal protocols. Lane filtering occurred in 27% of 43 possible instances in which there were one or more vehicles in the traffic queue and the traffic lights were red on approach to the intersection. A network analysis procedure, based on the verbal protocols provided by motorcyclists, was used to identify differences in motorcyclist situation awareness between filtering and non-filtering events. Although similarities in situation awareness across filtering and nonfiltering motorcyclists were found, the analysis revealed some differences. For example, filtering motorcyclists placed more emphasis on the timing of the traffic light sequence and on their own actions when moving to the front of the traffic queue, whilst non-filtering motorcyclists paid greater attention to traffic moving through the intersection and approaching from behind. Based on the results of this study, the paper discusses some methodological and theoretical issues to be addressed in a larger study comparing situation awareness between filtering and non-filtering motorcyclists.

Protocol for a randomised blocked design study using telephone and text-messaging to support cardiac patients with diabetes : a cross cultural international collaborative project

Background The prevalence of type 2 diabetes is rising internationally. Patients with diabetes have a higher risk of cardiovascular events accounting for substantial premature morbidity and mortality, and health care expenditure. Given healthcare workforce limitations, there is a need to improve interventions that promote positive self-management behaviours that enable patients to manage their chronic conditions effectively, across different cultural contexts. Previous studies have evaluated the feasibility of including telephone and Short Message Service (SMS) follow up in chronic disease self-management programs, but only for single diseases or in one specific population. Therefore, the aim of this study is to evaluate the feasibility and short-term efficacy of incorporating telephone and text messaging to support the care of patients with diabetes and cardiac disease, in Australia and in Taiwan. Methods/design A randomised controlled trial design will be used to evaluate a self-management program for people with diabetes and cardiac disease that incorporates the use of simple remote-access communication technologies. A sample size of 180 participants from Australia and Taiwan will be recruited and randomised in a one-to-one ratio to receive either the intervention in addition to usual care (intervention) or usual care alone (control). The intervention will consist of in-hospital education as well as follow up utilising personal telephone calls and SMS reminders. Primary short term outcomes of interest include self-care behaviours and self-efficacy assessed at baseline and four weeks. Discussion If the results of this investigation substantiate the feasibility and efficacy of the telephone and SMS intervention for promoting self management among patients with diabetes and cardiac disease in Australia and Taiwan, it will support the external validity of the intervention. It is anticipated that empirical data from this investigation will provide valuable information to inform future international collaborations, while providing a platform for further enhancements of the program, which has potential to benefit patients internationally.

A cross-discipline and collaborative approach to identifying the predictors of environmentally friendly and health protective behaviour

Background. As a society, our interaction with the environment is having a negative impact on human health. For example, an increase in car use for short trips, over walking or cycling, has contributed to an increase in obesity, diabetes and poor heart health and also contributes to pollution, which is associated with asthma and other respiratory diseases. In order to change the nature of that interaction, to be more positive and healthy, it is recommended that individuals adopt a range of environmentally friendly behaviours (such as walking for transport and reducing the use of plastics). Effective interventions aimed at increasing such behaviours will need to be evidence based and there is a need for the rapid communication of information from the point of research, into policy and practice. Further, a number of health disciplines, including psychology and public health, share a common mission to promote health and well-being. Therefore, the objective of this project is to take a cross-discipline and collaborative approach to reveal psychological mechanisms driving environmentally friendly behaviour. This objective is further divided into three broad aims, the first of which is to take a cross-discipline and collaborative approach to research. The second aim is to explore and identify the salient beliefs which most strongly predict environmentally friendly behaviour. The third aim is to build an augmented model to explain environmentally friendly behaviour. The thesis builds on the understanding that an interdisciplinary collaborative approach will facilitate the rapid transfer of knowledge to inform behaviour change interventions. Methods. The application of this approach involved two surveys which explored the psycho-social predictors of environmentally friendly behaviour. Following a qualitative pilot study, and in collaboration with an expert panel comprising academics, industry professionals and government representatives, a self-administered, Theory of Planned Behaviour (TPB) based, mail survey was distributed to a random sample of 3000 residents of Brisbane and Moreton Bay Region (Queensland, Australia). This survey explored specific beliefs including attitudes, norms, perceived control, intention and behaviour, as well as environmental altruism and green identity, in relation to walking for transport and switching off lights when not in use. Following analysis of the mail survey data and based on feedback from participants and key stakeholders, an internet survey was employed (N=451) to explore two additional behaviours, switching off appliances at the wall when not in use, and shopping with reusable bags. This work is presented as a series of interrelated publications which address each of the research aims. Presentation of Findings. Chapter five of this thesis consists of a published paper which addresses the first aim of the research and outlines the collaborative and multidisciplinary approach employed in the mail survey. The paper argued that forging alliances with those who are in a position to immediately utilise the findings of research has the potential to improve the quality and timely communication of research. Illustrating this timely communication, Chapter six comprises a report presented to Moreton Bay Regional Council (MBRC). This report addresses aim's one and two. The report contains a summary of participation in a range of environmentally friendly behaviours and identifies the beliefs which most strongly predicted walking for transport and switching off lights (from the mail survey). These salient beliefs were then recommended as targets for interventions and included: participants believing that they might save money; that their neighbours also switch off lights; that it would be inconvenient to walk for transport and that their closest friend also walks for transport. Chapter seven also addresses the second aim and presents a published conference paper in which the salient beliefs predicting the four specified behaviours (from both surveys) are identified and potential applications for intervention are discussed. Again, a range of TPB based beliefs, including descriptive normative beliefs, were predictive of environmentally friendly behaviour. This paper was also provided to MBRC, along with recommendations for applying the findings. For example, as descriptive normative beliefs were consistently correlated with environmentally friendly behaviour, local councils could engage in marketing and interventions (workshops, letter box drops, internet promotions) which encourage parents and friends to model, rather than simply encourage, environmentally friendly behaviour. The final two papers, presented in Chapters eight and nine, addresses the third aim of the project. These papers each present two behaviours together to inform a TPB based theoretical model with which to predict environmentally friendly behaviour. A generalised model is presented, which is found to predict the four specific behaviours under investigation. The role of demographics was explored across each of the behaviour specific models. It was found that some behaviour's differ by age, gender, income or education. In particular, adjusted models predicted more of the variance in walking for transport amongst younger participants and females. Adjusted models predicted more variance in switching off lights amongst those with a bachelor degree or higher and predicted more variance in switching off appliances amongst those on a higher income. Adjusted models predicted more variance in shopping with reusable bags for males, people 40 years or older, those on a higher income and those with a bachelor degree or higher. However, model structure and general predictability was relatively consistent overall. The models provide a general theoretical framework from which to better understand the motives and predictors of environmentally friendly behaviour. Conclusion. This research has provided an example of the benefits of a collaborative interdisciplinary approach. It has identified a number of salient beliefs which can be targeted for social marketing campaigns and educational initiatives; and these findings, along with recommendations, have been passed on to a local council to be used as part of their ongoing community engagement programs. Finally, the research has informed a practical model, as well as behaviour specific models, for predicting sustainable living behaviours. Such models can highlight important core constructs from which targeted interventions can be designed. Therefore, this research represents an important step in undertaking collaborative approaches to improving population health through human-environment interactions.

A grand challenge : reflecting on university science curriculum design, collaborative partnerships and integration of information literacy skills

In 2012, Queensland University of Technology (QUT) committed to the massive project of revitalizing its Bachelor of Science (ST01) degree. Like most universities in Australia, QUT has begun work to align all courses by 2015 to the requirements of the updated Australian Qualifications Framework (AQF) which is regulated by the Tertiary Education Quality and Standards Agency (TEQSA). From the very start of the redesigned degree program, students approach scientific study with an exciting mix of theory and highly topical real world examples through their chosen “grand challenge.” These challenges, Fukushima and nuclear energy for example, are the lenses used to explore science and lead to 21st century learning outcomes for students. For the teaching and learning support staff, our grand challenge is to expose all science students to multidisciplinary content with a strong emphasis on embedding information literacies into the curriculum. With ST01, QUT is taking the initiative to rethink not only content but how units are delivered and even how we work together between the faculty, the library and learning and teaching support. This was the desired outcome but as we move from design to implementation, has this goal been achieved? A main component of the new degree is to ensure scaffolding of information literacy skills throughout the entirety of the three year course. However, with the strong focus on problem-based learning and group work skills, many issues arise both for students and lecturers. A move away from a traditional lecture style is necessary but impacts on academics’ workload and comfort levels. Therefore, academics in collaboration with librarians and other learning support staff must draw on each others’ expertise to work together to ensure pedagogy, assessments and targeted classroom activities are mapped within and between units. This partnership can counteract the tendency of isolated, unsupported academics to concentrate on day-to-day teaching at the expense of consistency between units and big picture objectives. Support staff may have a more holistic view of a course or degree than coordinators of individual units, making communication and truly collaborative planning even more critical. As well, due to staffing time pressures, design and delivery of new curriculum is generally done quickly with no option for the designers to stop and reflect on the experience and outcomes. It is vital we take this unique opportunity to closely examine what QUT has and hasn’t achieved to be able to recommend a better way forward. This presentation will discuss these important issues and stumbling blocks, to provide a set of best practice guidelines for QUT and other institutions. The aim is to help improve collaboration within the university, as well as to maximize students’ ability to put information literacy skills into action. As our students embark on their own grand challenges, we must challenge ourselves to honestly assess our own work.

Pattern-based topic models for information filtering

Topic modelling, such as Latent Dirichlet Allocation (LDA), was proposed to generate statistical models to represent multiple topics in a collection of documents, which has been widely utilized in the fields of machine learning and information retrieval, etc. But its effectiveness in information filtering is rarely known. Patterns are always thought to be more representative than single terms for representing documents. In this paper, a novel information filtering model, Pattern-based Topic Model(PBTM) , is proposed to represent the text documents not only using the topic distributions at general level but also using semantic pattern representations at detailed specific level, both of which contribute to the accurate document representation and document relevance ranking. Extensive experiments are conducted to evaluate the effectiveness of PBTM by using the TREC data collection Reuters Corpus Volume 1. The results show that the proposed model achieves outstanding performance.

Doing collaborative reflection in the professional experience

Despite the importance of the school-based practicum experience in teacher education programs, only limited research has investigated how supervising and pre-service teacher roles and relationships are interactively achieved in situ. Using conversation analysis, we interrogate extracts of practicum talk between supervising teachers (STs) and their pre-service teachers (PTs), revealing that asymmetrical relationships are talked into being within and through their conversations about classroom practice. We compare the different structural conversational arrangements that are employed when STs provide either positive feedback or raise issues of potential difficulty in relation to their PTs’ observed classroom activities. Analysis reveals that STs tend to make evaluative statements when providing positive feedback, in contrast to initiating a process of critical reflection when talking about the need for improvements in PTs’ pedagogic practices. We conclude by arguing the need for STs to instigate sustained critical and collaborative professional conversations with their PTs during the practicum experience, enabling them to engage in reflective practice and providing them with opportunities to extend their professional knowledge and skills, thereby potentially improving their future classroom practices.

Validation of an instrument to measure governance and performance on collaborative infrastructure projects

Collaborative infrastructure projects use hybrid formal and informal governance structures to manage transactions. Based on previous desk-top research, the authors identified the key mechanisms underlying project governance, and posited the performance implications of the governance (Chen et al. 2012). The current paper extends that qualitative research by testing the veracity of those findings using data from 320 Australian construction organisations. The results provide, for the first time, reliable and valid scales to measure governance and performance of collaborative projects, and the relationship between them. The results confirm seven of seven hypothesised governance mechanisms; 30 of 43 hypothesised underlying actions; eight of eight hypothesised key performance indicators; and the dual importance of formal and informal governance. A startling finding of the study was that the implementation intensity of informal mechanisms (non-contractual conditions) is a greater predictor of project performance variance than that of formal mechanisms (contractual conditions). Further, contractual conditions do not directly impact project performance; instead their impact is mediated by the non-contractual features of a project. Obligations established under the contract are not sufficient to optimise project performance.

Perioperative medication management: expanding the role of the preadmission clinic pharmacist in a single centre, randomised controlled trial of collaborative prescribing

Objectives Current evidence to support non-medical prescribing is predominantly qualitative, with little evaluation of accuracy, safety and appropriateness. Our aim was to evaluate a new model of service for the Australia healthcare system, of inpatient medication prescribing by a pharmacist in an elective surgery preadmission clinic (PAC) against usual care, using an endorsed performance framework. Design Single centre, randomised controlled, two-arm trial. Setting Elective surgery PAC in a Brisbane-based tertiary hospital. Participants 400 adults scheduled for elective surgery were randomised to intervention or control. Intervention A pharmacist generated the inpatient medication chart to reflect the patient's regular medication, made a plan for medication perioperatively and prescribed venous thromboembolism (VTE) prophylaxis. In the control arm, the medication chart was generated by the Resident Medical Officers. Outcome measures Primary outcome was frequency of omissions and prescribing errors when compared against the medication history. The clinical significance of omissions was also analysed. Secondary outcome was appropriateness of VTE prophylaxis prescribing. Results There were significantly less unintended omissions of medications: 11 of 887 (1.2%) intervention orders compared with 383 of 1217 (31.5%) control (p<0.001). There were significantly less prescribing errors involving selection of drug, dose or frequency: 2 in 857 (0.2%) intervention orders compared with 51 in 807 (6.3%) control (p<0.001). Orders with at least one component of the prescription missing, incorrect or unclear occurred in 208 of 904 (23%) intervention orders and 445 of 1034 (43%) controls (p<0.001). VTE prophylaxis on admission to the ward was appropriate in 93% of intervention patients and 90% controls (p=0.29). Conclusions Medication charts in the intervention arm contained fewer clinically significant omissions, and prescribing errors, when compared with controls. There was no difference in appropriateness of VTE prophylaxis on admission between the two groups.

Pattern-based topics for document modelling in information filtering

Many mature term-based or pattern-based approaches have been used in the field of information filtering to generate users’ information needs from a collection of documents. A fundamental assumption for these approaches is that the documents in the collection are all about one topic. However, in reality users’ interests can be diverse and the documents in the collection often involve multiple topics. Topic modelling, such as Latent Dirichlet Allocation (LDA), was proposed to generate statistical models to represent multiple topics in a collection of documents, and this has been widely utilized in the fields of machine learning and information retrieval, etc. But its effectiveness in information filtering has not been so well explored. Patterns are always thought to be more discriminative than single terms for describing documents. However, the enormous amount of discovered patterns hinder them from being effectively and efficiently used in real applications, therefore, selection of the most discriminative and representative patterns from the huge amount of discovered patterns becomes crucial. To deal with the above mentioned limitations and problems, in this paper, a novel information filtering model, Maximum matched Pattern-based Topic Model (MPBTM), is proposed. The main distinctive features of the proposed model include: (1) user information needs are generated in terms of multiple topics; (2) each topic is represented by patterns; (3) patterns are generated from topic models and are organized in terms of their statistical and taxonomic features, and; (4) the most discriminative and representative patterns, called Maximum Matched Patterns, are proposed to estimate the document relevance to the user’s information needs in order to filter out irrelevant documents. Extensive experiments are conducted to evaluate the effectiveness of the proposed model by using the TREC data collection Reuters Corpus Volume 1. The results show that the proposed model significantly outperforms both state-of-the-art term-based models and pattern-based models

Hospital pharmacists' views on collaborative pharmacist prescribing

In 2008, Weeks et al. published the results of a postal survey, which explored the views of the Society of Hospital Pharmacists of Australia’s (SHPA) members on collaborative prescribing, and the extent of de facto prescribing in their institution. Since then, significant work has been undertaken on non-medical prescribing, such as pilots of pharmacist prescribing across Australia and a National Health Workforce report on developing a nationally consistent approach to prescribing by nonmedical health professionals. The first stage of the Health Workforce Australia Health Practitioner Prescribing Pathway project is complete and the recommendations for implementation have been approved by the Standing Council in November 2013. New Zealand pharmacists obtained prescribing rights in 2013, and the first cohort of 14 prescribers have completed the postgraduate pharmacist prescribing course (jointly run by the Otago and Auckland Universities).

A collaborative robotics engagement project

This paper reports on an evaluation of a collaborative robotics engagement project involving teachers from local schools and an academic from Queensland University of Technology (QUT). Engaged community projects are aimed at building stronger relationships between universities and their local communities (Sandman, Williams & Abrams, 2009). This partnership leads to mutually beneficial outcomes, builds community capacity, and can focus on aspirations and access to higher education for school students (Scull & Cuthill, 2010). The Robotics@QUT project aimed to build a partnership between local teachers and the university in order to provide students from a low SES area opportunity to engage in robotics-based Science, Technology, Engineering, and Mathematics (STEM) activities. Students from low SES regions are underrepresented at university and less likely to pursue studies in these fields (Bradley, Noonan, Nugent, & Scales, 2008). Having teachers who provide engaging STEM activities is an important motivating factor for students to enjoy STEM and do well in STEM subjects (Tytler, Osborne Williams Tytler & Clark, 2008).

Improved rural health through a collaborative model of care – qualitative findings

INTRODUCTION Health disparity between urban and rural regions in Australia is well-documented. In the Wheatbelt catchments of Western Australia there is higher incidence and rate of avoidable hospitalisation for chronic diseases. Structured care approach to chronic illnesses is not new but the focus has been on single disease state. A recent ARC Discovery Project on general practice nurse-led chronic disease management of diabetes, hypertension and stable ischaemic heart disease reported improved communication and better medical administration.[1] In our study we investigated the sustainability of such a multi-morbidities general practice –led collaborative model of care in rural Australia. METHODS A QUAN(qual) design was utilised. Eight pairs of rural general practices were matched. Inclusion criteria used were >18 years and capable of giving informed consent, at least one identified risk factor or diagnosed with chronic conditions. Patients were excluded if deemed medically unsuitable. A comprehensive care plan was formulated by the respective general practice nurse in consultation with the treating General Practitioner (GP) and patient based on the individual’s readiness to change, and was informed by available local resource. A case management approach was utilised. Shediaz-Rizkallah and Lee’s conceptual framework on sustainability informed our evaluation.[2] Our primary outcome on measures of sustainability was reduction in avoidable hospitalisation. Secondary outcomes were patients and practitioners acceptance and satisfaction, and changes to pre-determined interim clinical and process outcomes. RESULTS The qualitative interviews highlighted the community preference for a ‘sustainable’ local hospital in addition to general practice. Costs, ease of access, low prioritisation of self chronic care, workforce turnover and perception of losing another local resource if underutilised influenced the respondents’ decision to present at local hospital for avoidable chronic diseases regardless. CONCLUSIONS Despite the pragmatic nature of rural general practice in Australia, the sustainability of chronic multi-morbidities management in general practice require efficient integration of primary-secondary health care and consideration of other social determinants of health. What this study adds: What is already known on this subject: Structured approach to chronic disease management is not new and has been shown to be effective for reducing hospitalisation. However, the focus has been on single disease state. What does this study add: Sustainability of collaborative model of multi-morbidities care require better primary-secondary integration and consideration of social determinants of health.

Professional conversations : a collaborative approach to support policy implementation, professional learning and practice change in ECEC

In 2012, Australia introduced a new National Quality Framework, comprising enhanced quality expectations for early childhood education and care services, two national learning frameworks and a new Assessment and Rating System spanning child care centres, kindergartens and preschools, family day care and outside school hours care. This is the linchpin in a series of education reforms designed to support increased access to higher quality early childhood education and care (ECEC) and successful transition to school. As with any policy change, success in real terms relies upon building shared understanding and the capacity of educators to apply new knowledge and to support change and improved practice within their service. With this in mind, a collaborative research project investigated the efficacy of a new approach to professional learning in ECEC: the professional conversation. This paper reports on the trial and evaluation of a series of professional conversations to support implementation of one element of the NQF, the Early Years Learning Framework (DEEWR,2009), and their capacity to promote collaborative reflective practice, shared understanding, and improved practice in ECEC. Set against the backdrop of the NQF, this paper details the professional conversation approach, key challenges and critical success factors, and the learning outcomes for conversation participants. Findings support the efficacy of this approach to professional learning in ECEC, and its capacity to support policy reform and practice change in ECEC.