Exploring the Possibilities and Methodological Challenges of Evaluation Practice : A Democratic Question

In our exploration of the methodological possibilities and challenges of evaluation practice that foregrounds a democratic approach, we argue that having sensitivity to participants’ different positions, ideological perspectives, and values, and an understanding of the existing power relationships are paramount for the evaluators if the evaluation is to achieve its goals. This chapter draws on our experiences as evaluators in Australia, where school leaders and teachers are currently experiencing significant curriculum and assessment changes, and now work in a context of heightened accountability. Our evaluation practices with schools are used to illustrate how democratic practices can be incorporated into evaluation in ways that provide opportunities and support for school self-evaluation. It is paramount to recognise that evaluation is a political and ideological practice, and therefore is not a neutral process. Our argument is that it remains the responsibility of evaluators to design spaces that build sustainable relationships with participants and the targets of the programmes being evaluated; ensure that dialogue and deliberation are valued between all stakeholders; represent a range of interests; and give a service back to the evaluated communities by offering understandings and promoting reflective practice and informed decision-making throughout programme implementation.

The potential of combining phenomenography, variation theory and threshold concepts to inform curriculum design in higher education

This chapter describes an innovative method of curriculum design that is based on combining phenomenographic research, and the associated variation theory of learning, with the notion of disciplinary threshold concepts to focus specialised design attention on the most significant and difficult parts of the curriculum. The method involves three primary stages: (i) identification of disciplinary concepts worthy of intensive curriculum design attention, using the criteria for threshold concepts; (ii) action research into variation in students’ understandings/misunderstandings of those concepts, using phenomenography as the research approach; (iii) design of learning activities to address the poorer understandings identified in the second stage, using variation theory as a guiding framework. The curriculum design method is inherently theory and evidence based. It was developed and trialed during a two-year project funded by the Australian Learning and Teaching Council, using physics and law disciplines as case studies. Disciplinary teachers’ perceptions of the impact of the method on their teaching and understanding of student learning were profound. Attempts to measure the impact on student learning were less conclusive; teachers often unintentionally deviated from the design when putting it into practice for the first time. Suggestions for improved implementation of the method are discussed.

A methodology for effective implementation of lean strategies and its performance evaluation in manufacturing organizations

Purpose – The purpose of this paper is to develop an effective methodology for implementing lean manufacturing strategies and a leanness evaluation metric using continuous performance measurement (CPM). Design/methodology/approach – Based on five lean principles, a systematic lean implementation methodology for manufacturing organizations has been proposed. A simplified leanness evaluation metric consisting of both efficiency and effectiveness attributes of manufacturing performance has been developed for continuous evaluation of lean implementation. A case study to validate the proposed methodology has been conducted and proposed CPM metric has been used to assess the manufacturing leanness. Findings – Proposed methodology is able to systematically identify manufacturing wastes, select appropriate lean tools, identify relevant performance indicators, achieve significant performance improvement and establish lean culture in the organization. Continuous performance measurement matrices in terms of efficiency and effectiveness are proved to be appropriate methods for continuous evaluation of lean performance. Research limitations/implications – Effectiveness of the method developed has been demonstrated by applying it in a real life assembly process. However, more tests/applications will be necessary to generalize the findings. Practical implications – Results show that applying the methods developed, managers can successfully identify and remove manufacturing wastes from their production processes. By improving process efficiency, they can optimize their resource allocations. Manufacturers now have a validated step by step methodology for successfully implementing lean strategies. Originality/value – According to the authors’ best knowledge, this is the first known study that proposed a systematic lean implementation methodology based on lean principles and continuous improvement techniques. Evaluation of performance improvement by lean strategies is a critical issue. This study develops a simplified leanness evaluation metric considering both efficiency and effectiveness attributes and integrates it with the lean implementation methodology.

Evaluation framework for green procurement in road construction

Purpose Most barriers and enablers of sustainable projects are related to procurement. This study proposes a framework for evaluating green procurement practices throughout the lifecycle of road construction projects and demonstrates its application through an Australian case study. Design/methodology/approach The study is based on linking the phases of road construction with incentive mechanisms for proactively motivating behavioural change. A holistic view on utilised and potential incentives is attempted with a literature review and a state-of-practice review. The latter is based on interviews and 90 policy and procurement documents across five Australian states. Findings An evaluation framework with seven procurement stages is suggested to describe current state green procurement incentives throughout the delivery lifecycle of road construction projects. The Australian case study was found to provide useful data to identify gaps and strong points of the different states regarding their level of integration of sustainability and greenhouse gas emissions GHG) reduction elements in their procurement practices. This understanding was used to draw recommendations on future advancement of green procurement. Originality/value: Government entities across the globe can impact considerably the achievement of sustainability and GHG targets, by using their procurement practices and requirements to create incentives for contractors and suppliers to engage in more GHG conscious practices. The present study provides a systematic account of how green procurement practices can be underpinned using the Australian road construction industry as a case study, and distinguish between strong and weak links in the green procurement chain to draw recommendations for future initiatives.

Recovering from the earthquake : early childhood teachers and children collaboratively telling stories about their experiences

Purpose – Traumatic events can cause post-traumatic stress disorder due to the severity of the often unexpected events. The purpose of this paper is to reveal how conversations around lived experiences of traumatic events, such as the Christchurch earthquake in February 2011, can work as a strategy for people to come to terms with their experiences collaboratively. By encouraging young children to recall and tell of their earthquake stories with their early childhood teachers they can begin to respond, renew, and recover (Brown, 2012), and prevent or minimise more stress being developed. Design/methodology/approach – The study involved collecting data of the participating children taking turns to wear a wireless microphone where their interactions with each other and with teachers were video recorded over one week in November 2011. A total of eight hours and 21 minutes of footage was collected; four minutes and 19 seconds of that footage are presented and analysed in this paper. The footage was watched repeatedly and transcribed using conversation analysis methods (Sacks, 1995). Findings – Through analysing the detailed turn-taking utterances between teachers and children, the orderliness of the co-production of remembering is revealed to demonstrate that each member orients to being in agreement about what actually happened. These episodes of story telling between the teachers and children demonstrate how the teachers encourage the children to tell about their experiences through actively engaging in conversations with them about the earthquake. Originality/value – The conversation analysis approach used in this research was found to be useful in investigating aspects of disasters that the participants themselves remember as important and real. This approach offers a unique insight into understanding how the earthquake event was experienced and reflected on by young children and their teachers, and so can inform future policy and provision in post-disaster situations.

Nutrition prescription to achieve positive outcomes in Chronic Kidney Disease : a systematic review

In Chronic Kidney Disease (CKD), management of diet is important in prevention of disease progression and symptom management, however evidence on nutrition prescription is limited. Recent international CKD guidelines and literature was reviewed to address the following question “What is the appropriate nutrition prescription to achieve positive outcomes in adult patients with chronic kidney disease?” Databases included in the search were Medline and CINAHL using EBSCOhost search engine, Embase and the Cochrane Database of Systematic Reviews published from 2000 to 2009. International guidelines pertaining to nutrition prescription in CKD were also reviewed from 2000 to 2013. Three hundred and eleven papers and eight guidelines were reviewed by three reviewers. Evidence was graded as per the National Health and Medical Research Council of Australia criteria. The evidence from thirty six papers was tabulated under the following headings: protein, weight loss, enteral support, vitamin D, sodium, fat, fibre, oral nutrition supplements, nutrition counselling, including protein and phosphate, nutrients in peritoneal dialysis solution and intradialytic parenteral nutrition, and was compared to international guidelines. While more evidence based studies are warranted, the customary nutrition prescription remains satisfactory with the exception of Vitamin D and phosphate. In these two areas, additional research is urgently needed given the potential of adverse outcomes for the CKD patient.

Reflections for the challenges of tomorrow : where we started

Dietetics in Australia commenced in the 1930’s, inspired by formation of the profession in Britain and the United States. Almost exclusively a female profession, its roots were in nursing management in Britain and food science and home economics in the US, incorporating the principles of food preparation and supply, food and medical science, public health, management, all based on principles of social justice. Victoria employed the first dietitian in 1929, followed by NSW in 1936. Right from the start there was rivalry between the two states over the standard of training, Victoria and later Western Australia, following a college based institutional management focus and NSW, university graduates in science. Both however were committed to the hospital internship as a prelude to safe practice. Soon after, professional associations arose independently in both states, the purpose being to promote appropriate standards in education and to define the level of training required to enter the profession, though it was not until 1950 that a national association was formed. Parallel to the establishment of dietetic training and professional activities, a number of significant public health and medical science initiatives influenced the scope of dietetic practice. The Commonwealth Government commissioned dietary surveys in Australian cities as well as remote communities of Aboriginal and Papuan peoples. Newly discovered components of a healthy diet provided the beginnings of nutrition policy. Pioneer dietitians were trailblazers, with few role models. Their influence ranged across Australia to the United Nations and WHO and we can learn much from their endeavour.

Dietetic practice by Australian and Canadian dietitians with people with Parkinson's disease

Although Parkinson’s disease (PD) is a complex disease for which appropriate nutrition management is important, limited evidence is currently available to support dietetic practice. Existing PD-specific guidelines do not span all phases of the Nutrition Care Process (NCP). This study aimed to document PD-specific nutrition management practice by Australian and Canadian dietitians. DAA members and PEN subscribers were invited to participate in an online survey (late 2011). Eighty-four dietitians responded (79.8% Australian). The majority (70.2%) worked in the clinical setting. Existing non-PD guidelines were used by 52.4% while 53.6% relied on self-initiated literature reviews. Weight loss/malnutrition, protein intake, dysphagia and constipation were common issues in all NCP phases. Respondents also requested more information/evidence for these topics. Malnutrition screening (82.1%) and assessment (85.7%) were routinely performed. One-third did not receive referrals for weight loss for overweight/obesity. Protein intake meeting gender/age recommendations (69.0%), and high energy/high protein diets to manage malnutrition (82.1%) were most commonly used. Constipation management was through high fibre diets (86.9%). Recommendations for spacing of meals and PD medications varied with 34.5% not making recommendations. Nutritional diagnosis (70.2%) and stage of disease (61.9%) guided monitoring frequency. Common outcome measures included appropriate weight change (97.6%) and regular bowel movements (88.1%). With limited PD-specific guidance, dietitians applied best available evidence for other groups with similar issues. Dietitians requested evidence-based guidelines specifically for the nutritional management of PD. Guideline development should focus on those areas reported as commonly encountered. This process can identify the gaps in evidence to guide future research.

Use of a photographic food diary method to document dietary intake in people with Parkinson's Disease

Cognitive impairment and physical disability are common in Parkinson’s disease (PD). As a result diet can be difficult to measure. This study aimed to evaluate the use of a photographic dietary record (PhDR) in people with PD. During a 12-week nutrition intervention study, 19 individuals with PD kept 3-day PhDRs on three occasions using point-and-shoot digital cameras. Details on food items present in the PhDRs and those not photographed were collected retrospectively during an interview. Following the first use of the PhDR method, the photographer completed a questionnaire (n=18). In addition, the quality of the PhDRs was evaluated at each time point. The person with PD was the sole photographer in 56% of the cases, with the remainder by the carer or combination of person with PD and the carer. The camera was rated as easy to use by 89%, keeping a PhDR was considered acceptable by 94% and none would rather use a “pen and paper” method. Eighty-three percent felt confident to use the camera again to record intake. Of the photos captured (n=730), 89% were of adequate quality (items visible, in-focus), while only 21% could be used alone (without interview information) to assess intake. Over the study, 22% of eating/drinking occasions were not photographed. PhDRs were considered an easy and acceptable method to measure intake among individuals with PD and their carers. The majority of PhDRs were of adequate quality, however in order to quantify intake the interview was necessary to obtain sufficient detail and capture missing items.

Innovations in dietary interventions : Use of a photographic food diary method and telephone linked computerized care in chronic disease

Access to dietetic care is important in chronic disease management and innovative technologies assists in this purpose. Photographic dietary records (PhDR) using mobile phones or cameras are valid and convenient for patients. Innovations in providing dietary interventions via telephone and computer can also inform dietetic practice. Three studies are presented. A mobile phone method was validated by comparing energy intake (EI) to a weighed food record and a measure of energy expenditure (EE) obtained using the doubly labelled water technique in 10 adults with T2 diabetes. The level of agreement between mean (±sd) energy intake mobile phone (8.2±1.7 MJ) and weighed record (8.5±1.6 MJ) was high (p=0.392), however EI/EE for both methods gave similar levels of under-reporting (0.69 and 0.72). All subjects preferred using the mobile phone vs. weighed record. Nineteen individuals with Parkinsons disease kept 3-day PhDRs on three occasions using point-and-shoot digital cameras over a 12 week period. The camera was rated as easy to use by 89%, keeping a PhDR was considered acceptable by 94% and none would rather use a “pen and paper” method. Eighty-three percent felt confident to use the camera again to record intake. An interactive, automated telephone system designed to coach people with T2 diabetes to adopt and maintain diabetes self-care behaviours, including nutrition, showed trends for improvements in total fat, saturated fat and vegetable intake of the intervention group compared to control participants over 6 months. Innovative technologies are acceptable to patients with chronic conditions and can be incorporated into dietetic care.

'Haunts of the street bully' : social reform and the Queensland children's playground movement

This article is about the Queensland children's playground movement and its development in Brisbane. It pays particular attention to three Brisbane playgrounds: Neal Macrossan Playground (formerly Paddington Playground); Bedford Playground (formerly Spring Hill Playground); and the Valley Playground, which has since been replaced by a building. The paper pays especial attention to the work of the local children's playground protagonist Mary Josephine Bedford, which will be seen within the context of the international movement.

Consumer perceptions of trade show effectiveness : scale development and validation within a B2C context

Purpose This paper develops and estimates a model to measure consumer perceptions of trade show effectiveness. Design/methodology/approach Data were collected at three separate B2C trade shows. Study 1 (n=47) involved field interviews with data subjected to qualitative item generation and content analysis. Study 2 data (n=147) were subjected to exploratory factor analysis and item-total correlation to identify a preliminary factor structure for the effectiveness construct and to test for reliability. In Study 3 (n=592), confirmatory factor analysis was undertaken to more rigorously test the factor structure and generalise across industries. Validity testing was also performed. Findings A three-dimensional factor structure for assessing consumer visitors’ perceptions of trade show effectiveness was produced incorporating research, operational, and entertainment components. Research limitations/implications Data were collected in Australia and results may not generalise across cultural boundaries. Practical implications The resulting measurement model may be used as a reliable post-hoc diagnostic tool to identify areas of trade show effectiveness where specific performance improvements are needed. Results indicate that exhibitors and organisers of B2C trade shows should consider effectiveness as a multidimensional phenomenon with entertainment, product / industry research, and the facilitation of purchase decision-making processes and problem resolution being key objectives for consumer attendees. These elements of effectiveness should each be addressed by exhibitors and organisers in planning their displays and events. Originality/value This is the first study to provide an empirically valid model for assessing trade show effectiveness from the consumer visitor’s perspective.

Emissions trading and the GATS financial services provisions : a case study of the Australian carbon pricing mechanism

Purpose The purpose of this paper is to determine whether greenhouse gas (GHG) tradeable instruments will be classified as financial products within the scope of the World Trade Organization (WTO) law and to explore the implications of this finding. Design/methodology/approach This purpose is achieved through examination of the units of the Australian Carbon Pricing Mechanism (CPM), namely eligible emissions units. These units are analysed through the lens of the definition of financial products provided in the General Agreement for Trade in Services (the GATS). Findings This paper finds that eligible emissions units will be classified as financial instruments, and therefore the provisions that govern their trade will be regulated by the GATS. Considering this, this paper explores the limitations that are introduced by the Australian legislation on the trade of eligible emissions units. Research limitations/implications This paper is limited in its analysis to the Australian CPM. In order to draw conclusions on the issues raised by this analysis it is necessary to consider the WTO requirements against an operating emissions trading scheme. The Australian CPM presents a contemporary model of an appropriate scheme. Originality/value The findings in this paper are crucial in a GHG constrained society. This is because emissions trading schemes are becoming popular measures for pricing GHG emissions, and for this reason the units that are traded and surrendered for emissions liabilities must be classified appropriately on a global scale. Failing to do this could result in differential treatment that may be contrary to the intentions of important global agreements, such as the WTO covered agreements.

Inter-sector senior leader transitions : experience and outcomes

Purpose The purpose of this article is to explore the experience of senior leaders who move into the public sector from other sectors of the economy, a process we refer to as inter-sector senior leader transitions. This is a little researched area of public sector leadership yet has significant implications for fundamental public sector reform. Design/methodology/approach We employed an interview design to elicit senior leaders' stories of their transition into the public sector. Findings The data suggest that successful senior leader transitions are more likely when a set of conditions is met; (1) the leader transitions into CEO role, rather than levels below CEO, (2) Ministers provide inter-sector transition support, (3) senior leaders develop responses to stress, (4) senior leaders reject high formalization, (5) their change processes focus on building capacity, and (6) senior leaders confront dysfunctional organizational relationships directly. Research limitations/implications The research relies on a relatively small sample. However, access to senior managers at this level can be difficult. Nevertheless, those senior managers who participated were very willing to share their stories. Practical implications If public sector organizations are to realize the value of successful leaders from other sectors, they need to invest in structured processes that facilitate the transition. A laissez-faire approach is not viable given the cost of such transitions. Originality/value The paper focuses on a little researched area of leadership experience which has significant implications for the development and change of the public sector.

New graduate occupational therapists' perceptions of near-misses and mistakes in the workplace

Purpose - The purpose of this paper is to explore the perceptions of near-misses and mistakes among new graduate occupational therapists from Australia and Aotearoa/New Zealand (NZ), and their knowledge of current incident reporting systems. Design/methodology/approach - New graduate occupational therapists in Australia and Aotearoa/NZ in their first year of practice (n=228) participated in an online electronic survey that examined five areas of work preparedness. Near-misses and mistakes was one focus area. Findings - The occurrence and disclosure of practice errors among new graduate occupational therapists are similar between Australian and Aotearoa/NZ participants. Rural location, structured supervision and registration status significantly influenced the perceptions and reporting of practice errors. Structured supervision significantly impacted on reporting procedure knowledge. Current registration status was strongly correlated with perceptions that the workplace encouraged event reporting. Research limitations/ implications - Areas for further investigation include investigating the perceptions and knowledge of practice errors within a broader profession and the need to explore definitional aspects and contextual factors of adverse events that occur in allied health settings. Selection bias may be a factor in this study. Practical implications - Findings have implications for university and workplace structures, such as clinical management, supervision, training about practice errors and reporting mechanisms in allied health. Originality/value - Findings may enable the development of better strategies for detecting, managing and preventing practice errors in the allied health professions.