The AIFS evaluation of the 2006 family law reforms : a summary

In 2006, the Australian Government, through the Attorney- General's Department (AGD) and the Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA), commissioned the Australian Institute of Family Studies (AIFS) to undertake an evaluation of the impact of the 2006 changes to the family law system: Evaluation of the 2006 Family Law Reforms (Kaspiew et al., 2009) (the Evaluation). This article provides a summary of the key findings of the Evaluation.

Evaluation of the use of Assessment Centre methodology to enhance development planning, work placement outcomes and work readiness for postgraduate students : a pilot

Background: Placements as a form of Work Integrated Learning are widely recognised for the positive impact they have on improving student employability and work readiness. Students can maximise strengths, improve areas of weakness, and develop a strong understanding of the requirements of their chosen field within the confines of a well monitored and rich learning environment. Assessment Centres (ACs) are commonly used in corporate settings for recruitment, selection and more recently to provide developmental feedback to participants. Based on a recent literature review, the present the present project evaluates the application of AC methodology as a developmental tool within the placement milieu. The review, which is also included the current conference proceedings details the benefits of utilising the AC process forming the impetus for the present pilot (Sturre; von Treuer & Keele 2010).
Aims: The primary aim of the paper was to evaluate the application of AC methodology as a tool for measuring and subsequently enhancing professional competencies in a sample of postgraduate students in organisational psychology (n=15).
Method: A longitudinal design was utilised with numerous evaluation points from placement stakeholders. This paper presents the first wave of findings. Students undertook a range of activities, including an in-tray exercise, role play, written report, leaderless group discussion and a personality assessment. Comprehensive feedback was provided by organisational psychologists who also fulfil the role of placement co-ordinators. With the assistance of Placement Co-ordinators, students prepared development plans relating to the competencies identified as requiring development. These plans were to be addressed and progress monitored during consecutive placements.
Results: Initial perceptions gathered from students regarding the AC process were very encouraging. Performance
evaluations collected to date, as measured by behaviourally based ratings scales completed by the students themselves and their workplace supervisors illustrate the positive effect of this methodology. The rigour and comprehensive techniques offered by the methodology enabled students to focus on and improve areas identified for development.
Conclusions: It is important to note that the present design formed a pilot study and as mentioned was undertaken with a limited sample. Future implementation is planned with larger samples, enabling a more comprehensive analysis of the methodology. Nevertheless, the methodology appears to provide a much needed strategy for the assessment and ongoing development of students prior to and during work placements. The application provides early intervention enabling students to address development needs with input from both university and organisational stakeholders based on an established, standardised process.

A study on the accuracy of frequency measures and its impact on knowledge discovery in single sequences

In knowledge discovery in single sequences, different results could be discovered from the same sequence when different frequency measures are adopted. It is natural to raise such questions as (1) do these frequency measures reflect actual frequencies accurately? (2) what impacts do frequency measures have on discovered knowledge? (3) are discovered results accurate and reliable? and (4) which measures are appropriate for reflecting frequencies accurately? In this paper, taking three major factors (anti-monotonicity, maximum-frequency and window-width restriction) into account, we identify inaccuracies inherent in seven existing frequency measures, and investigate their impacts on the soundness and completeness of two kinds of knowledge, frequent episodes and episode rules, discovered from single sequences. In order to obtain more accurate frequencies and knowledge, we provide three recommendations for defining appropriate frequency measures. Following the recommendations, we introduce a more appropriate frequency measure. Empirical evaluation reveals the inaccuracies and verifies our findings. 

Intervention for depression among palliative care patients and their families : a study protocol for evaluation of a training program for professional care staff

Background: Clinical depression is highly prevalent yet under-detected and under-treated in palliative care settings and is associated with a number of adverse medical and psychological outcomes for patients and their family members. This article presents a study protocol to evaluate a training intervention for non-physician palliative care staff to improve the recognition of depression and provide support for depressed patients and their family members. Details of the hypotheses and expected outcomes, study design, training program development and evaluation measures are described.
Methods and Design: A randomised controlled trial will be implemented across two palliative care services to evaluate the “Training program for professional carers to recognise and manage depression in palliative care settings”. Pre-, post- and three-month follow-up data will be collected to assess: the impact of the training on the knowledge, attitudes, self-efficacy and perceived barriers of palliative care staff when working with depression; referral rates for depression; and changes to staff practices. Quantitative and qualitative methods, in the form of self-report questionnaires and interviews with staff and family members, will be used to evaluate the effectiveness of the intervention.
Discussion: This study will determine the effectiveness of an intervention that aims to respond to the urgent need for innovative programs to target depression in the palliative care setting. The expected outcome of this study is the validation of an evidence-based training program to improve staff recognition and appropriate referrals for depression, as well as improve psychosocial support for depressed patients and their family members.

Implementation and evaluation of brief depression and anxiety screening in clients contacting a drug and alcohol service

Introduction and Aims. High prevalence mental health (HPMH) comorbidity is common in clients seeking alcohol and other drug (AOD) treatment yet can remain undetected. Although research has reported on the introduction of screening into AOD services, little research has reported on the processes surrounding the introduction or evaluated its effectiveness.This study reports on the implementation and evaluation of brief anxiety and depression screening within a specialised, publicly funded AOD service in South-East Victoria.
Design and Methods. Study one examined the implementation of standardised HPMH screening with 114 adult clients (Mean age = 35.49, SD = 9.53; 64% male) telephoning an AOD service over a 5 week period. Measures included severity of HPMH problems,AOD use, care plans and referrals. Study two used semistructured interviews with nine staff/managers to evaluate the effectiveness of screening and its impact on service delivery.
Results. Ninety-four per cent of clients were identified at risk of anxiety or depression. Most care plans incorporated counselling, and concurrent referrals commonly involved a general practitioner. Staff and management found systematic screening increased identification and understanding of comorbid issues and enhanced client interaction but impacted on resource requirements.
Discussion and Conclusions. Most AOD treatment seekers were identified HPMH comorbid and care plans generally included counselling.Adjunctive referrals were more common for severely depressed clients. Screening was effective and enhanced client rapport.Evaluations revealed low confidence in treating HPMH issues in-house.Training may increase worker confidence in managing mental health interventions with subclinical cases, enhancing services’ ability to move towards dual diagnosis capability.

Factors influencing quality of behaviour support plans and the impact of plan quality on restrictive intervention use

Background: The quality and effectiveness of the support provided to people with disability who show challenging behaviour can be influenced by the design and content of their behaviour support plans (BSPs). This study examined some of the factors that might influence the quality of behaviour support plans and the impact of quality of BSPs on the use of restrictive intervention.

Method: An audit of the quality of a sample of BSPs submitted to the Senior Practitioner in Victoria in the years 2009 and 2010 was conducted using the Behavior Support Plan Quality Evaluation, 2nd Edition (BSP-QE II).

Results: Factors found to positively influence quality of BSPs included: involvement of behaviour consultants and involvement of clinicians from the Office of the Senior Practitioner (Office). Overall quality of plans was also negatively related to restrictive intervention use over time.

Conclusions: The findings support the need for behaviour intervention and provision of clinical support. The findings also provide tentative support for the notion that the inclusion of evidence-based quality components into behaviour support plan formats may reduce the use of restrictive interventions.

Do healthy cities work? A logic of method for assessing impact and outcome of healthy cities

In this article, we discuss an appropriate methodology for assessing complex urban programs such as the WHO European Healthy Cities Network. The basic tenets and parameters for this project are reviewed, and situated in the broader urban health tradition. This leads to a delineation of the types of questions researchers can address when looking at a complex urban health program. Such questions reach appropriately beyond traditional public health concepts involving proximal and distal determinants of health (and associated upstream, midstream, and downstream rhetoric). Espousing a multi-level, reciprocal pathways perspective on Healthy Cities research, we also adopt a distinction between impacts and outcomes of Healthy Cities. The former are value driven, the latter intervention-driven. These approaches lead to the acknowledgment of a logic of method that includes situational and contextual appreciation of unique Healthy City experiences in a Realist Evaluation paradigm. The article concludes with a reflection of evaluation and assessment procedures applied to Phase IV (2003-2008) of the WHO European Healthy Cities Network and an interpretation of response rates to the range of methods that have been adopted.

The delivery of a telephone coaching programme to people with type 2 diabetes by practice nurses in Victoria, Australia : a qualitative evaluation

Aim.  To evaluate telephone coaching undertaken by practice nurses in a randomised controlled trial of self-management support for people with type 2 diabetes.

Background.  Qualitative evaluation of the processes that take place in randomised controlled trials has the advantage of providing information on those variables that contribute to the success or failure of the randomised controlled trial. This additional information can be used to improve or modify chronic disease management programme designs.

Methods.  Grounded theory was used to analyse transcriptions of telephone coaching sessions between practice nurses and patient participants in the randomised controlled trial.

Findings.   Analysis of transcriptions found that patient participants had complex multiple medical conditions to manage, as well as maintaining their daily lives. Two approaches to working with this complexity by practice nurses emerged. We characterised one as ‘treat to target’ and the other as ‘personalised care’. While each approach shapes identities available to patients within the relationship with the practice nurse, the impact or effectiveness of these approaches on outcomes has yet to be reported.

Conclusions.  Telephone coaching takes place in complex social contexts as well as complex medical conditions. People with type 2 diabetes must manage their diabetes care and their care of other conditions within their social contexts. This means a constant negotiation of priorities.

Relevance to clinical practice.  Awareness of how health professional support for patients’ self-management becomes a relationship and element of the negotiated identity of patients is important in adapting clinical guideline-based protocols to achieving targets in the management of chronic illness.

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In-depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface-level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.

Performance evaluation of different ultrafiltration membranes for the reclamation and reuse of secondary effluent

This study investigates and compares the performance of two different types of ultrafiltration (UF) membranes in the recovery of water from secondary treated wastewater. Filtration experiments were carried out on a pilot scale cross-flow unit using synthetic wastewater similar to the quality of secondary treated wastewater by varying the operating parameters such as transmembrane pressure (TMP), feed composition and membrane configuration. The filtration experiments demonstrated that the flux recovery through spiral polymeric UF membrane was more sensitive to the variation in TMP compared to the tubular ceramic UF membrane over the range of TMP studied. The resistance in series model was used for the evaluation of the resistance to the permeate flux. The fouling resistance, particularly irreversible resistance compared to reversible resistance plays a major role in the total resistance for the tubular ceramic membrane. In contrast clean membrane resistance is the major contributor for the total resistance of the spiral polymeric membrane. Finally, the effectiveness of the filtration treatment was determined by evaluating the rejection coefficients for various pollution indices of the wastewater. Significant differences in the performance of the membrane types were observed which are likely to impact on the selection, operation and maintenance of the membrane system.

The inclusion of ‘then-test’ questions in post-test questionnaires alters post-test responses: a randomized study of bias in health program evaluation

Objectives Program evaluations are frequently based on ‘then-test’ data, i.e., pre-test collected in retrospect. While the application of the then-test has practical advantages, little is known about the validity of then-test data. Because of the collection of then-test in close proximity to post-test questions, this study was aimed at exploring whether the presence of then-test questions in post-test questionnaires influenced subjects’ responses to post-test.
Patients and methods To test the influence of then-test questions, we designed a randomized three-group study in the context of chronic disease self-management programs. Interventions had comparable goals and philosophies, and all 949 study participants filled out identical Health Education Impact Questionnaires (heiQ) at pre-test. At post-test, participants were then randomized to one of the following three groups: Group A responded to post-test questions only (n = 331); Group B filled out transition questions in addition to post-test (n = 304); and Group C filled out then-test questions in addition to post-test (n = 314).
Results Significant post-test differences were found in six of eight heiQ scales, with respondents who filled out then test questions reporting significantly higher post-test scores than respondents of the other groups.
Conclusions This study provides evidence that the inclusion of then-test questions alters post-test responses,
suggesting that change scores based on then-test data be interpreted with care.

Promoting active transport to school : evaluation of the Ride2School program

This study involved an evaluation of the Ride2School program, promoting active transport to school. The study found inconsistent evidence of a program impact, and identified the importance of addressing barriers to active transport among individual schools, parents and students; and building the capacity of schools to implement active transport initiatives.

Estimating the impact of mandatory folic acid fortification on the folic acid intake of Australian women of childbearing age

Objective: The primary aim of this study was to estimate the impact of mandatory folic acid (FA) fortification of bread-making flour on the FA intake of Australian women of childbearing age (16-44 years). The secondary objective was to investigate the relationship between estimated FA intake and socio-economic status (SES) and age.

Method: Dietary modelling was used to estimate FA intake under four mandatory fortification scenarios – no supplement use, supplement use unrelated to FA intake, supplement use only among the highest consumers of bread, and increased supplement use. Data were obtained from the 1995 National Nutrition Survey for food intake patterns, the 2007 Victorian Population Health Survey for FA supplement use, and a marketplace survey.

Results: It is estimated that the National Health and Medical Research Council (NHMRC) recommendation for an additional 400 mg/day FA will be achieved by a minimum of 3.9, 25.4, 21.7 and 30% of the target population under scenarios 1-4, respectively. The FA upper level of intake is exceeded by a maximum of 0.1, 1.7, 6.1 and 4.1% of the target population for scenarios 1-4, respectively.

Conclusions: Mandatory FA fortification is not sufficient for the NHMRC recommendations for minimum and maximum intakes to be met by all of the target population under a number of plausible behaviour scenarios.

Implications: Targeted nutrition education campaigns are needed for SES and age sub-groups and research of this nature should be extended to other population groups. Monitoring and evaluation of this policy will be important to ensure appropriate FA intake.

Growing community : the impact of the Stephanie Alexander kitchen garden program on the social and learning environment in primary schools

This article presents results from a mixed-method evaluation of a structured cooking and gardening program in Australian primary schools, focusing on program impacts on the social and learning environment of the school. In particular, we address the Stephanie Alexander Kitchen Garden Program objective of providing a pleasurable experience that has a positive impact on student engagement, social connections, and confidence within and beyond the school gates. Primary evidence for the research question came from qualitative data collected from students, parents, teachers, volunteers, school principals, and specialist staff through interviews, focus groups, and participant observations. This was supported by analyses of quantitative data on child quality of life, cooperative behaviors, teacher perceptions of the school environment, and school-level educational outcome and absenteeism data. Results showed that some of the program attributes valued most highly by study participants included increased student engagement and confidence, opportunities for experiential and integrated learning, teamwork, building social skills, and connections and links between schools and their communities. In this analysis, quantitative findings failed to support findings from the primary analysis. Limitations as well as benefits of a mixed-methods approach to evaluation of complex community interventions are discussed.

The CDM-Net project : the development, implementation and evaluation of a broadband-based network for managing chronic disease

Background. In Australia most chronic disease management is funded by Medicare Australia through General Practitioner Management Plans (GPMPs) and Team Care Arrangements (TCAs). Identified barriers may be reduced effectively using a broadband-based network known as the Chronic Disease Management Service (CDMS).

Aims. To measure the uptake and adherence to CDMS, test CDMS, and assess the adherence of health providers and patients to GPMPs and TCAs generated through CDMS.

Methods. A single cohort before and after study.

Results. GPMPs and TCAs increased. There was no change to prescribed medicines or psychological quality of life. Attendance at allied health professionals increased, but decreased at pharmacies. Overall satisfaction with CDMS was high among GPs, allied health professionals, and patients.

Conclusion. This study demonstrates proof of concept, but replication or continuation of the study is desirable to enable the impact of CDMS on diabetes outcomes to be determined.