Assessment in experiential learning : the case of a public policy internship

Assessment is a significant issue for learning in the workplace. In some professions there are key indicators of success shared by workplace and academic supervisors alike. Beyond specific professions, however, assessment becomes more diffuse in workplaces that do not have explicit criteria established to judge performance of students in experiential learning. Assessing learning in these workplaces may be associated with methods that rely more upon student self appraisal and workplace supervisor reports. This article reports on the approach used for assessment in a public policy internship program in one Australian university - Deakin University in Victoria. The article argues that assessment, rather than being an add-on or a test of pre-ordained information, is central to the process of learning itself. This means that before students embark upon a policy internship they need to build their critical thinking abilities; i.e. a process of purposeful, self- regulatory judgment. Secondly they need to discuss how to negotiate their tasks in different workplaces and how to produce the criteria to be used in their evaluation.

Delivery and assessment strategies to improve on- and off-campus student performance in structural mechanics

This paper considers the delivery and assessment strategies used in two structural mechanics units at Deakin University, a leader in distance education in Australia. The two units have had unacceptably high rates of student failure. Student perceptions of the delivery method were analysed and an investigation was carried out of the performance of 329 (173 on- and 156 off-campus) students enrolled in the two units. An analysis of the assignment, laboratory and examination marks is presented. Consideration is also given to the total marks. The results show that on-campus students performed better in structural mechanics than their off-campus counterparts. Plots of the distributions of student performance for the three assessment methods are provided (for each unit) and high failure rates are linked to low examination marks. Students tended to perform best in assignments and worst in examinations. Parametric statistical tests show a correlation between the marks obtained in continuous assessment and in examinations, and it is therefore proposed that, in order to improve performance, the students must be encouraged to participate fully in all aspects of the course. Many students were unenthusiastic about laboratory practical sessions and did not think they aided their understanding of the theoretical material. Motivation to participate is often dependent on the perceived relevance of a given task and its contribution to the total mark and, thus, to help motivate students to participate fully in the continuous assessment tasks, the authors propose several changes to the delivery methods, as well as to assessment criteria and marking schemes.

Online self and peer assessment in large, multi-campus, multi-cohort contexts

The increasing use of team assignments within higher education is well documented. The driving forces behind this include desires to facilitate reflective and collaborative learning, to develop generic teamwork skills for graduate employment and to reduce the grading workloads of faculty staff. Students however consistently report dissatisfaction when the assessment of team assignments produces a common grade for all team members. Self-andpeer-assessment (SAPA) is presented as a fair, valid and reliable method of producing information about ongoing team processes. This information can provide ongoing feedback to team members and rich formative data to instructors attempting to assess the team process and students’ teamwork skills. This data can also enable individualised summative assessment in dysfunctional teams or situations of uneven team member contributions. Whilst manual SAPA protocols can work effectively for smaller classes, computer-assisted SAPA offers a solution to the problems of large classes. This paper reports on the early stages of an online SAPA tool, originally developed for small classes of architecture students, adapted for use by very large business communication classes comprising up to 1000 students in a semester. This large unit is delivered on four Australian campuses as well as off-campus and in off-shore mode, by up to fourteen instructors at any one time. The paper documents how three researchers from very different backgrounds worked to create their own research team, implement a pilot study, and adapt the online tool, whilst adhering to comparability of assessment constraints and maintaining integrity of research design.

Fostering deep and elaborative learning and generic(soft) skill development: the strategic use of case studies in accounting education

It is vital that accounting educators take responsibility for the development of students' generic (soft) skills in conjunction with, discipline-specific skills. Research indicates that the typical learning styles of accounting students are not suited to the acquisition of generic skills. In this paper learning theory is used to provide a framework to support the use of case studies as a tool to promote appropriate learning styles and thereby enhance generic skill development. The paper details a number of strategies that may be implemented with case studies to achieve these goals. The implications for accounting educators, which are significant, are discussed.

`Silly, soft and otherwise suspect`: doctoral education as risky business

This article investigates how certain doctoral practices come to count as scandalous and with what effects on universities. To do so, it engages with a number of recent media allegations that relate to doctoral practice in Australia and elsewhere. The analysis of these allegations is developed in terms of three broad categories, namely allegations of silliness in relation to thesis content, allegations of softness in relation to entry, rigour and assessment, and allegations of suspect conduct and/or credentials. The impact of such allegations on university governance is then addressed.

Lack of integration of medical education in Australia: the need for change

The lack of cohesion across health and education sections and national and state jurisdictions is counterproductive to effective national policies in medical education and training. Existing systems in Australia for medical education and training lack coordination, and are under resourced and under pressure. There is a need for a coordinated national approach to assessment of international medical graduates, and for meeting their education and training needs. The links between prevocational and vocational training must be improved. Tensions between workforce planning, education and training can only be resolved if workforce and training agencies work collaboratively. All prevocational positions should be designed and structured to ensure that service, training, teaching and research are appropriately balanced. There is a need for more health education research in Australia.

Changing places : testing the relevance of educational research and teacher education

Frequent reviews of teacher education in recent times seem to indicate that those charged with the responsibility for teacher education have little understanding of the contemporary needs of teachers. Just how does educational research inform teachers in their day-to-day practice and how is its relevance tested? As an educational researcher Professor Shirley Grundy challenges her own notions of 'relevance' by returning from academia to the school system in Western Australia. This experience expanded her understanding of how conceptual frameworks and the realities of schools complemented each other. She reflects on the agenda for change faced by education systems and the ensuing transformation of curriculum planning, pedagogy, assessment and reporting procedures in some schools while not in others.

Her reference to Habermas' work relating to 'system' and 'lifeworld' provided her with an explanation for the social conditions of schooling, while her exploration of discourse theory provides some understanding of the practices related to the exercise of power in school settings. Interchanges such as that experienced by Grundy are to the mutual benefit of both parties.

Equity-focused health impact assessment framework

Background to the Development of the Equity-Focused HIA Framework
The equity focused health impact assessment (EFHIA) framework arises out of a two year research project funded for the most part by the Australian Government’s Public Health Education Research Program (PHERP) Innovations Grants (Round 2) scheme. This project had as its primary objective the development of a framework for health inequalities impact assessment, subsequently renamed equity focused health impact assessment. A partnership between the University of Newcastle, Deakin University and the University of New South Wales (the Project Management Steering Committee) received the funding and the Australasian Collaboration for Health Equity Impact Assessment (ACHEIA) was formed to undertake appropriate background research and to develop, pilot test, modify and disseminate the framework. The work commenced in September 2002 and concluded in October 2004. Part of the funding included a capacity building workshop in August 2004. ACT Health and the Division of Medicine at the John Hunter Hospital, Newcastle, also provided financial support for the project. The August 2004 Workshop was supported by NSW Health. All participants and organisations involved in the project gave extensive in-kind support.
The aims of the workshop were to bring together an international collaboration of multidisciplinary investigators, public health experts, and key senior health managers working in national, state and local settings, to inform the further development of the framework and to provide training in its application. The initial goals of the project were to work collaboratively to develop a strategic framework to assess the health inequalities of public health-related policies, plans, strategies, decisions, programs and services. The EFHIA framework as presented at the August workshop was developed through:
1. an extensive review of the relevant literature
2. formal and informal consultation with members of ACHEIA (the international
reference group), members of the Project Management Steering Committee and
other relevant experts; and
3. testing of the draft EFHIA framework with the 5 case study partners – who applied the draft framework in a range of health settings (see
Acknowledgements).
The result of this work has been the development of an equity focused health impact assessment framework that can be used to determine the unanticipated and systemic health inequities that may exist within the decision making processes or activities of a range of organisations and sectors. The EFHIA framework provides one approach that can be used to assist decision makers to put equity and health on their agenda in a more obvious and systematic way. The framework represents a ‘moment in time’ rather than a definitive statement or ‘toolkit’ on the best way to proceed. Further practice, refinement and adjustment will be needed over many years to consolidate both HIA and EFHIA. As well as this guide to the framework, additional outputs from the project team include:
- A literature review
- A position paper
- A report on the five case studies
- An evaluation report.
With the consent of the Australian Government, a monograph will be made available to workshop participants at the end of October which contains the framework and the appropriate background papers.

The primary health professional education : current models and barriers to participation

Introduction
Continuing Education (CE) for health professionals is a life-long process which endeavours to update or enhance knowledge, refine skills, reinforce professional values and support the delivery of professional practice. It plays a pivotal role in the maintenance of professional competence and in the past decade participation in CE has become an expectation of, rather than an option for, practising health professionals. The time and resources required from organisers and participants in
CE and the need to ensure practical outcomes justifies a review of current models being used for its delivery. This entails an understanding of the purpose of CE, a consideration of how it should be delivered, and the role played by assessment in achieving the goals of CE.
Aim of Report
The overall aim of this study is to identify important considerations and subsequently make recommendations for the development of an ideal model(s) of CE for community pharmacy.
Goals of Report
1. Define CE and its role.
2. Identify and assess current CE delivery models.
3. Examine the current status of continuing education and registration requirements for pharmacists.
4. Identify barriers to participation in CE.
5. Identify components and considerations for developing a model of CE delivery.
Methods
The following methods were employed for this project:
1. Literature review
A number of electronic databases were systematically searched in order to profile current trends and concepts in CE. CE structures currently in use were investigated by directly accessing the websites of appropriate associations.
2. Stakeholder interviews
A series of semi-structured interviews were completed with stakeholders from CE delivery organisations across a range of professions including pharmacy.
3. Community pharmacy focus groups
A series of focus group teleconferences were held with groups of pharmacists thought to have distinct CE needs: experienced pharmacists (qualified more than 5 years), recently-qualified pharmacists (5 years or less), rural/remote pharmacists, and pharmacists with specialist training
needs (such as Home Medication Reviews). These focus groups asked about participants’ experiences and opinions in relation to many aspects of CE including its delivery and its assessment.

Flexible delivery and student attrition in the vocational education and training sector: case studies of students who dropped out

Government policy in Australia is increasingly encouraging training organisations in the Vocational Education and Training (VET) sector to adopt flexible delivery approaches. This policy shift is supported by key VET stakeholders including Industry Training Advisory Boards. A recurring theme in VET policy documents is an apparent confidence that flexible delivery can meet the diverse needs of individual learners while at the same time providing cost savings. Yet evidence is emerging that Australian VET learners are not typically ready for flexible delivery, and this lack of readiness is reflected in high attrition rates and low pass rates in many flexibly delivered courses. One research project found that over 70% of learners in the Australian VET sector do not have the learning capabilities required to be ready for flexible delivery. A recent review of the module outcomes achieved by VET students nationally found that students studying by external/correspondence and self-paced unscheduled modes had lower module completion rates than students studying by other delivery strategies.

Research on student progress in flexible delivery within the Australian VET sector has largely been quantitative. That research provides useful statistical data on completion and attrition rates for various modes of delivery, but does not explore the reasons underlying the high attrition rates found in flexible delivery. The qualitative research that is available tends to focus on students who successfully complete their courses, not on those who withdraw. As a result, the Australian literature on flexible delivery in the VET sector is lacking in-depth qualitative information about students who enrol in courses but do not complete. In comparison, the broader literature on distance education and flexible delivery in other educational sectors offers some useful insights into student attrition, and can be can be used to inform research into attrition within the Australian VET sector.

This paper reports on aspects of a research project that followed up six adult learners who enrolled in VET courses but who either failed assessment or withdrew. The research project presented the students’ stories in the form of narrative case studies, focussing on the detailed examination of the barriers that each student experienced, and analysing these barriers in relation to issues raised in the literature. This paper reports on two particular themes that emerged from that research project. The literature on distance education and flexible delivery argues that:


· student dropout is often not determined by a single factor, but by the interaction of a number of factors that build up over time;

· students who experience difficulties when studying by flexible delivery can often be reluctant to access the support that is available to them.

This paper uses these themes as a point of reference in presenting the stories of some of the students who participated in the research project.

Teacher or trainer? The shaping of teacher identity in adult education contexts

In response to changing government funding priorities there has been a shift away from the provision of needs based language and personal development courses for adults in community based contexts towards the delivery of vocational education. Much vocational education is characterized by competency-based curriculum and outcomes influenced by the needs of the current labour market as well as economic driven initiatives such as competitive tendering for short-term course funding. These trends have resulted in changes to the nature of curriculum, assessment, and the purpose and nature of the delivery of courses to adult learners. In turn, these changes affect the ways in which teachers see themselves and carry out their roles as professionals. This paper explores the ways in which the current discourses of vocational education shape teacher identities across a variety of vocational education contexts and the ways in which teacher identities are played out through training and teaching practices.

Issues arising from the implementation of a comprehensive assessment approach in a large, multi-modal Bachelor of Commerce Unit

Recent literature in higher education argues university assessment has been too narrow and hasn’t adequately reflected the quality, breadth and depth of students’ learning. Research shows students often prioritise and learn what they need to know for formal, graded assessment and disregard other academic content seen as less relevant to those requirements. The predominance of essays and examinations has therefore tended to constrain learning. The case for a more comprehensive approach has been clearly articulated. So what happens when staff take up the unique challenge of designing fair and uniform assessment for a large, core, multi-modal, multi-campus unit offered nationally and internationally?
When developing an undergraduate Bachelor of Commerce unit at Deakin University, staff considered the most appropriate ways to assess a range of conceptual understandings and communication skills. This resulted in the mapping and adoption of a comprehensive approach incorporating teacher, peer, and self-assessment aspects, individual and group work, oral and written presentations, and the use of portfolios and journals. Particular practices were adopted to control workloads, ensure fairness in marking, and overcome some problems generally associated with group work. When implementing the approach, practical issues arose that demanded adjustments. This paper details the approach taken, outlines research activities, and discusses the practical implications of issues that arose.

Optimising the impact of education for community pharmacy in Australia

Aims & rationale/Objectives : This paper examines the extent to which different models of community pharmacist continuing education (CE) are evidence-based. It also describes the impact of varying education models on attendance and attitudes within the profession.

Methods : A literature review was conducted to establish principles that should be applied to health professional education, and pharmacy in particular. Interviews were conducted with representatives from four organisations involved in the education of pharmacists to understand their current models. Four focus groups were held with community pharmacists to understand their educational experiences and attitudes.

Principal findings : The purpose of CE is to improve the clinical performance of health practitioners. Literature examining outcomes from CE underlines the importance of adult learning principles. Focus groups supported the view that consideration of these principles is beneficial. These principles, including problem-based learning, clinical applicability, relevance, and active involvement in the learning process, are currently incorporated into educational models to varying extents. Access problems such as cost, distance, insufficient flexibility in delivery, and poor promotion of educational opportunities prevent many pharmacists from taking responsibility for their own learning. A lack of appropriate assessment by some registering authorities is counterproductive to achieving CE outcomes in clinical practice. Participants already engaged in continuing professional development (CPD) agreed with the principles of its introduction.

Discussion : Optimising outcomes from CE requires considerable input from numerous stakeholders. The recent introduction of mandatory pharmacist CPD across Australia should encourage an individual focus on learning outcomes. Focus group participants are likely to be education enthusiasts and may not represent the views of the entire profession.

Implications : This study identifies the need for a system-wide approach for achieving outcomes from CE. It is therefore advisable that a coordinated strategy be developed by all stakeholders for education delivery so as to optimise the impact of CE.

The assessment of formal management development : a method, a baseline and the need to incorporate learning styles

Purpose – To assess the impact of a course on the development of management competencies for students over the duration of the course, by examining changes in the competencies of the students. The potential impact of student learning styles was also examined in the study.

Design/methodology/approach – A “Pre, Post, Then” design and a combination of paired sample t-tests, and alpha and beta change statistics were used to examine the change in competencies over time.

Findings – This study found that even a standard subject design had a significant and substantial impact on the management competence development of the students.

Practical implications – From the perspective of students-as-products, the base line provided here allows for the testing of alternative designs of educational systems, whereby alternative designs have to prove a benefit above and beyond that of a basic teaching system. Therefore, the present study should assist the field of management development to create a range of design options.

Originality/value – This study applies a powerful and under-utilised research method to provide a base line of the amount of management development that can occur in a typical formal management course, but is unique in that it includes the impact of the students’ learning styles.