Improving criteria and feedback in student assessment in law

Spurred on by both the 1987 Pearce Report1 and the general changes to higher education spawned by the “Dawkins revolution” from 1988, there has been much critical self-evaluation leading to profound improvements to the quality of teaching in Australian law schools.2 Despite the changes there are still areas of general law teaching practice which have lagged behind recent developments in our understanding of what constitutes high quality teaching. One such area is assessment criteria and feedback. The project Improving Feedback in Student Assessment in Law is an attempt to remedy this. It aims to produce a manual containing key principles for the design of assessment and the provision of feedback, with practical yet flexible ideas and illustrations which law teachers may adopt or modify. Most of the examples have been developed by teachers at the University of Melbourne Law School. The project was supported in 1996 by a Committee for the Advancement of University Teaching grant and the manual will be published late in 1997.3 This note summarises the core principles which are elaborated further in the manual.

A mixed-methods research approach to the review of competency standards for orthotist/prosthetists in Australia

Aim: The requirement for an allied health workforce is expanding as the global burden of disease increases internationally. To safely meet the demand for an expanded workforce of orthotist/prosthetists in Australia, competency based standards, which are up-to-date and evidence-based, are required. The aims of this study were to determine the minimum level for entry into the orthotic/prosthetic profession; to develop entry level competency standards for the profession; and to validate the developed entry-level competency standards within the profession nationally, using an evidence-based approach. Methods: A mixed-methods research design was applied, using a three-step sequential exploratory design, where step 1 involved collecting and analyzing qualitative data from two focus groups; step 2 involved exploratory instrument development and testing, developing the draft competency standards; and step 3 involved quantitative data collection and analysis – a Delphi survey. In stage 1 (steps 1 and 2), the two focus groups – an expert and a recent graduate group of Australian orthotist/prosthetists – were led by an experienced facilitator, to identify gaps in the current competency standards and then to outline a key purpose, and work roles and tasks for the profession. The resulting domains and activities of the first draft of the competency standards were synthesized using thematic analysis. In stage 2 (step 3), the draft-competency standards were circulated to a purposive sample of the membership of the Australian Orthotic Prosthetic Association, using three rounds of Delphi survey. A project reference group of orthotist/prosthetists reviewed the results of both stages. Results: In stage 1, the expert (n = 10) and the new graduate (n = 8) groups separately identified work roles and tasks, which formed the initial draft of the competency standards. Further drafts were refined and performance criteria added by the project reference group, resulting in the final draft-competency standards. In stage 2, the final draft-competency standards were circulated to 56 members (n = 44 final round) of the Association, who agreed on the key purpose, 6 domains, 18 activities, and 68 performance criteria of the final competency standards. Conclusion: This study outlines a rigorous and evidence-based mixed-methods approach for developing and endorsing professional competency standards, which is representative of the views of the profession of orthotist/prosthetists.

A framework for revising preservice curriculum for nonphysician clinicians: The mozambique experience.

Mozambique, with approximately 0.4 physicians and 4.1 nurses per 10,000 people, has one of the lowest ratios of health care providers to population in the world. To rapidly scale up health care coverage, the Mozambique Ministry of Health has pushed for greater investment in training nonphysician clinicians, Tιcnicos de Medicina (TM). Based on identified gaps in TM clinical performance, the Ministry of Health requested technical assistance from the International Training and Education Center for Health (I-TECH) to revise the two-and-a-half-year preservice curriculum. A six-step process was used to revise the curriculum: (i) Conducting a task analysis, (ii) defining a new curriculum approach and selecting an integrated model of subject and competency-based education, (iii) revising and restructuring the 30-month course schedule to emphasize clinical skills, (iv) developing a detailed syllabus for each course, (v) developing content for each lesson, and (vi) evaluating implementation and integrating feedback for ongoing improvement. In May 2010, the Mozambique Minister of Health approved the revised curriculum, which is currently being implemented in 10 training institutions around the country. Key lessons learned: (i) Detailed assessment of training institutions' strengths and weaknesses should inform curriculum revision. (ii) Establishing a Technical Working Group with respected and motivated clinicians is key to promoting local buy-in and ownership. (iii) Providing ready-to-use didactic material helps to address some challenges commonly found in resource-limited settings. (iv) Comprehensive curriculum revision is an important first step toward improving the quality of training provided to health care providers in developing countries. Other aspects of implementation at training institutions and health care facilities must also be addressed to ensure that providers are adequately trained and equipped to provide quality health care services. This approach to curriculum revision and implementation teaches several key lessons, which may be applicable to preservice training programs in other less developed countries.

Education in J.J. Rousseau: a methodological antecedent of competence based teaching

Taking into account the huge repercussion and influence that J.J. Rousseau has had on modern pedagogy, the recent tercentenary of his birth is a good opportunity to think about his outstanding relevance nowadays. This paper is a theoretical and educative research developed with an analytic and comparative hermeneutical method. The main objective is to show how some concepts of his philosophy of education have a great similarity with certain changes that the present competency based teaching is demanding, so it could be considered its methodological background. In order to achieve this objective this exposure has been divided in three parts. The first part is an analysis of Rousseau's educational theory as developed in the first three books of the Emilio, in which one of the main themes is self experience-based learning, fostering self-sufficiency, curiosity and the motivation for learning. Rousseau proposed as a method the negative education, which requires, among other conditions, a constant monitoring of the learner by the tutor. In the second part, a brief summary of the most relevant changes and characteristics of competency-based teaching is developed, as well as its purpose. The student’s participation and activity are highlighted within their own learning process through the carrying out of tasks. The new educational model involves a radical change in the curriculum, in which it is highlighted the transformation of the methodology used in the classroom as well as the role of the teacher. Finally, the aim of the third part is to offer a comparative synthesis of both proposals grouping the parallelisms found in 4 topics: origin of the two models, its aims, methodology, and change in the teaching roles.

Instrumentos de evaluación utilizados en la formación inicial del profesorado y su coherencia para el desarrollo de competencias profesionales en los estudiantes: visión del alumnado, egresados y profesorado

El objetivo de este trabajo es analizar la frecuencia con que se utilizan diferentes instrumentos y procedimientos de evaluación desde el punto de vista del profesorado, del alumnado y de los egresados en la formación inicial de maestros y profesores especialistas en Educación Física, así como el grado de coherencia –relación- percibido por cada uno de esos grupos implicados entre dichos instrumentos y procedimientos de evaluación y el desarrollo de las competencias profesionales que se pretende adquieran los estudiantes. La muestra objeto de estudio está compuesta por 199 alumnos, 67 egresados y 53 profesores de las titulaciones de Magisterio (especialidad de Educación Física) y Ciencias de la Actividad Física y el Deporte de las universidades de Valladolid, Salamanca y León. Los resultados muestran una clara discrepancia entre los distintos grupos analizados, tanto en lo referido al tipo de instrumentos de evaluación utilizados, como en la valoración del grado de coherencia entre estos y el desarrollo de las competencias profesionales de los futuros maestros.

A Competence-Based Post-Disaster Reconstruction Process: Findings from Sri Lanka

As natural disasters continue to escalate in frequency and magnitude, NGOs are faced with numerous barriers as they attempt to implement post-disaster reconstruction (PDR) projects. In many cases, a lack of competency in key areas leads to a reduction in overall project success. This paper utilizes the competency-based framework of von Meding et al. (2010) as the starting point of its inquiry. In this context, a leading NGO responsible for the implementation of reconstruction and rehabilitation in Sri Lanka following the Asian Tsunami has been investigated in depth using a causal mapping interview procedure with key project staff. The combined barriers within this organization’s PDR operations have been identified and measured and solutions articulated. The study found that within this organization key objectives were to achieve the ‘build back better’ mantra and to effectively plan interventions in advance. The primary barriers to successful reconstruction were identified as the high turnover rate of humanitarian staff and a poor level of communication and co-operation between agencies. An essential strategy employed to combat these barriers is the consideration of staff capabilities, which links us back to competence-based theory. The results are highly valuable in the context of an ongoing wider research study on competence within humanitarian organizations.

Abstract: Psychometric properties of two self-report clinical competence scales for nursing students.

Abstract: Psychometric properties of two self-report clinical competence scales for nursing students.
Background: It is important to assess the clinical competence of nursing students to gauge their professional development and educational needs. This can be measured by self-assessment tools. Anema and McCoy (2010) contended that the currently available measures need further psychometric testing.
Aim: To test the psychometric properties of Nursing Competencies Questionnaire (NCQ) and Self-Efficacy in Clinical Performance (SECP) clinical competence scales.

Method: A non-randomly selected sample of n=248 2nd year nursing students completed NCQ, SECP and demographic questionnaires (June and September 2013). Mokken Scaling Analysis (MSA) was used to test the structural validity and scale properties, convergent and discriminant validity and reliability were subsequently tested.

Results: The NCQ provided evidence of a unidimensional scale which had strong scale scalability coefficients Hs =0.581; but limited evidence of item rankability HT =0.367. MSA undertaken with the SECP scale identified two potential unidimensional scales the SECP28 and SECP7, each with adequate evidence of good/reasonable scalablity psychometric properties as a summed scale but no/very limited evidence of scale rankability (SECP28: Hs = 0.55, HT=0.211; SECP7: Hs = 0.61, HT=0.049). Analysis of between cohort differences and NCQ/ SECP scale scores produced evidence of convergent and discriminant validity and good internal reliability: NCQ α = 0.93, SECP28 α = 0.96, and SECP7 α=0.89.

Discussion: The NCQ was verified to have evidence of reliability and validity; however, as the SECP findings are new, and the sample small, with reference to Straat and colleagues (2014), the SECP results should be interpreted with caution and verified on a second sample.

Conclusions: Measurement of perceived self-competence could inform the development of nursing competence and could start early in a nursing programme. Further testing of the NCQ and SECP scales with larger samples and from different years is indicated.


References:
Anema, M., G and McCoy, JK. (2010) Competency-Based Nursing Education: Guide to Achieving Outstanding Learner Outcomes. New York: Springer.
Straat, JH., van der Ark, LA and Sijtsma, K. (2014) Minimum Sample Size Requirements for Mokken Scale Analysis Educational and Psychological Measurement 74 (5), 809-822.

Acute care teaching in the undergraduate nursing curriculum

AIM: To incorporate basic aspects of acute care into the undergraduate nursing programme by providing an opportunity for the development of knowledge and skills in the early recognition and assessment of deteriorating patients on general hospital wards.
BACKGROUND: Acute care initiatives implemented in the hospital setting to improve the identification and management of 'at risk' patients have focused on the provision of education for trained or qualified staff. However, to ensure student nurses are 'fit to practice' at the point of registration, it has been recommended that acute care theory and skills are incorporated into the undergraduate nursing curriculum.
PRACTICE DEVELOPMENT INITIATIVE: An 'Integrated Nursing Care' module was incorporated into year 3 of the undergraduate nursing programme to introduce students to acute care theory and practice. Module content focuses on the early detection and management of acute deterioration in patients with respiratory, cardiac, neurological or renal insufficiencies. We used a competency-based framework to ensure the application of theory to practice through the use of group seminars. High-fidelity patient-simulated clinical scenarios were a key feature. The United Kingdom Resuscitation Council Intermediate Life Support course is also an important component of the module.
CONCLUSIONS: Incorporating the Integrated Nursing Care module into the undergraduate nursing curriculum provides pre-registration students the opportunity to develop their knowledge and skills in acute care.
RELEVANCE TO CLINICAL PRACTICE: The provision of undergraduate education in care of the acutely ill patient in hospital is essential to improve nurses' competence and confidence in assessing and managing deteriorating patients in general wards at the point of registration.

Nursing Needs Assessment Tool - Users Guide 2002 (PDF 32 KB)

This assessment tool is designed to assess the registered nursing needs of a person needing long-term care. The tool is designed to encapsulate a systematic approach to assessment whilst at the same time embracing professional decision-making that takes place in the relationship between a nurse and another person. For this reason, the tool takes the assessment through a staged approach, moving from a general ‘narrative’ based assessment of ‘domains’ of care need, to a focused assessment of risk and complexity. åÊ

Teachers' perspectives on the implementation of expectations-based curricula /

This qualitative study examines teachers' experiences implementing new standardized curricula in Ontario schools. This new curricula contained several policy changes and an expectations based format which directed what knowledge and skills students were to demonstrate in each subject. This level of specificity of subject-content served to control teachers in relation to curricula; however, data suggested that at the same time, teachers had enormous flexibility in terms of pedagogy. Four secondary teachers who were implementing a Grade 10 course in the 2000-2001 school year participated in the study. The qualitative framework supported the researcher's emphasis on examining the participants' perspectives on the implementation of expectation-based curricula. Data collected included transcripts from interviews conducted with teacher participants and a representative of the Ontario Ministry of Education and Training, field notes, and a research journal. Many of the factors often cited in the literature as influencing implementation practices were found to have affected the participants' experiences of curriculum implementation: time, professional development, and teachers' beliefs, particularly concerning students. In addition, the format of the policy documents proved to both control and free teachers during the implementation process. Participants believed that the number of specific expectations did not provide them an opportunity to add content to the curriculum; at the same time, teachers also noted that the general format of the policy document allowed them to direct instruction to match students' needs and their own teaching preferences. Alignment between teachers' beliefs about education and their understanding of the new curriculum affected the ways in which many participants adapted during the implementation process.

I Believe in me, not OCB: Evaluation of Group Function-Based CBT as a Treatment for Obsessive Compulsive Behaviour in Four Children with HFA

Researchers have conceptualized repetitive behaviours in individuals with Autism Spectrum Disorder (ASD) on a continuum oflower-Ievel, motoric, repetitive behaviours and higher-order, repetitive behaviours that include symptoms ofOCD (Hollander, Wang, Braun, & Marsh, 2009). Although obsessional, ritualistic, and stereotyped behaviours are a core feature of ASD, individuals with ASD frequently experience obsessions and compulsions that meet DSM-IV-TR (American Psychiatric Association, 2000) criteria for Obsessive-Compulsive Disorder (OCD). Given the acknowledged difficulty in differentiating between OCD and Autism-related obsessive-compulsive phenomena, the present study uses the term Obsessive Compulsive Behaviour (OCB) to represent both phenomena. This study used a multiple baseline design across behaviours and ABC designs (Cooper, Heron, & Heward, 2007) to investigate if a 9-week Group Function-Based Cognitive Behavioural Therapy (CBT) decreased OCB in four children (ages 7 - 11 years) with High Functioning Autism (HFA). Key treatment components included traditional CBT components (awareness training, cognitive-behavioural skills training, exposure and response prevention) as well as function-based assessment and intervention. Time series data indicated significant decreases in OCBs. Standardized assessments showed decreases in symptom severity, and increases in quality of life for the participants and their families. Issues regarding symptom presentation, assessment, and treatment of a dually diagnosed child are discussed.

Élaboration d'une grille permettant l'analyse de la qualité de situations d'apprentissage et d'évaluation en lecture auprès d'élèves du parcours de formation axée sur l'emploi

Ce mémoire s’inscrit dans le cadre d’une recherche de développement pédagogique d’objet, destinée à des enseignants du secondaire. L’objectif porte sur la conception, et la validation auprès d’un groupe d’experts, d’une grille visant l’analyse de la qualité de situations d’apprentissage et d’évaluation en lecture (GASAEL) conçues pour des élèves de niveau secondaire en difficulté d’apprentissage, notamment ceux du parcours de formation axée sur l’emploi (PFAE). La problématique expose la nécessité chez les enseignants de trouver des réponses ajustées aux besoins d’apprentissage des élèves ciblés, dans un contexte de programmes élaborés par compétences. La situation d’apprentissage et d’évaluation (SAE) est une instrumentation pertinente, mais demeure sous-utilisée. Le cadre conceptuel, guidé par le modèle de la situation pédagogique de Legendre (2005), assure l’analyse du contexte de la SAE et permet d’identifier ses caractéristiques. Ce cadre est suivi par la méthodologie de l’Analyse de la valeur pédagogique de Roque, Langevin et Riopel (1998) exposant la procédure d’analyse des fonctions du produit, au regard des besoins des utilisateurs. L’innovation de ce produit pédagogique tient en son caractère pragmatique; il propose à la fois l’analyse de la qualité des SAE, et une référence rapide à propos des concepts porteurs d’une SAE, fournis par des exemples ou des explications.

Façons de faire l’évaluation formative d’enseignants de français sénégalais : une analyse de leurs savoirs pratiques en contexte d’effectifs pléthoriques au Lycée.

Cette recherche documente les façons de faire l’évaluation formative d’enseignants de français du Lycée exerçant en contexte de classes pléthoriques au Sénégal. Le choix récent dans ce pays de l’approche par compétences invite à privilégier cette fonction de l’évaluation, au regard de son potentiel pour la progression des apprentissages des élèves (Allal & Mottier Lopez, 2005; Black & Wiliam, 2009; Morrissette, 2010). Cependant, les orientations ministérielles concernant sa mise en œuvre sont très générales, et jusqu’ici, la recherche a laissé dans l’ombre son application en contexte de classes pléthoriques. Puisant au domaine des savoirs pratiques (Schön, 1983) et à une vision interactive et située de l’évaluation formative (Mottier Lopez, 2007; Morrissette, 2010), j’ai conduit une démarche de recherche collaborative auprès de 14 enseignants de français exerçant dans le même Lycée, ponctuée par 6 entretiens de groupe. Un premier registre d’analyse a décrit des façons de faire rattachées à trois dimensions de la pratique de l’évaluation formative: l’analyse du contexte de la pratique, la construction négociée du savoir et la gestion de l’effectif. Un second registre d’analyse de leurs façons de faire en contexte d’«étrangeté culturelle» (Douville, 2002) a permis de conceptualiser leur savoir-évaluer en relation avec leur façon d’interpréter les problèmes qui se posent aux élèves, leur conception de l’erreur et leurs manières de réinventer les modes d’accomplissement traditionnels de l’évaluation ancrés dans la culture scolaire.

Documenter les façons de faire d'enseignants de la 6e année du primaire dans toutes les étapes de la démarche d'évaluation

Le changement de paradigme amené par l'approche par compétences dans le système éducatif québécois au début des années 2000 devait conduire les enseignants à adopter de nouvelles façons de faire prescrites par la Politique en évaluation des apprentissages (MEQ, 2003) qui s’inscrivent dans une évaluation au service de l’apprentissage. Brookhart (2004) pour sa part, considère que la recherche en ce qui concerne les apprentissages et l'évaluation au primaire sont limitées et Boutin (2007) se questionne sur le changement effectif des pratiques considérant les vives critiques suscitées quant à l'implantation de l'approche par compétences dans les écoles québécoises. Cette recherche vise à apporter un éclairage quant aux façons de faire d'enseignants de la 6e année du primaire à toutes les étapes de la démarche d'évaluation en lecture, écriture et mathématique en les situant dans des perspectives nouvelles ou traditionnelles de l'évaluation et en observant si la fréquence des énoncés reliés aux façons de faire diffère selon les caractéristiques personnelles des enseignants. Suivant une méthodologie de type simultanée imbriquée, 55 enseignants ont rempli un questionnaire et 14 d'entre eux ont participé à des entrevues semi-dirigées. L’analyse des données montre que tout au long de la démarche d'évaluation, les façons de faire des enseignants varient, et ce, dans les trois matières: ils utilisent des façons de faire qui sont situées tantôt dans un continuum allant des perspectives traditionnelles aux perspectives nouvelles de l'évaluation comme dans la planification, la collecte de données, l'interprétation et la communication et tantôt dans les nouvelles perspectives de l'évaluation comme dans l'étape du jugement, et ce, selon leurs caractéristiques sociodémographiques. Nous avons ainsi pu catégoriser les enseignants selon le modèle de Schwartz, Bransford et Sears (2005) en distinguant ceux qui sont efficients de ceux qui ne le sont pas, même s'ils innovent. Afin d’améliorer les pratiques des enseignants à chaque étape de la démarche d’évaluation, nous pensons qu'il serait intéressant de valider un tableau synthèse inspiré de Scallon (2004) et l'OCDE (2005) afin de créer une métagrille pour analyser la pertinence et la cohérence des façons de faire des enseignants en cours de cycle dans une optique d’évaluation intégrée à l’apprentissage.