A preliminary study of open learner model representation formats to support formative assessment

Open learner models provide a way of showing teachers and students the learning progress of a student against expectations. Creating an effective interface to present a learner model is an important part in open learner modeling. Usefulness of the learner model relies on using a clear andeffective representation format to facilitate users’understanding of the information presented. In this paper, wepropose a range of open learner model representation formatsto display students’ learning task status and achievement interms of learning outcomes. We investigate different types ofrepresentation formats and data useful for users to inspect thelearner model. An interface design prototype has been built toprovide potential users with an evaluation platform thatenables investigation of these aspects. The results obtained willallow us to develop a more effective new open learner modelvisualisation tool.

Using online peer assessment tools to provide feedback and promote learning

A set of slides used for the RAP SIG event on 19 Jan 2017

THE SAAF STUDY: evaluation of the Safeguarding Children Assessment and Analysis Framework (SAAF), compared with management as usual, for improving outcomes for children and young people who have experienced, or are at risk of, maltreatment: study protocol for a randomised controlled trial

Background: Serious case reviews and research studies have indicated weaknesses in risk assessments conducted by child protection social workers. Social workers are adept at gathering information but struggle with analysis and assessment of risk. The Department for Education wants to know if the use of a structured decision-making tool can improve child protection assessments of risk.

Methods/design: This multi-site, cluster-randomised trial will assess the effectiveness of the Safeguarding Children Assessment and Analysis Framework (SAAF). This structured decision-making tool aims to improve social workers' assessments of harm, of future risk and parents' capacity to change. The comparison is management as usual.

Inclusion criteria: Children's Services Departments (CSDs) in England willing to make relevant teams available to be randomised, and willing to meet the trial's training and data collection requirements.

Exclusion criteria: CSDs where there were concerns about performance; where a major organisational restructuring was planned or under way; or where other risk assessment tools were in use.

Six CSDs are participating in this study. Social workers in the experimental arm will receive 2 days training in SAAF together with a range of support materials, and access to limited telephone consultation post-training. The primary outcome is child maltreatment. This will be assessed using data collected nationally on two key performance indicators: the first is the number of children in a year who have been subject to a second Child Protection Plan (CPP); the second is the number of re-referrals of children because of related concerns about maltreatment. Secondary outcomes are: i) the quality of assessments judged against a schedule of quality criteria and ii) the relationship between the three assessments required by the structured decision-making tool (level of harm, risk of (re) abuse and prospects for successful intervention).

Discussion: This is the first study to examine the effectiveness of SAAF. It will contribute to a very limited literature on the contribution that structured decision-making tools can make to improving risk assessment and case planning in child protection and on what is involved in their effective implementation.

Peer Assessment and Self-assessment: Effective Learning Tools in Higher Education.

When used appropriately, self- and peer-assessment are very effective learning tools. In the present work, instructor formative assessment and feedback, self-assessment (SA), and peer-assessment (PA) have been compared. During the first part of a semester, the students followed a continuous formative assessment. Subsequently, they were divided into two subgroups based on similar performances. One subgroup performed SAs, and the other followedPAduring the last part of the course. The performances of the two groups in solving problems were compared. Results suggest that PA is a more effective learning tool than SA, and both are more effective than instructor formative assessment. However, a survey that was conducted at the end of the experiment showed higher student confidence in instructor assessment than in PA. The students recognized the usefulness of acting as peer assessors, but believed that SA helped them more than PA.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

Using formative feedback to identify and support first year chemistry students with missing or misconceptions. A Practice Report

Students entering tertiary studies possess a diverse range of prior experiences in their academic preparation for tertiary chemistry so academics need tools to enable them to respond to issues in diversity in conceptual models possessed by entering students. Concept inventories can be used to provide formative feedback to help students identify concepts that they need to address to improve construction of subsequent understanding enabling their learning. Modular, formative learning activities that can be administered inside or outside of class in first year chemistry courses have been developed. These activities address key missing and mis-conceptions possessed by incoming student. Engagement in these learning activities by students and academics will help shift the culture of diagnostic and formative assessment within the tertiary context and address issues around the secondary/tertiary transition. This diagnostic/intervention framework is currently being trialed across five Australian tertiary institutions encompassing a large heterogeneous sample of students.

Positive development and assessment

Purpose – There has been a tendency in sustainability science to be passive. The purpose of this paper is to introduce an alternative positive framework for a more active and direct approach to sustainable design and assessment that de-couples environmental impacts and economic growth. Design/methodology/approach – This paper deconstructs some systemic gaps that are critical to sustainability in built environment management processes and tools, and reframes negative “sustainable” decision making and assessment frameworks into their positive counterparts. In particular, it addresses the omission of ecology, design and ethics in development assessment. Findings – Development can be designed to provide ecological gains and surplus “eco-services,” but assessment tools and processes favor business-as-usual. Despite the tenacity of the dominant paradigm (DP) in sustainable development institutionalized by the Brundtland Report over 25 years ago, these omissions are easily corrected. Research limitations/implications – The limitation is that the author was unable to find exceptions to the omissions cited here in the extensive literature on urban planning and building assessment tools. However, exceptions prove the rule. The implication is that it is not too late for eco-positive retrofitting of cities to increase natural and social capital. The solutions are just as applicable in places like China and India as the USA, as they pay for themselves. Originality/value – Positive development (PD) is a fundamental paradigm shift that reverses the negative models, methods and metrics of the DP of sustainable development. This paper provides an example of how existing “negative” concepts and practices can be converted into positive ones through a PD prism. Through a new form of bio-physical design, development can be a sustainability solution.

Developing formal mathematical assessment for 4-to-8-year-olds

The assessment of children in their years before school and their first years of school has been, traditionally, informal. Further, assessment of children's mathematical skills at this level has been infrequent compared to social, emotional and physical assessments. However, there are contexts where reliable, valid, standardised data from assessment in mathematics are required. This paper outlines the development of two assessment tools for mathematics that were originally developed for such contexts. Item Response Theory (IRT) analyses enabled the construction of assessment forms that address the range of abilities of 4- to 8-year-old children, and provided the scales used for constructing formative and summative reports of achievement. A description of the development of the assessment tools and the IRT analysis that provides the reporting formats are presented together with some research uses of the tools.

Online self and peer assessment in large, multi-campus, multi-cohort contexts

The increasing use of team assignments within higher education is well documented. The driving forces behind this include desires to facilitate reflective and collaborative learning, to develop generic teamwork skills for graduate employment and to reduce the grading workloads of faculty staff. Students however consistently report dissatisfaction when the assessment of team assignments produces a common grade for all team members. Self-andpeer-assessment (SAPA) is presented as a fair, valid and reliable method of producing information about ongoing team processes. This information can provide ongoing feedback to team members and rich formative data to instructors attempting to assess the team process and students’ teamwork skills. This data can also enable individualised summative assessment in dysfunctional teams or situations of uneven team member contributions. Whilst manual SAPA protocols can work effectively for smaller classes, computer-assisted SAPA offers a solution to the problems of large classes. This paper reports on the early stages of an online SAPA tool, originally developed for small classes of architecture students, adapted for use by very large business communication classes comprising up to 1000 students in a semester. This large unit is delivered on four Australian campuses as well as off-campus and in off-shore mode, by up to fourteen instructors at any one time. The paper documents how three researchers from very different backgrounds worked to create their own research team, implement a pilot study, and adapt the online tool, whilst adhering to comparability of assessment constraints and maintaining integrity of research design.

Improving assessment management using tools

This paper firstly explains the importance of assessment management, then introduces two assessment tools currently used in the School of Information Technology at Deakin University. A comparison of assignment marking was conducted after collecting test data from three sets of assignments. The importance of providing detailed marking guides and personalized comments is emphasized and future possible extension to the tools is also discussed at the end of this paper.

Multi-dimensional Review of Uruguay. Volume 1: initial assessment

Includes bibliography.

Bootcamp EMMA MOOC Assessment for learning in practice

At the EDEN Conference an introduction to the EMMA platform will be combined with learning activities relevant to the topic of Formative Assessment. EDEN conference participants will have an opportunity to join the MOOC prior to the conference or at the conference. A range of learning activities will be offered during the training to ensure that all interested have sufficient opportunities to learn.

Bootcamp EMMA MOOC Assessment for learning in practice

Formative assessment or assessment for learning is a relevant theme for teachers and educationalists. Formative assessment is a valuable tool for supporting the learning process. It is applied during learning and offers you more and better opportunities to guide your students. Formative assessment allows for more individual and/or personalised guidance. In this MOOC Assessment for learning in practice we will provide you with theory and guidelines for knowledge construction on the topic of formative assessment while offering support in designing assessments that can be applied as a tool for learning and training of competences. In this MOOC you can learn what formative assessment is, learn to differentiate between summative and formative assessment, and how formative assessment can contribute to the learning of your pupils or students. Design of rubrics, the role and functions of feedback, the use of technology for formative assessment are the topics of the MOOC.

Exploring the mobile structural assessment tool: Concept maps for learning website: Concept Maps for Learning Website

In this paper, we describe how the pathfinder algorithm converts relatedness ratings of concept pairs to concept maps; we also present how this algorithm has been used to develop the Concept Maps for Learning website (www.conceptmapsforlearning.com) based on the principles of effective formative assessment. The pathfinder networks, one of the network representation tools, claim to help more students memorize and recall the relations between concepts than spatial representation tools (such as Multi- Dimensional Scaling). Therefore, the pathfinder networks have been used in various studies on knowledge structures, including identifying students’ misconceptions. To accomplish this, each student’s knowledge map and the expert knowledge map are compared via the pathfinder software, and the differences between these maps are highlighted. After misconceptions are identified, the pathfinder software fails to provide any feedback on these misconceptions. To overcome this weakness, we have been developing a mobile-based concept mapping tool providing visual, textual and remedial feedback (ex. videos, website links and applets) on the concept relations. This information is then placed on the expert concept map, but not on the student’s concept map. Additionally, students are asked to note what they understand from given feedback, and given the opportunity to revise their knowledge maps after receiving various types of feedback.