Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.

Characterising creativity : a holistic methodology for the assessment of creative works

This paper reports on the early stages of a design experiment in educational assessment that challenges the dichotomous legacy evident in many assessment activities. Combining social networking technologies with the sociology of education the paper proposes that assessment activities are best understood as a negotiable field of exchange. In this design experiment students, peers and experts engage in explicit, "front-end" assessment (Wyatt-Smith, 2008) to translate holistic judgments into institutional, and potentiality economic capital without adhering to long lists of pre-set criteria. This approach invites participants to use social networking technologies to judge creative works using scatter graphs, keywords and tag clouds. In doing so assessors will refine their evaluative expertise and negotiate the characteristics of creative works from which criteria will emerge (Sadler, 2008). The real-time advantages of web-based technologies will aggregate, externalise and democratise this transparent method of assessment for most, if not all, creative works that can be represented in a digital format.

Trouble and autoethnography in assessment genre : a case for postnational design in online internationalized pedagogy

This chapter is interested in the difference between local places with implicit codes and more global spaces with explicit directions, through the case study of the design and conduct of assessment in an online internationalized MBA unit. Online learning is understood to offer new ways of belonging in 'postnational' communities less reliant on locality for their frames of reference. This study reports and analyses firstly a series of troubles which erupted over the international students' desire for more explication of the desired genre for their assessment task. Then it analyses the different, 'autoethnographic' genre structure that emerged when students started to acknowledge the diverse backgrounds within the class.The chapter then offers practical considerations for the design of online internationalized programs.

A medical data reliability assessment model

There is currently a strong focus worldwide on the potential of large-scale Electronic Health Record (EHR) systems to cut costs and improve patient outcomes through increased efficiency. This is accomplished by aggregating medical data from isolated Electronic Medical Record databases maintained by different healthcare providers. Concerns about the privacy and reliability of Electronic Health Records are crucial to healthcare service consumers. Traditional security mechanisms are designed to satisfy confidentiality, integrity, and availability requirements, but they fail to provide a measurement tool for data reliability from a data entry perspective. In this paper, we introduce a Medical Data Reliability Assessment (MDRA) service model to assess the reliability of medical data by evaluating the trustworthiness of its sources, usually the healthcare provider which created the data and the medical practitioner who diagnosed the patient and authorised entry of this data into the patient’s medical record. The result is then expressed by manipulating health record metadata to alert medical practitioners relying on the information to possible reliability problems.

A time-variant medical data trustworthiness assessment model

Electronic Health Record (EHR) systems are being introduced to overcome the limitations associated with paper-based and isolated Electronic Medical Record (EMR) systems. This is accomplished by aggregating medical data and consolidating them in one digital repository. Though an EHR system provides obvious functional benefits, there is a growing concern about the privacy and reliability (trustworthiness) of Electronic Health Records. Security requirements such as confidentiality, integrity, and availability can be satisfied by traditional hard security mechanisms. However, measuring data trustworthiness from the perspective of data entry is an issue that cannot be solved with traditional mechanisms, especially since degrees of trust change over time. In this paper, we introduce a Time-variant Medical Data Trustworthiness (TMDT) assessment model to evaluate the trustworthiness of medical data by evaluating the trustworthiness of its sources, namely the healthcare organisation where the data was created and the medical practitioner who diagnosed the patient and authorised entry of this data into the patient’s medical record, with respect to a certain period of time. The result can then be used by the EHR system to manipulate health record metadata to alert medical practitioners relying on the information to possible reliability problems.

Selected elements of housing affordability impacting otherwise potentially sustainable communities

It is widely held that strong relationships exist between housing, economic status, and well being. Therefore, recent events emerging from the United States, culminating in widespread housing stock surpluses in that country and others, threaten to destabilise many aspects related to individuals and community. However, despite global impact, the position of housing demand and supply is not consistent. The Australian position provides a strong contrast whereby continued strong housing demand generally remains a critical issue affecting the socio-economic landscape. Underpinned by strong levels of immigration, and further buoyed by sustained historically low interest rates, increasing income levels, and increased government assistance for first home buyers, this strong housing demand ensures elements related to housing affordability continue to gain prominence. A significant, but less visible factor impacting housing affordability – particularly new housing development – relates to holding costs. These costs are in many ways “hidden” and cannot always be easily identified. Although it is only one contributor, the nature and extent of its impact requires elucidation. In its simplest form, it commences with a calculation of the interest or opportunity cost of land holding. However, there is significantly more complexity for major new developments - particularly greenfield development. Analysis suggests that even small shifts in primary factors impacting holding costs can appreciably affect housing affordability. Those factors of greatest significance not only include interest rates and the rate of inflation, but even less apparent factors such as the regulatory assessment period. These are not just theoretical concepts but real, measurable price drivers. Ultimately, the real impact is felt by the one market segment whom can typically least afford it – new home, first home buyers. They can be easily pushed out of affordability. This paper suggests the stability and sustainability of growing, new communities require this problem to be acknowledged and accurately identified if the well being of such communities is to be achieved.

Incorporating an economic model in the health impact assessment approach

The current policy decision making in Australia regarding non-health public investments (for example, transport/housing/social welfare programmes) does not quantify health benefits and costs systematically. To address this knowledge gap, this study proposes an economic model for quantifying health impacts of public policies in terms of dollar value. The intention is to enable policy-makers in conducting economic evaluation of health effects of non-health policies and in implementing policies those reduce health inequalities as well as enhance positive health gains of the target population. Health Impact Assessment (HIA) provides an appropriate framework for this study since HIA assesses the beneficial and adverse effects of a programme/policy on public health and on health inequalities through the distribution of those effects. However, HIA usually tries to influence the decision making process using its scientific findings, mostly epidemiological and toxicological evidence. In reality, this evidence can not establish causal links between policy and health impacts since it can not explain how an individual or a community reacts to changing circumstances. The proposed economic model addresses this health-policy linkage using a consumer choice approach that can explain changes in group and individual behaviour in a given economic set up. The economic model suggested in this paper links epidemiological findings with economic analysis to estimate the health costs and benefits of public investment policies. That is, estimating dollar impacts when health status of the exposed population group changes by public programmes – for example, transport initiatives to reduce congestion by building new roads/ highways/ tunnels etc. or by imposing congestion taxes. For policy evaluation purposes, the model is incorporated in the HIA framework by establishing association among identified factors, which drive changes in the behaviour of target population group and in turn, in the health outcomes. The economic variables identified to estimate the health inequality and health costs are levels of income, unemployment, education, age groups, disadvantaged population groups, mortality/morbidity etc. However, though the model validation using case studies and/or available database from Australian non-health policy (say, transport) arena is in the future tasks agenda, it is beyond the scope of this current paper.

The Children's Depression Scale : assessment of factor structure with data from a normal population

A study investigated the reliability and construct validity of the Children's Depression Scale. The revised subscales were shown to have strong construct and face validity and high reliability.

Assessment in tertiary mathematics

There is a growing consensus among many educators that the goals of teaching and learning mathematics are to help students solve real-life problems, participate intelligently in daily affairs, and prepare them for jobs (Gardiner, 1994; Roeber, 1995). These goals suggest that the role of routine procedural skills should be diminished while more emphasis ought to be placed on learners gaining conceptual insights and analytical skills that appear essential in real-life mathematical problem solving (Schoenfeld, 1993; Stenmark, 1989).

Implementation of an integrated, technology-based, discovery mode assessment item involving an incubation period to enhance learning outcomes for engineering maths students

In this paper we discuss our current efforts to develop and implement an exploratory, discovery mode assessment item into the total learning and assessment profile for a target group of about 100 second level engineering mathematics students. The assessment item under development is composed of 2 parts, namely, a set of "pre-lab" homework problems (which focus on relevant prior mathematical knowledge, concepts and skills), and complementary computing laboratory exercises which are undertaken within a fixed (1 hour) time frame. In particular, the computing exercises exploit the algebraic manipulation and visualisation capabilities of the symbolic algebra package MAPLE, with the aim of promoting understanding of certain mathematical concepts and skills via visual and intuitive reasoning, rather than a formal or rigorous approach. The assessment task we are developing is aimed at providing students with a significant learning experience, in addition to providing feedback on their individual knowledge and skills. To this end, a noteworthy feature of the scheme is that marks awarded for the laboratory work are primarily based on the extent to which reflective, critical thinking is demonstrated, rather than the amount of CBE-style tasks completed by the student within the allowed time. With regard to student learning outcomes, a novel and potentially critical feature of our scheme is that the assessment task is designed to be intimately linked to the overall course content, in that it aims to introduce important concepts and skills (via individual student exploration) which will be revisited somewhat later in the pedagogically more restrictive formal lecture component of the course (typically a large group plenary format). Furthermore, the time delay involved, or "incubation period", is also a deliberate design feature: it is intended to allow students the opportunity to undergo potentially important internal re-adjustments in their understanding, before being exposed to lectures on related course content which are invariably delivered in a more condensed, formal and mathematically rigorous manner. In our presentation, we will discuss in more detail our motivation and rationale for trailing such a scheme for the targeted student group. Some of the advantages and disadvantages of our approach (as we perceived them at the initial stages) will also be enumerated. In a companion paper, the theoretical framework for our approach will be more fully elaborated, and measures of student learning outcomes (as obtained from eg. student provided feedback) will be discussed.

Changing assessment practices : the case for online moderation

Online moderation meetings have the potential to support the collaborative professional development of teachers, and the formation of a common understanding of what denotes quality in student work in a standards based assessment system. In doing so systemic calls for consistency across education systems are also being met. In this paper a case for employing online moderation meetings is developed through recourse to the demands of learning in the twenty-first century and the place of assessment within those discourses. It is argued that empirical data is needed on the efficacy of online moderation meetings to guide future practice as the use of information and communication technologies increases in education systems. Online moderation is one way of gathering teachers across vast distances to share their understandings and develop common meanings of assessment. While it is suggested that online moderation is one possible procedure to meet systemic requirements and support teachers’ professional collaboration, the implementation of such a system also introduces new challenges for schools and teachers. Meeting online to discuss professional understandings is a new way of operating for teachers and involves technology that has not yet been fully utilised within education departments. Issues such as the types of interactions that are afforded within such an environment, as well as technical operating problems that occur when using technology impact on the employment of online meetings. Online moderation meetings while potentially solving the issue of developing common understandings across an entire department also pose new issues to be resolved. There is a need for research into the efficacy of online moderation meetings so that future policy decisions may be based on sound empirical data. It is imperative that as new ways of knowing and acting are incorporated into school curriculum and pedagogy, assessment practices are also aligned. Online moderation meetings can support such practices by enabling teachers to communicate with a wider and more diverse group of teachers to establish common understandings.

Aligning ICT in assessment with teaching and learning : enhancing student achievement in the middle years

Student underachievement in the middle years (typically Years 4 to 9) is a concern in education. Incorporating Information and Communication Technologies (ICT) in assessment that is aligned to teaching and learning has the potential to engage students in higher cognitive processes that lead to increased student achievement. To examine this proposition an investigation was undertaken into teachers’ perceptions of alignment and the implications of those for student achievement in ICT enhanced middle years assessment tasks. This investigation used a collective case study design underpinned by socio-cultural theory. Two methods were used for data collection, namely, semi-structured interviews with individual teachers and a focus group discussion with teachers and another with students. Findings revealed teachers’ perceptions that alignment: assists in mediating achievement of learning outcomes in quality middle years assessment tasks, assists in creating a challenging but supportive environment in which positive learning dispositions and success is encouraged for all students, and contributes to more rigorous use of ICT in assessment. The process of implementing alignment was found to be complex but assisted through prioritising particular practices. These findings enabled the development of eight steps which serve as a guide to the effective implementation of alignment in middle years assessment tasks.

Formative and Summative Assessment and the Notion of Constructive Allignment

This paper discusses perceptions of first year accounting students about their tutorial activities and their engagements in assessment. As the literature suggests, unless participation in learning activities forms part of graded assessment it is often difficult to engage students in these activities. Using an action research model, this paper reports the study of first year accounting students' responses to action-orientated learning tasks in tutorials. The paper focuses on the importance of aligning curriculum objectives, learning and teaching activities and assessment,i.e. the notion of constructive alignment. However, as the research findings indicate, without support at institutional level, applying constructive alignment to facilitate quality student learning outcomes is a difficult task. Thus, the impacts of policy constraints on curriculum issues are also discussed, focusing on the limitations faced by tutors and their lack of involvement in curriculum development.

Assessment and curriculum reform in China : the college English test and tertiary English as a foreign language education.

Educational assessment was a worldwide commonplace practice in the last century. With the theoretical underpinnings of education shifting from behaviourism and social efficiency to constructivism and cognitive theories in the past two decades, the assessment theories and practices show a widespread changing movement. The emergent assessment paradigm, with a futurist perspective, indicates a deviation away from the prevailing large scale high-stakes standardised testing and an inclination towards classroom-based formative assessment. Innovations and reforms initiated in attempts to achieve better education outcomes for a sustainable future via more developed learning and assessment theories have included the 2007 College English Reform Program (CERP) in Chinese higher education context. This paper focuses on the College English Test (CET) - the national English as a Foreign Language (EFL) testing system for non-English majors at tertiary level in China. It seeks to explore the roles that the CET played in the past two College English curriculum reforms, and the new role that testing and assessment assumed in the newly launched reform. The paper holds that the CET was operationalised to uplift the standards. However, the extended use of this standardised testing system brings constraints as well as negative washback effects on the tertiary EFL education. Therefore in the newly launched reform -CERP, a new assessment model which combines summative and formative assessment approaches is proposed. The testing and assessment, assumed a new role - to engender desirable education outcomes. The question asked is: will the mixed approach to formative and summative assessment provide the intended cure to the agony that tertiary EFL education in China has long been suffering - spending much time, yet achieving little effects? The paper reports the progresses and challenges as informed by the available research literature, yet asserts a lot needs to be explored on the potential of the assessment mix in this examination tradition deep-rooted and examination-obsessed society.

Assessment of Speech Dialog Systems using Multi-Modal Cognitive Load Analysis and Driving Performance Metrics

In this paper, cognitive load analysis via acoustic- and CAN-Bus-based driver performance metrics is employed to assess two different commercial speech dialog systems (SDS) during in-vehicle use. Several metrics are proposed to measure increases in stress, distraction and cognitive load and we compare these measures with statistical analysis of the speech recognition component of each SDS. It is found that care must be taken when designing an SDS as it may increase cognitive load which can be observed through increased speech response delay (SRD), changes in speech production due to negative emotion towards the SDS, and decreased driving performance on lateral control tasks. From this study, guidelines are presented for designing systems which are to be used in vehicular environments.