The impact of high stakes testing: the Australian story

High stakes testing in Australia was introduced in 2008 by way of the National Assessment Program – Literacy and Numeracy (NAPLAN). Currently, every year all students in Years 3, 5, 7 and 9 are assessed on the same days using national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy. In 2010 the NAPLAN results were published on the Federal Government MySchool website. The impact of these high stakes tests on jurisdictions, school principals, parents and students is considered in this article. We draw on reported observations from the Australian Primary Principals Association during 2009–10 testing periods across the country and published Australian research on the impact of high stakes literacy and numeracy testing. We also examine alternative approaches that include the use of assessment evidence for learning improvement purposes and for accountability purposes. In considering alternatives to the current large-scale testing approach we draw on key insights from research on teacher judgement, achievement standards and social moderation in the context of national curriculum and assessment reform in support of the suggested directions forward.

The ascendancy of oracy in university courses: Implications for teachers and second language users

This chapter reports on a study of oracy in a first-year university Business course, with particular interest in the oracy demands for second language-using international students. The research is relevant at a time when Higher Education is characterised by the confluence of increased international enrolments, more dialogic teaching and learning, and imperatives for teamwork and collaboration. Data sources for the study included videotaped lectures and tutorials, course documents, student surveys, and an interview with the lecturer. The findings pointed to a complex, oracy-laden environment where interactive talk fulfilled high-stakes functions related to social inclusion, the co-construction of knowledge, and the accomplishment of assessment tasks. The salience of talk posed significant challenges for students negotiating these core functions in their second language. The study highlights the oracy demands in university courses and foregrounds the need for university teachers, curriculum writers and speaking test developers to recognise these demands and explicate them for the benefit of all students.

Enterprise information security policy assessment : an extended framework for metrics development utilising the goal-question-metric approach

Effective enterprise information security policy management requires review and assessment activities to ensure information security policies are aligned with business goals and objectives. As security policy management involves the elements of policy development process and the security policy as output, the context for security policy assessment requires goal-based metrics for these two elements. However, the current security management assessment methods only provide checklist types of assessment that are predefined by industry best practices and do not allow for developing specific goal-based metrics. Utilizing theories drawn from literature, this paper proposes the Enterprise Information Security Policy Assessment approach that expands on the Goal-Question-Metric (GQM) approach. The proposed assessment approach is then applied in a case scenario example to illustrate a practical application. It is shown that the proposed framework addresses the requirement for developing assessment metrics and allows for the concurrent undertaking of process-based and product-based assessment. Recommendations for further research activities include the conduct of empirical research to validate the propositions and the practical application of the proposed assessment approach in case studies to provide opportunities to introduce further enhancements to the approach.

Raising the stakes: the challenges for teacher assessment

These are changing times in Australia for teachers and their students, with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to use assessment in the support of and improvement of learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the putative purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are the questions that are addressed in this article, with reference to innovative research from global networks that have maintained the assessment focus on learning rather than accountability practices.

The role of cultural beliefs in contributing to risky road use behaviour in Pakistan

Background Pakistan has the highest population rate of road fatalities in South Asia (25.3 fatalities per 100,000 people: Global Status Report on Road Safety, WHO 2009). Along with road environment and vehicle factors, human factors make a substantial contribution to traffic safety in Pakistan. Beliefs about road crash causation and prevention have been demonstrated to contribute to risky road use behaviour and resistance to preventive measures in a handful of other developing countries, but has not been explored in Pakistan. In particular, fatalism (whether based on religion, other cultural beliefs or experience) has been highlighted as a barrier to achieving changes in attitudes and behaviour. Aims The research reported here aimed (i) to explore perceptions of road crash causation among policy makers, police officers, professional drivers and car drivers in Pakistan; (ii) to identify how cultural and religious beliefs influence road use behaviour in Pakistan; and (iii) to understand how fatalistic beliefs may work as obstacles to road safety interventions. Methods In-depth interviews were conducted by the primary author (mostly in Urdu) in Lahore, Rawalpindi and Islamabad with 12 professional drivers (taxi, bus and truck), 4 car drivers, 6 police officers, 4 policy makers and 2 religious orators. All but two were Muslim, two were female, and they were drawn from a wide range of ages (24 to 60) and educational backgrounds. The interviews were taped and transcribed, then translated into English and analysed for themes related to the aims. Results Fatalism emerged as a pervasive belief utilised to justify risky road use behaviour and to resist messages about preventive measures. There was a strong religious underpinning to the statement of fatalistic beliefs (this reflects popular conceptions of Islam rather than scholarly interpretations), but also an overlap with superstitious beliefs which have longer-standing roots in Pakistani culture. These beliefs were not limited to people of poor educational background or position. A particular issue which was explored in more detail was the way in which these beliefs and their interpretation within Pakistani society contributed to poor police reporting of crashes. Discussion and conclusions The pervasive nature of fatalistic beliefs in Pakistan affects road user behaviour by supporting continued risk taking behaviour on the road, and by interfering with public health messages about behaviours which would reduce the risk of traffic crashes. The widespread influence of these beliefs on the ways that people respond to traffic crashes and the death of family members contribute to low crash reporting rates and to a system which is difficult to change. The promotion of an evidence-based approach to road user behaviour is recommended, along with improved professional education for police and policy makers.

Teachers' knowledge for social education : perspectives from the middle years of schooling

Social education in Australia is a divisive educational issue. The last decade has been marked by the controversial integrated social studies curriculum, Studies of Society and Environment (SOSE) where history, geography and environmental studies were integrated with civics and citizenship. The introduction of a compulsory K-10 Australian Curriculum from 2011, however, marks the return to history and geography and the abandonment of SOSE. Curriculum reform aside, what do teachers think is essential knowledge for middle years social education? The paper reports on a phenomenographical exploration of thirty-one middle school teachers’ conceptions of essential knowledge for SOSE. Framed by Shulman’s (1986, 1987) theoretical framework of the knowledge base for teaching, the research identified seven qualitatively different ways of understanding essential knowledge for social education. The study indicates a professional practice-based theorisation of social education that justifies attention to discipline-based knowledge and teacher identity in the middle years.

Explicit, latent and meta-criteria : types of criteria at play in professional judgement practice

This paper engages with debates about whether comprehensive prior specification of criteria and standards is sufficient for informed professional judgement. A preoccupation has emerged with the specificity and explication of criteria intended to regulate judgement. This has resulted in criteria-compliance in the use of defined standards to validate judgements and improve reliability and consistency. Compliance has become a priority, the consequence being the prominence of explicit criteria, to the lack of acknowledgement of the operation of latent and meta-criteria within judgement practice. This paper examines judgement as a process involving three categories of assessment criteria in the context of standards-referenced systems: explicit, latent and meta-criteria. These are understood to be wholly interrelated and interdependent. A conceptualisation of judgement involving the interplay of the three criteria types is presented with an exploration of how they function to focus or alter assessments of quality in judgements of achievement in English and Mathematics.

Health Assessment and Physical Examination

"This first Australia and New Zealand edition of the comprehensive Estes’ Health Assessment and Physical Examination is designed to teach students to assess a patient’s physical, psychological, cultural and emotional dimensions of health as a foundation of nursing care. The skills of interviewing, inspection, percussion, palpation, auscultation, and documentation are defined to help students to make clinical assessments and promote healthy patient outcomes. A strong emphasis on science encompasses all the technical aspects of anatomy, physiology and assessment, while highlighting clinically relevant information. Emphasis on caring is displayed through themes of assessment of the whole person, which also encourages nurses to think about care for themselves as well as patients."--publisher website

Challenging industry conceptions with provotypes

Design researchers have an important role to play when engaged with user-driven design projects in industry. Design researchers can craft ethnographic material to facilitate transfers of user-knowledge to industry, and demonstrate how this material can be used in the design of new products and services. However, ethnographic findings can reveal issues that are in tension with conceptions of the project members from industry. Instead of brushing these tensions aside, we propose provotyping (provocative prototyping) as an approach to constructively build on them as a resource for change. Provotypes are ethnographically rooted, technically working, robust artefacts that deliberately challenge stakeholder conceptions by reifying and exposing tensions that surround a field of organisational interest. The daily and local experience of provotypes aims to stir dialectical processes of reflection on how conceptions currently are, and fuel the front end of a development process by speculating how conceptions could be different. In this article we start by making explicit the relation between provotypes, practices of critical design and organisational sense-making. We then illustrate, through a multi-stakeholder project concerning the field of indoor climate, how provotypes facilitate transfers of user knowledge to industry, and how they contribute to the development of new products and services. We end by framing the role of the design researcher and discuss the politics that are inherent to design provocations.

Framing Movements for Gesture Interface Design

Gesture interfaces are an attractive avenue for human-computer interaction, given the range of expression that people are able to engage when gesturing. Consequently, there is a long running stream of research into gesture as a means of interaction in the field of human-computer interaction. However, most of this research has focussed on the technical challenges of detecting and responding to people’s movements, or on exploring the interaction possibilities opened up by technical developments. There has been relatively little research on how to actually design gesture interfaces, or on the kinds of understandings of gesture that might be most useful to gesture interface designers. Running parallel to research in gesture interfaces, there is a body of research into human gesture, which would seem a useful source to draw knowledge that could inform gesture interface design. However, there is a gap between the ways that ‘gesture’ is conceived of in gesture interface research compared to gesture research. In this dissertation, I explore this gap and reflect on the appropriateness of existing research into human gesturing for the needs of gesture interface design. Through a participatory design process, I designed, prototyped and evaluated a gesture interface for the work of the dental examination. Against this grounding experience, I undertook an analysis of the work of the dental examination with particular focus on the roles that gestures play in the work to compare and discuss existing gesture research. I take the work of the gesture researcher McNeill as a point of focus, because he is widely cited within gesture interface research literature. I show that although McNeill’s research into human gesture can be applied to some important aspects of the gestures of dentistry, there remain range of gestures that McNeill’s work does not deal with directly, yet which play an important role in the work and could usefully be responded to with gesture interface technologies. I discuss some other strands of gesture research, which are less widely cited within gesture interface research, but offer a broader conception of gesture that would be useful for gesture interface design. Ultimately, I argue that the gap in conceptions of gesture between gesture interface research and gesture research is an outcome of the different interests that each community brings to bear on the research. What gesture interface research requires is attention to the problems of designing gesture interfaces for authentic context of use and assessment of existing theory in light of this.

From vision to reality : views of primary school principals on inclusive education in New South Wales, Australia

This paper discusses the findings of a research study that used semi-structured interviews to explore the views of primary school principals on inclusive education in New South Wales, Australia. Content analysis of the transcript data indicates that principals’ attitudes towards inclusive education and their success in engineering inclusive practices within their school are significantly affected by their own conception of what “inclusion” means, as well as the characteristics of the school community, and the attitudes and capacity of staff. In what follows, we present two parallel conversations that arose from the interview data to illustrate the main conceptual divisions existing between our participants’ conceptions of inclusion. First, we discuss the act of “being inclusive” which was perceived mainly as an issue of culture and pedagogy. Second, we consider the mechanics of “including,” which reflected a more instrumentalist position based on perceptions of individual student deficit, the level of support they may require and the amount of funding they can attract.

On the reliability of classifying programming tasks using a Neo-Piagetian theory of cognitive development

Recent research has proposed Neo-Piagetian theory as a useful way of describing the cognitive development of novice programmers. Neo-Piagetian theory may also be a useful way to classify materials used in learning and assessment. If Neo-Piagetian coding of learning resources is to be useful then it is important that practitioners can learn it and apply it reliably. We describe the design of an interactive web-based tutorial for Neo-Piagetian categorization of assessment tasks. We also report an evaluation of the tutorial's effectiveness, in which twenty computer science educators participated. The average classification accuracy of the participants on each of the three Neo-Piagetian stages were 85%, 71% and 78%. Participants also rated their agreement with the expert classifications, and indicated high agreement (91%, 83% and 91% across the three Neo-Piagetian stages). Self-rated confidence in applying Neo-Piagetian theory to classifying programming questions before and after the tutorial were 29% and 75% respectively. Our key contribution is the demonstration of the feasibility of the Neo-Piagetian approach to classifying assessment materials, by demonstrating that it is learnable and can be applied reliably by a group of educators. Our tutorial is freely available as a community resource.

Examining the Assessment Opportunities for Cultural Connectedness for Student Learning : A Sociocultural Analysis

The disparity that exists between the highest and lowest achievers together with deficit approaches to teaching, learning and assessment raise serious equity issues related to fairness, validity, culture and access, which were analysed in a recent Australian Research Council funded project. This chapter explores the potential that exists for teachers to work with Indigenous Teacher Assistants (ITAs) to secure cultural connectedness in teaching, learning and assessment of Indigenous students. The study was a design experiment, which was conducted in seven Catholic and Independent primary schools in northern Queensland and involved semi-structured focus group interviews with Year 4 and 6 Indigenous students, principals, teachers and Indigenous Teacher Assistants. Classroom observations and document analyses were also conducted. This corpus of data was analysed using a sociocultural theoretical lens. The use of a sociocultural analysis helped to identify cultural influences, Indigenous students’ funds of knowledge and values. The information from this analysis was made explicit to teachers to demonstrate how they could enhance their pedagogic and assessment practices by embracing and extending the cultural spaces for learning and teaching of Indigenous students. The way in which teachers construct their interactions for greater cultural connectedness and enhanced learning would appear to rely on relationship building with Indigenous staff, Indigenous students’ cultural knowledge, and improved understanding of assessment and related equity issues.

P.I.C.U., H.D.U., A.O.A. What treatment do we provide?: Current descriptions of the function of intensive care for inpatient Psychiatric Health Care.

Purpose – This paper adds to growing research of psychiatric intensive care units (PICU) by recounting descriptions of psychiatric intensive care settings and discusses the perceptions held by nurses of the organisational interfaces, arrangements and provisions of care in these settings. Design/methodology/approach – Data gathered from focus groups held with nurses from two PICUs was used to establish terminology, defining attributes, related concepts, antecedents, values, processes and concepts related to current practices. A literature search was conducted to permit a review of the conceptual arrangements and contemporary understanding of intensive care for people experiencing acute psychiatric illness based on the perspectives held by the nurses from the focus groups. Findings – Dissonance between service needs and the needs and management of individual patients overshadow strategies to implement comprehensive recovery-oriented approaches. Three factors are reported in this paper that influence standards and procedural practice in PICU; organisational structures; physical structures; and subtype nomenclature. Practical implications – Acute inpatient care is an important part of a comprehensive approach to mental health services. Commonly intensive acute care is delivered in specialised wards or units co-located with acute mental health inpatient units mostly known as PICU. Evidence of the most effective treatment and approaches in intensive care settings that support comprehensive recovery for improved outcomes is nascent. Originality/value – Current descriptions from nurses substantiate wide variations in the provisions, design and classifications of psychiatric intensive care. Idiosyncratic and localised conceptions of psychiatric intensive care are not adequately entailing effective treatment and methods in support of recovery principles for improved and comprehensive outcomes. The authors suggest that more concrete descriptions, guidelines, training and policies for provision of intensive psychiatric health care encompassing the perspective of nursing professionals, would reinforce conceptual construction and thus optimum treatments within a comprehensive, recovery-oriented approach to mental health services.

Investigating ways of seeing IT research : a tool for facilitating effective research partnerships?

Effective research partnerships, both intra- and interdisciplinary, as well as academy-industry partnerships, rely on shared understandings of particular aspects of the research endeavour. Research partnerships are essentially learning partnerships, if we accept the argument that research may be seen as learning at the collective level. This paper establishes the need to investigate information technology (IT) researchers' varying ways of seeing certain aspects of their research world, in order to assist the process of forging effective research partnerships. We analyse the importance of facilitating effective partnerships in IT research, discuss our plan for investigating the collective consciousness of IT researchers, and explain some of the strategies to be used in the investigation.