Towards improving public understanding of judgement practice in standards-referenced assessment : an Australian perspective

Internationally there has been a move towards standards-referenced assessment with countries such as Australia developing a National Curriculum and Achievement Standards, New Zealand adopting National Standards for literacy and numeracy that involve schools making and reporting judgements about the reading, writing and mathematics achievement of children up to Year 8 (the end of primary school) and in Canada, classroom assessment standards aimed at the improvement of assessment practice of K-12 education are being formulated. Standards-driven reform has major implications for teachers’ work. The consequences of adopting a standards-driven approach to educational change by systems are often under-estimated with the unintended effects not fully understood by either the policy writers, and the public, including parents. It is for these reasons that the contention developed in this article relates to the teacher’s role, which it is argued remains central to policy focused on the improvement of the quality of education and educational standards.

Levelling expectations across different years in an undergraduate degree to support work-related learning : evidence for a multi-pronged approach.

One of the ways in which university departments and faculties can enhance the quality of learning and assessment is to develop a ‘well thought out criterion‐referenced assessment system’ (Biggs, 2003, p. 271). In designing undergraduate degrees (courses) this entails making decisions about the levelling of expectations across different years through devising objectives and their corresponding criteria and standards: a process of alignment analogous to what happens in unit (subject) design. These decisions about levelling have important repercussions in terms of supporting students’ work‐related learning, especially in relation to their ability to cope with the increasing cognitive and skill demands made on them as they progress through their studies. They also affect the accountability of teacher judgments of students’ responses to assessment tasks, achievement of unit objectives and, ultimately, whether students are awarded their degrees and are sufficiently prepared for the world of work. Research reveals that this decision‐making process is rarely underpinned by an explicit educational rationale (Morgan et al, 2002). The decision to implement criterion referenced assessment in an undergraduate microbiology degree was the impetus for developing such a rationale because of the implications for alignment, and therefore ‘levelling’ of expectations across different years of the degree. This paper provides supporting evidence for a multi‐pronged approach to levelling, through backward mapping of two revised units (foundation and exit year). This approach adheres to the principles of alignment while combining a work‐related approach (via industry input) with the blended disciplinary and learner‐centred approaches proposed by Morgan et al. (2002). It is suggested that this multi‐pronged approach has the potential for making expectations, especially work‐related ones across different year levels of degrees, more explicit to students and future employers.

Orals ain’t orals : the role of prepared oral presentations in the classroom and beyond. How do instruction and feedback practices guide delivery choices?

Despite an ostensibly technology-driven society, the ability to communicate orally continues to feature as an essential ability for students at school and university, as it is for graduates in the workplace. Pedagogically, one rationale is that the need to develop effective oral communication skills is tied to life-long learning which includes successful participation in future work-related tasks. One tangible way that educators have assessed proficiency in the area of communication is through prepared oral presentations. While much of the literature uses the terms 'oral communication' and 'oral presentation' interchangeably, some writers question the role more formal presentations play in the overall development of oral communication skills. However, such formal speaking tasks continue to be a recognised assessment practice in both the secondary school and academy, and, therefore, worthy of further investigation. Adding to the discussion, this thesis explores the knowledge and skills students bring into the academy from previous educational experiences. It examines some of the teaching and assessment methods used in secondary schools to develop oral communication skills through the use of formal oral presentations. Specifically, it investigates criterion-referenced assessment sheets and how these tools are used as a form of instruction, as well as their role and effectiveness in the evaluation of student ability. The focus is on the student's perspective and includes 12 semi-structured interviews with school students. The purpose of this thesis is to explore key thematics underpinning oral communication and to identify tensions between expectations and practice. While acknowledging the breadth and depth of material available under the heading of 'communication theory', this study specifically draws on an expanded view of the rhetorical tradition to fully interrogate the assumptions supporting the practice of assessing oral presentations. Finally, this thesis recommends reconnecting with an updated understanding of rhetoric as a way of assisting in the development of expressive, articulate and discerning communicators.

Desempenho funcional nas atividades de rotina escolar de alunos com necessidades educacionais especiais na persectiva do professor

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Higher order thinking in chemistry curriculum and its assessment

Higher-order thinking has featured persistently in the reform agenda for science education. The intended curriculum in various countries sets out aspirational statements for the levels of higher-order thinking to be attained by students. This study reports the extent to which chemistry examinations from four Australian states align and facilitate the intended higher-order thinking skills stipulated in curriculum documents. Through content analysis, the curriculum goals were identified for each state and compared to the nature of question items in the corresponding examinations. Categories of higher-order thinking were adapted from the OECD’s PISA Science test to analyze question items. There was considerable variation in the extent to which the examinations from the states supported the curriculum intent of developing and assessing higher-order thinking. Generally, examinations that used a marks-based system tended to emphasize lower-order thinking, with a greater distribution of marks allocated for lower-order thinking questions. Examinations associated with a criterion-referenced examination tended to award greater credit for higher-order thinking questions. The level of complexity of chemistry was another factor that limited the extent to which examination questions supported higher-order thinking. Implications from these findings are drawn for the authorities responsible for designing curriculum and assessment procedures and for teachers.

A study of the TAGS, an enduring teaching tool: teachers’ perspectives on the Teacher Assessment of Grammatical Structures teaching practices and application

This paper is studied look at the Teacher’s Assessment of Grammatical Structures (TAGS), Teacher Assessment of Spoken Language (TASL), and Cottage Acquisition Scales for Listening, Language and Speech (CASLLS) criterion-referenced language assessments as well as an inquiry into how teachers of the deaf use the TAGS currently as a teaching tool.

Improving math and science assessment : report on the Secretary's third conference on mathematics and science education /

"PIP 94-1201"--P. [4] of cover.

The effects of school mobility in pre -assessment grade levels on student performance and school accountability in urban Title I schools

This study examined the effects of student mobility and educational enrollment experiences on academic achievement. The educational progress, school enrollments and transfers of inner-city elementary students were tracked over a four-year period. Student achievement was measured by criterion-referenced reading tests administered in the second semester of the third grade. It further analyzed the degree to which the switch to different basal reading textbooks interrupted the continuity of education thereby contributing to the detrimental effects of intra-district mobility. ^ Mobility histories of 2,913 third grade students were collected to evaluate the number of times each student entered or withdrew from a Miami-Dade County Public School beginning in August 2000 through March 2004, and distinguished between transfers that occurred during the academic school year and those that occurred during summer months. Data were analyzed using Pearson correlations and multiple regressions to determine if school mobility contributed to performance on the Florida Comprehensive Assessment Third Grade Reading Test (FCAT). Transferring from one school to another was found to have a significant negative impact on student test scores. Transfers within the academic school year were more detrimental than transfers that occurred during the summer months. Third grade students who transferred into schools that used the same reading textbook series were found to have significantly higher FCAT reading scores than third graders who transferred into schools that used different reading textbooks. ^ The effects of mobility rates on overall school performance were also examined. Data was collected on 124 Title I elementary schools to determine the degree to which mobility affected school accountability scores. Title I schools with high student mobility rates had significantly lower accountability scores than schools with lower student mobility rates. ^ The results of this study highlight the impact of education and housing policy and imply a need for programs and practices that promote stability in the early elementary years. ^

Closing the loop 21st century style : providing feedback on written assessment via MP3 recordings

Feedback on student performance, whether in the classroom or on written assignments, enables them to reflect on their understandings and restructure their thinking in order to develop more powerful ideas and capabilities. Research has identified a number of broad principles of good feedback practice. These include the provision of feedback that facilitates the development of reflection in learning; helps clarify what good performance is in terms of goals, criteria and expected standards; provides opportunities to close the gap between current and desired performance; delivers high quality information to students about their learning; and encourages positive motivational beliefs and self-esteem. However, high staff–student ratios and time pressures often result in a gulf between this ideal and reality. Whilst greater use of criteria referenced assessment has enabled an improvement in the extent of feedback being provided to students, this measure alone does not go far enough to satisfy the requirements of good feedback practice. Technology offers an effective and efficient means by which personalised feedback may be provided to students. This paper presents the findings of a trial of the use of the freely available Audacity program to provide individual feedback via MP3 recordings to final year Media Law students at the Queensland University of Technology on their written assignments. The trial has yielded wide acclaim by students as an effective means of explaining the exact reasons why they received the marks they were awarded, the things they did well and the areas needing improvement. It also showed that good feedback practice can be achieved without the burden of an increase in staff workload.

Teachers using annotations to engage students in assessment conversations : recontextualising knowledge

Assessment for Learning practices with students such as feedback, and self- and peer assessment are opportunities for teachers and students to develop a shared understanding of how to create quality learning performances. Quality is often represented through achievement standards. This paper explores how primary school teachers in Australia used the process of annotating work samples to develop shared understanding of achievement standards during their curriculum planning phase, and how this understanding informed their teaching so that their students also developed this understanding. Bernstein's concept of the pedagogic device is used to identify the ways teachers recontextualised their assessment knowledge into their pedagogic practices. Two researchers worked alongside seven primary school teachers in two schools over a year, gathering qualitative data through focus groups and interviews. Three general recontextualising approaches were identified in the case studies; recontextualising standards by reinterpreting the role of rubrics, recontextualising by replicating the annotation process with the students and recontextualising by reinterpreting practices with students. While each approach had strengths and limitations, all of the teachers concluded that annotating conversations in the planning phase enhanced their understanding, and informed their practices in helping students to understand expectations for quality.

Integrating ‘role play’ in assessment to strengthen professional conduct and accountability of students : a pilot study

BACKGROUND In a process engineering setting, graduates are frequently allocated reviews of existing operations or required to scope new production processes by their supervisors with a view to improving or expanding on operations and overall productivity. These tasks may be carried out in teams and in consultation with the process engineer’s immediate line manager or a more experienced engineer, such as the Production or Maintenance Manager; ultimately reporting to senior management, which is frequently a non-engineer. Although professional skills development is part of engineering curricula, ‘professional conduct’ and ‘accountability’ required for dealing with peers and superiors in industry is not very well addressed at university. Consequently, upon graduation, many students are, in terms of knowledge and experience in this area, underprepared to work effectively in industry settings. PURPOSE The purpose of this study was to develop and implement a role-play scenario within a core 2nd year process engineering unit, so that students could gain knowledge, skills and experience in different aspects (and nuances) of professional conduct and accountability. DESIGN/METHOD In the role-play scenario, students worked in ‘engineering production teams’ to design a process for an iconic Queensland fruitcake and to present their solution and recommendations (culminating in a poster presentation) to an assessment panel consisting of staff, role-playing as, ‘production and plant managers’. Students were assessed on several areas, including professionalism using a criteria referenced assessment guide by a 3-member cross-disciplinary staff panel consisting of a Business Faculty lecturer, an engineer from industry and the lecturer of the Process Engineering unit. Professional conduct and accountability was gauged through direct questioning by the panel. Feedback was also sought from students on various aspects through a survey questionnaire after the role play activity at the end of semester. RESULTS Overall, the role play was very well performed with students achieving an average score of 79.3/100 (distinction grade). Professional conduct as assessed by panel was on average better than scores given for professional accountability (4.0 compared with 3.6 out of 5). Feedback from students indicated that the learning activities had contributed to their overall understanding of the content and the role of process engineers. Industry involvement was rated very highly as contributing to their learning at 4.8 (on Likert scale from 1 – 5) and the poster presentation was rated at 3.6. CONCLUSIONS This pilot study was successful in implementing a new assessment task for modelling professional conduct and accountability within a 2nd year core unit. This task incorporated a role-play activity and there was evidence to suggest that this and associated learning tasks were successful in broadening students’ understanding and skills in this area required for engineering practice. Following feedback given by students and staff, improvements will be made to the nature of the problem, how it is defined, its assessment, and the approach taken in the role-play scenario when the unit is offered in 2014.

Increased body fat is independently and negatively related with cardiorespiratory fitness levels in children and adolescents with normal weight.

Body mass index (BMI) is related with cardiorespiratory fitness (CRF), but less is known regarding the combined relationships between BMI and body fat (BF) on CRF. Cross-sectional study included 2361 girls and 2328 boys aged 10–18 years living in the area of Lisbon, Portugal. BMI was calculated by measuring height and weight, and obesity was assessed by international criteria. BF was assessed by bioimpedance. CRF was assessed by the 20-m shuttle run and the participants were classified as normal-to-high or low-CRF level according to Fitness gram criterion-referenced standards. The prevalence of low CRF was 47 and 39% in girls and boys, respectively. The corresponding values for the prevalence of obesity were 4.8 and 5.6% (not significant) and of excess BF of 12.1 and 25.1% (P <0.001), respectively. In both sexes, BMI and BF were inversely related with CRF: r = – 0.53 and – 0.45 for BMI and % BF, respectively, in boys and the corresponding values in girls were – 0.50 and – 0.33 (all P <0.01). When compared with a participant with normal BMI and BF, the odds ratios (95% confidence interval) for low CRF were 1.94 (1.46–2.58) for a participant with normal BMI and high BF, and 6.19 (5.02–7.63) for a participant with high BMI and high BF. The prevalence of low-CRF levels is high in Portuguese youths. BF negatively influences CRF levels among children/adolescents with normal BMI.

Item and task difficulty in a B2 reading examination: perceptions of test-takers and CEFR alignment experts compared with psychometric measurements

The article presents a study of a CEFR B2-level reading subtest that is part of the Slovenian national secondary school leaving examination in English as a foreign language, and compares the test-taker actual performance (objective difficulty) with the test-taker and expert perceptions of item difficulty (subjective difficulty). The study also analyses the test-takers’ comments on item difficulty obtained from a while-reading questionnaire. The results are discussed in the framework of the existing research in the fields of (the assessment of) reading comprehension, and are addressed with regard to their implications for item-writing, FL teaching and curriculum development.