Assessing students in senior science: an analysis of questions in contextualised chemistry exams

This study explores the development of a coding system for analysing test questions in two context-based chemistry exams. We describe our unique analytical procedures before contrasting the data from both tests. Our findings indicate that when a new curriculum is developed such as a context-based curriculum, teachers are required to combine the previously separate domains of context and concept to develop contextualised assessment. We argue that constructing contextualised assessment items requires teachers to view concepts and context as interconnected rather than as separate entities that may polarise scientific endeavour. Implications for practice, curriculum and assessment-development in context-based courses are proposed.

Short answer versus multiple choice examination questions for first year chemistry

Multiple choice (MC) examinations are frequently used for the summative assessment of large classes because of their ease of marking and their perceived objectivity. However, traditional MC formats usually lead to a surface approach to learning, and do not allow students to demonstrate the depth of their knowledge or understanding. For these reasons, we have trialled the incorporation of short answer (SA) questions into the final examination of two first year chemistry units, alongside MC questions. Students’ overall marks were expected to improve, because they were able to obtain partial marks for the SA questions. Although large differences in some individual students’ performance in the two sections of their examinations were observed, most students received a similar percentage mark for their MC as for their SA sections and the overall mean scores were unchanged. In-depth analysis of all responses to a specific question, which was used previously as a MC question and in a subsequent semester in SA format, indicates that the SA format can have weaknesses due to marking inconsistencies that are absent for MC questions. However, inclusion of SA questions improved student scores on the MC section in one examination, indicating that their inclusion may lead to different study habits and deeper learning. We conclude that questions asked in SA format must be carefully chosen in order to optimise the use of marking resources, both financial and human, and questions asked in MC format should be very carefully checked by people trained in writing MC questions. These results, in conjunction with an analysis of the different examination formats used in first year chemistry units, have shaped a recommendation on how to reliably and cost-effectively assess first year chemistry, while encouraging higher order learning outcomes.

Outcomes of the Chemistry Discipline Network Mapping Exercises: Are the Threshold Learning Outcomes met?

The Chemistry Discipline Network has recently completed two distinct mapping exercises. The first is a snapshot of chemistry taught at 12 institutions around Australia in 2011. There were many similarities but also important differences in the content taught and assessed at different institutions. There were also significant differences in delivery, particularly laboratory contact hours, as well as forms and weightings of assessment. The second mapping exercise mapped the chemistry degrees at three institutions to the Threshold Learning Outcomes for chemistry. Importantly, some of the TLOs were addressed by multiple units at all institutions, while others were not met, or were met at an introductory level only. The exercise also exposed some challenges in using the TLOs as currently written.

The continuing decline of science and mathematics enrolments in Australian high schools

Is there a crisis in Australian science and mathematics education? Declining enrolments in upper secondary Science and Mathematics courses have gained much attention from the media, politicians and high-profile scientists over the last few years, yet there is no consensus amongst stakeholders about either the nature or the magnitude of the changes. We have collected raw enrolment data from the education departments of each of the Australian states and territories from 1992 to 2012 and analysed the trends for Biology, Chemistry, Physics, two composite subject groups (Earth Sciences and Multidisciplinary Sciences), as well as entry, intermediate and advanced Mathematics. The results of these analyses are discussed in terms of participation rates, raw enrolments and gender balance. We have found that the total number of students in Year 12 increased by around 16% from 1992 to 2012 while the participation rates for most Science and Mathematics subjects, as a proportion of the total Year 12 cohort, fell (Biology (-10%), Chemistry (-5%), Physics (-7%), Multidisciplinary Science (-5%), intermediate Mathematics (-11%), advanced Mathematics (-7%) in the same period. There were increased participation rates in Earth Sciences (+0.3%) and entry Mathematics (+11%). In each case the greatest rates of change occurred prior to 2001 and have been slower and steadier since. We propose that the broadening of curriculum offerings, further driven by students' self-perception of ability and perceptions of subject difficulty and usefulness, are the most likely cause of the changes in participation. While these continuing declines may not amount to a crisis, there is undoubtedly serious cause for concern.