Understanding evaluation of learning support in mathematics and statistics

With rapid and continuing growth of learning support initiatives in mathematics and statistics found in many parts of the world, and with the likelihood that this trend will continue, there is a need to ensure that robust and coherent measures are in place to evaluate the effectiveness of these initiatives. The nature of learning support brings challenges for measurement and analysis of its effects. After briefly reviewing the purpose, rationale for, and extent of current provision, this article provides a framework for those working in learning support to think about how their efforts can be evaluated. It provides references and specific examples of how workers in this field are collecting, analysing and reporting their findings. The framework is used to structure evaluation in terms of usage of facilities, resources and services provided, and also in terms of improvements in performance of the students and staff who engage with them. Very recent developments have started to address the effects of learning support on the development of deeper approaches to learning, the affective domain and the development of communities of practice of both learners and teachers. This article intends to be a stimulus to those who work in mathematics and statistics support to gather even richer, more valuable, forms of data. It provides a 'toolkit' for those interested in evaluation of learning support and closes by referring to an on-line resource being developed to archive the growing body of evidence. © 2011 Taylor & Francis.

Do perceptions mirror reality? Student perceptions of learning versus grade outcomes

This research addresses whether educators should consider measuring if students have learned what was intended, as recommended by education researchers. Students in an Introductory Marketing subject were asked to complete a voluntary survey rating their own progress on the intended learning outcomes for the course. One hundred and one surveys were completed by students in the second-last teaching week of the semester. Student identification numbers were used to link student perceptions with their grade outcomes. Regression analysis was used to ascertain whether student perceptions of their progress on the intended learning outcomes for the course could be used to predict their grades. While the results were significant, student perceptions of their progress on learning outcomes were a poor predictor of grade outcomes. The results of this study suggest that student perceptions may not mirror the reality. These results are somewhat surprising and future research examining the degree of change in the learning outcomes perceived by students is warranted. This will further contribute to decisions surrounding whether educators should measure if students have learned what was intended.

Leading the transformation of learning and praxis in science classrooms

Individual science teachers who have inspired colleagues to transform their classroom praxis have been labelled transformational leaders. As the notion of distributed leadership became more accepted in the educational literature, the focus on the individual teacher-leader shifted to the study of leadership praxis both by individuals (whoever they might be) and by collectives within schools and science classrooms. This review traces the trajectory of leadership research, in the context of learning and teaching science, from an individual focus to a dialectical relationship between individual and collective praxis. The implications of applying an individual-collective perspective to praxis for teachers, students and their designated leaders are discussed.

New maps of learning for quality art education : what pre-service teachers should learn and be able to do

It would be a rare thing to visit an early years setting or classroom in Australia that does not display examples of young children’s artworks. This practice serves to give schools a particular ‘look’, but is no guarantee of quality art education. The Australian National Review of Visual Arts Education (NRVE) (2009) has called for changes to visual art education in schools. The planned new National Curriculum includes the arts (music, dance, drama, media and visual arts) as one of the five learning areas. Research shows that it is the classroom teacher that makes the difference, and teacher education has a large part to play in reforms to art education. This paper provides an account of one foundation unit of study (Unit 1) for first year university students enrolled in a 4-year Bachelor degree program who are preparing to teach in the early years (0–8 years). To prepare pre-service teachers to meet the needs of children in the 21st century, Unit 1 blends old and new ways of seeing art, child and pedagogy. Claims for the effectiveness of this model are supported with evidence-based research, conducted over the six years of iterations and ongoing development of Unit 1.

Middle school social sciences : exploring teachers’ conceptions of essential knowledge

This study examines teachers’ conceptions of essential knowledge in the humanities and social sciences, commonly referred to as "social education", in the middle years of schooling. Social education has long been a highly contested area of the curriculum in Australia. In Queensland, social education comprises the integrated learning area of Studies of Society and Environment (SOSE). However, the new Australian Curriculum marks a return to discipline-based study of history and geography. This phenomenographic study addresses a perceived lack of understanding in the current research literature in Australia of the nature of middle school teachers’ professional knowledge for teaching the social sciences. Teachers are conceptualised in this study as curriculum makers in the classroom and, as such, their conceptions of essential knowledge are significant. Shulman’s (1986, 1987) theory of teachers’ knowledge forms the theoretical foundation of the study, which is contextualised in Federal and State education policies and the literature on the middle phase of schooling. Transcripts of interviews conducted with a group of thirty-one Queensland middle school teachers of SOSE were subjected to phenomenographic analysis, revealing seven qualitatively different categories of description. Essential aspects of knowledge for social education emerging from the study were: (1) discipline-based knowledge; (2) curriculum knowledge; (3) knowledge derived from teaching experience; (4) knowledge of middle years learners; (5) knowledge of integration; (6) knowledge of current affairs; and (7) knowledge invested in teacher identity. The three dimensions of variation that linked and differentiated the categories were: (1) content; (2) inquiry learning; and (3) teacher autonomy. These findings are presented as an outcome space where the categories are grouped as knowledge of the learning area, knowledge of contexts and knowledge of self as teacher. The results of the study suggest that social education teachers’ identity and knowledge of self are critical aspects of their knowledge as curriculum makers. The results illustrate that the professional and personal domains intersect, extending Shulman’s (1986, 1987) original theorisation of teachers’ knowledge into the personal arena. Further, middle years teachers’ conceptions of essential knowledge reveal a practice-based theorisation of knowledge for social education that fits the goals of middle schooling. The research concludes that attention to teacher identity in teacher education and in-service professional development has considerable potential to grow teachers’ knowledge in the social sciences and enhance their capacity for school-based curriculum leadership.

Australian higher education evaluation through assurance of learning

A collaborative research project conducted by five Australian universities inquired into the philosophy and motivation for Assurance of Learning (AoL) as a process of education evaluation. Associate Deans Teaching and Learning representing Business schools from twenty-five universities across Australia participated in telephone interviews. Data was analysed using NVIVO9. Results indicated that articulated rationale for AoL was both ensuring that students had acquired the attributes and skills the universities claimed they had, and the philosophy of continuous improvement. AoL was motivated both by ritualistic objectives to satisfy accreditation requirements and virtuous agendas for quality improvement. Closing-the-loop was emphasised, but was mostly wishful thinking for next steps beyond data collection and reporting. AoL was conceptualised as one element within the larger context of quality review, but there was no evidence of comprehensive frameworks or strategic plans.

Urban pre-service teachers’ conceptions of teaching in rural communities

Encouraging quality teaching staff to apply for and accept teaching placements in rural and remote locations is an ongoing concern internationally. The value of different support mechanisms provided for pre-service teachers attending a rural and remote practicum[1] are investigated through theories of place and the school-community nexus. Qualitative data regarding the experiences of the pre-service teachers were collected through interviews and case study notes. This project adds to our understanding of practicum in rural areas by employing a conceptual understanding of place to propose how the experiences of a four-week practicum may contribute to urban pre-service teachers’ conceptions of work and life in a rural community

Students’ approaches to learning a new mathematical model

In this article, we report on the findings of an exploratory study into the experience of undergraduate students as they learn new mathematical models. Qualitative and quanti- tative data based around the students’ approaches to learning new mathematical models were collected. The data revealed that students actively adopt three approaches to under- standing a new mathematical model: gathering information for the task of understanding the model, practising with and using the model, and finding interrelationships between elements of the model. We found that the students appreciate mathematical models that have a real world application and that this can be used to engage students in higher level learning approaches.

Something’s coming, something good: Identifying TPACK competence in pre-service teachers’ analyses of learning objects

There is a song at the beginning of the musical, West Side Story, where the character Tony sings that “something’s coming, something good.” The song is an anthem of optimism, brimming with promise. This paper is about the long-held promise of information and communication technology (ICT) to transform teaching and learning, to modernise the learning environment of the classroom, and to create a new digital pedagogy. But much of our experience to date in the schooling sector tells more of resistance and reaction than revolution, of more of the same but with a computer in the corner and of ICT activities as unwelcome time-fillers/time-wasters. Recently, a group of pre-service teachers in a postgraduate primary education degree in an Australian university were introduced to learning objects in an ICT immersion program. Their analyses and related responses, as recorded in online journals, have here been interpreted in terms of TPACK (Technological Pedagogical and Content Knowledge). Against contemporary observation, these students generally displayed high levels of competence and highly positive dispositions of students to the integration of ICT in their future classrooms. In short, they displayed the same optimism and confidence as the fictional “Tony” in believing that something good was coming.

Digitally enabled?! An analysis of law and justice students' use of smart devices

The research seeks to understand the nature of law and justice students’ use of technology for their learning purposes. There is often an assumption made that all students have, and engage with, technology to the same degree. The research tests these assumptions by means of a survey conducted of first year law and justice students to determine their actual use of smart devices inside and outside classes. The analysis of results reveals that while the majority of respondents own at least one smart device; most rarely use their device for their learning purposes.

The changing motivations of students’ use of lecture podcasts across a semester: An extended theory of planned behaviour approach

We extended the previous work of Moss, O’Connor and White, to include a measure of group norms within the theory of planned behaviour (TPB), to examine the influences on students’ decisions to use lecture podcasts as part of their learning. Participants (N = 90) completed the extended TPB predictors before semester began (Time 1) and mid-semester (Time 2) and reported on their podcast use at mid-semester (Time 2) and end of semester (Time 3). We found that attitudes and perceived social pressures were important in informing intentions at both time points. At Time 1, perceptions of control over performing the behaviour and, at Time 2, perceptions of whether podcast use was normative among fellow students (group norms) also predicted intended podcast use. Intentions to use podcasting predicted self-reported use at both Time 2 and Time 3. These results provide important applied information for educators to encourage student use of novel on-line educational tools.

Scientific visualisations for developing students’ understanding of concepts in chemistry : some findings and some lessons learned

Scientific visualisations such as computer-based animations and simulations are increasingly a feature of high school science instruction. Visualisations are adopted enthusiastically by teachers and embraced by students, and there is good evidence that they are popular and well received. There is limited evidence, however, of how effective they are in enabling students to learn key scientific concepts. This paper reports the results of a quantitative study conducted in Australian chemistry classrooms. The visualisations chosen were from free online sources, intended to model the ways in which classroom teachers use visualisations, but were found to have serious flaws for conceptual learning. There were also challenges in the degree of interactivity available to students using the visualisations. Within these limitations, no significant difference was found for teaching with and without these visualisations. Further study using better designed visualisations and with explicit attention to the pedagogy surrounding the visualisations will be required to gather high quality evidence of the effectiveness of visualisations for conceptual development.

Looking back : students' perceptions of the relative enjoyment of primary and secondary school science

This paper reports and discusses a contentious result from an Australia-wide study of the influences on students' decisions about taking senior science subjects. As part of the Choosing Science study (Lyons and Quinn 2010) 3759 Year 10 students were asked to indicate which stage of their schooling (lower primary, upper primary, lower secondary, middle secondary) they had most enjoyed learning science. Crosstabulations of responses revealed that around 78% of students indicated that they had enjoyed learning science more in secondary than in primary school, and 55% enjoyed it the most during Years 9 and 10. The perception that school science was more enjoyable in high school was also found among students who did not intend taking science in Year 11, though to a lesser extent. These findings are unexpected and significant, challenging the prevailing view that enjoyment of school science steadily declines after primary school. The paper elaborates on the findings and suggests that the different conclusions arrived at by studies in this field may be due to the different methodologies employed.

Reframing Conceptions of Contemporary Literacy Capabilities in Pre-Service Primary Teacher Education

This chapter describes the challenges of integrating new technologies with literacy education in pre-service primary teacher education in Australia. The authors describe the policy context and regulatory mechanisms controlling pre-service education, including a national set of professional standards for graduate teachers, a new national curriculum for school students, the introduction of high stakes national assessment for school students, and the looming threat of decontextualized back-to-the-basics professional entry tests for aspiring teachers. The chapter includes three case studies of the authors’ pedagogical practices that attempt to reframe conceptions of the literacy capabilities of pre-service teachers to reflect the complex and sophisticated requirements of teachers in contemporary schooling. The authors conclude the chapter with a discussion of the implications of these case studies as they illustrate the ways that pre-service teachers can be scaffolded and supported to develop creative capacity and critical awareness of the kinds of literacies required in the digital age despite restrictive regimes.

Elements of Better Assessment for the Improvement of Learning

Building a community of shared practice at the classroom level calls for clarity about the important assessment capabilities and dispositions of teachers, especially when teachers are expected to take a direct focus on learning. In this chapter, we present new ways of thinking about teachers’ assessment literacies, offering a formulation of better assessment for the improvement of learning, including three elements, namely (i) assessment criteria and standards; (ii) the teacher’s professional judgment; and (iii) social moderation. The potential of the first element lies in teachers’ classroom practices that deliberately embed assessment criteria and standards in pedagogy in productive ways. The second element involves the engagement of teachers and students in judgment practice, that develops the understanding that judgment involves more than the application of explicit or stated criteria. More fundamental is the matter of how teachers bring to bear stated features of quality and other intellectual and experiential resources in arriving at judgment. That is to say, they range across and orient to explicit (stated), tacit (unstated) and meta-criteria in judgment making. These insights have direct relevance to teachers’ efforts to develop students’ own evaluative experience, especially as this involves students working with stated features of quality for self-assessment and peer-assessment purposes. Further, practices for social moderation are discussed, giving examples of good practice in moderation, how teachers experience moderation and the potential benefits of various types.