REFRAME : a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Incongruence between self-reported and observed senior physical education teaching styles : an analysis using Mosston and Ashworth’s spectrum

Pedagogical styles, methods, models, practices or strategies are valued for what they claim they can achieve. In recent times curriculum documents and governments have called for a range of teaching approaches to meet the variety of learner differences and allow students to make more independent decision making in physical education (Hardy and Mawer, 1999). One well known system of categorizing teaching styles is the Mosston and Ashworth’s Spectrum of Teaching Styles (2002). In Queensland, prior to 2005, no research had been conducted on the teaching styles used by teachers of Physical Education. However, many teachers self-reported that they employed a variety of teaching styles depending on the aims and content of the material to be taught (Cothran, et al., 2005). This research, for the first time, collected teacher’s self-reported use of teaching styles and through observations verify the styles that were being used to teach Senior Physical Education in Queensland. More specifically the aims of the research were to determine: a) What teaching styles teachers of Senior Physical Education in Queensland believe they use? i) Were they using a range of teaching styles? ii) Were teachers of Senior Physical Education in Queensland using teaching styles that the Queensland Senior Physical Education Syllabus (2004) required? b) If Mosston and Ashworth’s (2002) Spectrum of Teaching Styles were used to categorise styles observed during the teaching of Senior Physical Education did the styles being used provide opportunities for evaluating as described by the Queensland Senior Physical Education Syllabus (2004)? The research was conducted in two phases. Part A involved use of a questionnaire to determine the teaching styles Queensland teachers of Senior Physical Education reported using and how often they reported using them. The questionnaire was administered to 110 teachers throughout Queensland. The sample was determined from 346 schools teaching Senior Physical Education (in 2006) across the state of Queensland, Australia. 286 questionnaires were sent to 77 non-randomised schools. There were 66 male and 44 female respondents in the sample. A wide range of teaching styles were reportedly used by teachers of Senior Physical Education with Practice Style-Style B, Command Style-Style A and Divergent Discovery Style-Style H, the most reportedly used. The Self-Teaching Style-Style K was reportedly used the least by teachers involved in this study. From the respondents a group of teachers were identified to form the participants for Part B. Part B of the study involved observation of a group of volunteer participants (from those who had completed the questionnaire) who displayed many of the ‘typical’ characteristics, and a cross-section of backgrounds, of teachers of Senior Physical Education in Queensland. In the case of this study, the criteria used to select the group of teachers to be observed teaching were, teaching experience (number of years: 0-4, 5-10 and 11 years and over), gender, geographical location of schools (focused on Brisbane and near area for travel/access purposes), profile of the students at schools (girls, boys or co-educational), nature of school (Government or Private) and the physical activities being taught in a school (activities to reflect all the areas of physical activity outlined within the syllabus). A total of 27 questionnaire respondents from Part A indicated that they were willing to be observed teaching practical lessons. The respondents who volunteered to be involved in Part B of the study came from different regions across the state of Queensland and was not confined to the Brisbane metropolitan area or large cities. From the group of people who volunteered for Part B four came from outside Brisbane and 23 from the Brisbane area. The final observation group of nine participants included eight teachers from the Brisbane area and one from a rural area. The characteristics of the final group included three females and six males from private and public schools with a range of teaching experience in years and a range of physical activities. Four year 12 and five year 11 teachers and their classes were videoed on three occasions as they progressed through an eight – nine week unit of work. This resulted in 24 hours 48 minutes and 20 seconds (or 4465 observations) of video teaching data which was subsequently coded by several researchers (99% interobserver reliability) to determine the teaching styles employed by the participants. This research indicated that, based on Mosston and Ashworth’s (2002) Spectrum of Teaching Styles, teachers of Senior Physical Education in Queensland used predominantly one style to teach 27 observed lessons. This is in sharp contrast to the variety of styles 110 teachers self- reportedly used and in spite of the Queensland Senior Physical Education Syllabus (2004) suggesting a range of specific styles be used. These results are discussed in the context of the Queensland Senior Physical Education Syllabus (2004), teacher knowledge of teaching styles and high-stakes curriculum and external pressures such as national testing and the publication of data from schools in tabloid newspapers. The data and findings in this research provide a rationale for improving teacher knowledge regarding teaching styles and the need for a clear definition of terminology in syllabus documents. Careful examination of the effects that the publishing of school data may have on teaching styles is advised. This research not only collected teacher’s perceptions of the teaching styles they believed they used it also verified these claims through direct observations of the teachers while teaching. These findings are relevant to syllabus writers, teacher educators, policy makers within education and teachers.

(Re)-regulating care : employing foreign carers for older persons in Taiwan

The purpose of this research was to develop a theoretical understanding of the social phenomenon of the employment of foreign carers for older Taiwanese in households. Foreign carers were introduced into Taiwan in 1992 to address the care needs of the older population. By 2012, over 200,000 foreign caregivers from Indonesia, Philippines, and Vietnam were providing care in households in Taiwan. There has been little research on the interactions between and experiences of family employers, foreign carers and older persons receiving care. The theoretical framework brought together symbolic interactionist concepts and the social constructionism of Berger and Luckmann. Data collection and analysis were informed by Charmaz‘s formulation of grounded theory. Two focus groups and 54 in-depth interviews with a total of 57 Indonesian and Vietnamese foreign carers, Taiwanese family employers and older persons receiving care were undertaken. The analytical findings of the research reflect the ways in which the foreign carer, older persons receiving care and family employer participants were socially situated within the research context and how their respective social realities were shaped differently by changing social structures and cultural values within a globalising context. (Re)-regulating care was generated as the core category, forming a coherent and overarching framework that integrated the three analytical dimensions of the reality of the social change, resituating roles and struggling for control. The reality of social change refers to the employment of foreign carers as a manifestation of the reshaping of the social worlds of the three groups of participants. Resituating roles reflects the processes that underpin the hierarchical positioning of participants, the resultant asymmetrical power relations and associated interactions. Struggling for control, depicts how each group employed strategies to create space and identities that would sustain a sense of self and autonomy. In the current situation of economic and social change in Taiwan the three participant groups shared a desire for control. The autonomy of the women employers was negotiated through employment of foreign carers; for the foreign carers, a pragmatic decision to work abroad became a means for personal empowerment; and the older persons receiving care regained some authority through relationships with carers.

Vocational education in science, technology, engineering and maths (STEM) : Curriculum innovation through school industry partnerships

Governments have recognised that the technological trades rely on knowledge embedded traditionally in science, technology, engineering and mathematics (STEM) disciplines. However, there is substantial evidence that students are turning away from these subjects in schools because the school curriculum is seen as irrelevant, with clear implications for not just vocational education but also higher education. In this paper, we report preliminary findings on the development of two curricula that attempt to integrate science and mathematics with workplace knowledge and practices. We argue that these curricula provide educational opportunities for students to pursue their preferred career pathways. These curricula were co-developed by industry and educational personnel across three industry sectors, namely, mining industry, aerospace and wine tourism. The aim was to provide knowledge appropriate for students moving from school to the workplace as trade apprentices in the respective industries. The analysis of curriculum and associated policy documents reveals that the curricula adopt applied learning orientations through teaching strategies and assessment practices which focus on practical skills. However, although key theoretical science and maths concepts have been well incorporated, the extent to which knowledge deriving from workplace practices is included varies across the curricula. The extent to which applications of concepts are included in the models depends on a number of factors not least the relevant expertise of the teacher as a practitioner in the industry. Our findings highlight the importance of teachers having substantial practical industry experience and the role that whole school policies play in attempts to align the range of learning experiences with the needs of industry.

How can Australia sustain middle school education?

Middle school is a crucial area of education where adolescents experiencing physiological and psychological changes, and require expert guidance. As more research evidence is provided about adolescent learning, teachers are considered pivotal to adolescents’ educational development. Reform measures need to be targeted at the inservice and preservice teacher levels. This quantitative study employs a 40-item, five part Likert scale survey to understand preservice teachers’ (n = 142) perceptions of their confidence to teach in a middle school at the conclusion of their tertiary education. The survey instrument was developed from the literature, with connections to the Queensland College of Teachers' professional standards. Results indicated that they perceived themselves as capable of creating a positive classroom environment with seven items greater than 80%, except with behaviour management (< 80% for two items), and they considered their pedagogical knowledge to be adequate (i.e., 7 out of 8 items > 84%). Items associated with implementing a middle school curriculum had varied responses (e.g., implementing literacy and numeracy were 74%, while implementing learning with real world connections was 91%). This information may assist coursework designers. For example, if a significant percentages of preservice teachers indicate that they believe they were not well prepared for assessment and reporting at the middle school level, then course designers can target these areas more effectively.

How to meet stakeholders' expectations, not manage them?

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

REFRAME: a new approach to evaluation in higher education

Evaluation practices in the higher education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology began a project, entitled REFRAME, to review its approach to evaluation, particularly the student survey system it had been using for the past five years. This presentation will outline the scholarly, evidence based methodology used to undertake institution-wide change, meet the needs of stakeholders suitable to the cultural needs of the institution. It is believed that this approach is broadly applicable to other institutions contemplating change with regard to evaluation of learning and teaching.

Reconceptualizing statistical learning in the early years

This chapter argues for the need to restructure children’s statistical experiences from the beginning years of formal schooling. The ability to understand and apply statistical reasoning is paramount across all walks of life, as seen in the variety of graphs, tables, diagrams, and other data representations requiring interpretation. Young children are immersed in our data-driven society, with early access to computer technology and daily exposure to the mass media. With the rate of data proliferation have come increased calls for advancing children’s statistical reasoning abilities, commencing with the earliest years of schooling (e.g., Langrall et al. 2008; Lehrer and Schauble 2005; Shaughnessy 2010; Whitin and Whitin 2011). Several articles (e.g., Franklin and Garfield 2006; Langrall et al. 2008) and policy documents (e.g., National Council of Teachers ofMathematics 2006) have highlighted the need for a renewed focus on this component of early mathematics learning, with children working mathematically and scientifically in dealing with realworld data. One approach to this component in the beginning school years is through data modelling (English 2010; Lehrer and Romberg 1996; Lehrer and Schauble 2000, 2007)...

Perspectives on Reconceptualizing Early Mathematics Learning

This introductory section provides an overview of the different perspectives on reconceptualizing early mathematics learning. The chapters provide a broad scope in their topics and approaches to advancing young children’s mathematical learning. They incorporate studies that highlight the importance of pattern and structure across the curriculum, studies that target particular content such as statistics, early algebra, and beginning number, and studies that consider how technology and other tools can facilitate early mathematical development. Reconceptualizing the professional learning of teachers in promoting young children’s mathematics, including a consideration of the role of play, is also addressed. Although these themes are diffused throughout the chapters, we restrict our introduction to the core focus of each of the chapters.

Persistent robot tasking for environmental monitoring through crowd-sourcing

Persistent monitoring of the ocean is not optimally accomplished by repeatedly executing a fixed path in a fixed location. The ocean is dynamic, and so should the executed paths to monitor and observe it. An open question merging autonomy and optimal sampling is how and when to alter a path/decision, yet achieve desired science objectives. Additionally, many marine robotic deployments can last multiple weeks to months; making it very difficult for individuals to continuously monitor and retask them as needed. This problem becomes increasingly more complex when multiple platforms are operating simultaneously. There is a need for monitoring and adaptation of the robotic fleet via teams of scientists working in shifts; crowds are ideal for this task. In this paper, we present a novel application of crowd-sourcing to extend the autonomy of persistent-monitoring vehicles to enable nonrepetitious sampling over long periods of time. We present a framework that enables the control of a marine robot by anybody with an internet-enabled device. Voters are provided current vehicle location, gathered science data and predicted ocean features through the associated decision support system. Results are included from a simulated implementation of our system on a Wave Glider operating in Monterey Bay with the science objective to maximize the sum of observed nitrate values collected.

Reimagining school libraries : emerging teacher pedagogic practices

Agile learning spaces have the potential to afford flexible and innovative pedagogic practice. However there is little known about the experiences of teachers and learners in newly designed learning spaces, and whether the potential for reimagined pedagogies is being realised. This paper uses data from a recent study into the experiences of teacher-librarians, teachers, students and leaders of seven Queensland school libraries built with Building the Education Revolution (BER) funding, to explore the question, “how does the physical environment of school libraries influence pedagogic practices?” This paper proposes that teachers explored new pedagogies within the spaces when there was opportunity for flexibility and experimentation and the spaces sufficiently supported their beliefs about student learning. The perspectives of a range of library users were gathered through an innovative research design incorporating student drawings, videoed library tours and reflections, and interviews. The research team collected qualitative data from school libraries throughout 2012. The libraries represented a variety of geographic locations, socioeconomic conditions and both primary and secondary campuses. The use of multiple data sources, and also the perspectives of the multiple researchers who visited the sites and then coded the data, enabled complementary insights and synergies to emerge. Principles of effective teacher learning that can underpin school wide learning about the potential for agile learning spaces to enhance student learning, are identified. The paper concludes that widespread innovative use of the new library spaces was significantly enhanced when the school leadership fostered whole school discussions about the type of learning the spaces might provoke. This research has the potential to inform school designers, teachers and teacher-librarians to make the most of the transformative potential of next generation learning spaces.

Innovation through action research in environmental education : from project to praxis

This thesis is a work-in-progress that articulates my research journey based on the development of a curriculum innovation in environmental education. This journey had two distinct, but intertwined phases: action research based fieldwork, conducted collaboratively, to create a whole school approach to environmental education curriculum planning; and a phase of analysis and reflection based on the emerging findings, as I sought to create personal "living educational theory" about change and innovation. A key stimulus for the study was the perceived theory-practice gap in environmental education, which is often presented in the literature as a criticism of teachers for failing to achieve the values and action objectives of critical environmental education. Hence, many programs and projects are considered to be superficial and inconsequential in terms of their ability to seriously address environmental issues. The intention of this study was to work with teachers in a project that would be an exemplar of critical environmental education. This would be in the form of a whole school "learnscaping" curriculum in a primary school whereby the schoolgrounds would be utilised for interdisciplinary critical environment education. Parallel with the three cycles of action research in this project, my research objectives were to identify and comment upon the factors that influence the generation of successful educational innovation. It was anticipated that the project would be a collaboration involving me, as researcher-facilitator, and many of the teachers in the school as active participants. As the project proceeded through its action cycles, however, it became obvious that the goal of developing a critical environmental education curriculum, and the use of highly participatory processes, were unrealistic. Institutional and organisational rigidities in education generally, teachers' day-to-day work demands, and the constant juggle of work, family and other responsibilities for all participants acted as significant constraints. Consequently, it became apparent that the learnscaping curriculum would not be the hoped-for exemplar. Progress was slow and, at times, the project was in danger of stalling permanently. While the curriculum had some elements of critical environmental education, these were minor and not well spread throughout the school. Overall, the outcome seemed best described as a "small win"; perhaps just another example of the theory-practice gap that I had hoped this project would bridge. Towards the project's end, however, my continuing reflection led to an exploration of chaos/complexity theory which gave new meaning to the concept of a "small win". According to this theory, change is not the product of linear processes applied methodically in purposeful and diligent ways, but emerges from serendipitous events that cannot be planned for, or forecast in advance. When this perspective of change is applied to human organisations - in this study, a busy school - the context for change is recognised not as a stable, predictable environment, but as a highly complex system where change happens all the time, cannot be controlled, and no one can be really sure where the impacts might lead. This so-called "butterfly effect" is a central idea of this theory where small changes or modifications are created - the effects of which are difficult to know, let alone determine - and which can have large-scale impacts. Allied with this effect is the belief that long term developments in an organisation that takes complexity into account, emerge by spontaneous self-organising evolution, requiring political interaction and learning in groups, rather than systematic progress towards predetermined goals or "visions". Hence, because change itself and the contexts of change are recognised as complex, chaos/complexity theory suggests that change is more likely to be slow and evolutionary - cultural change - rather than fast and revolutionary where the old is quickly ushered out by radical reforms and replaced by new structures and processes. Slow, small-scale changes are "normal", from a complexity viewpoint, while rapid, wholesale change is both unlikely and unrealistic. Therefore, the frustratingly slow, small-scale, imperfect educational changes that teachers create - including environmental education initiatives - should be seen for what they really are. They should be recognised as successful changes, the impacts of which cannot be known, but which have the potential to magnify into large-scale changes into the future. Rather than being regarded as failures for not meeting critical education criteria, "small wins" should be cause for celebration and support. The intertwined phases of collaborative action research and individual researcher reflection are mirrored in the thesis structure. The first three chapters, respectively, provide the thesis overview, the literature underpinning the study's central concern, and the research methodology. Chapters 4, 5, and 6 report on each of the three action research cycles of the study, namely Laying the Groundwork, Down to Work!, and The Never-ending Story. Each of these chapters presents a narrative of events, a literature review specific to developments in the cycle, and analysis and critique of the events, processes and outcomes of each cycle. Chapter 7 provides a synthesis of the whole of the study, outlining my interim propositions about facilitating curriculum change in schools through action research, and the implications of these for environmental education.

Recontextualising policy discourses : a Bernsteinian perspective on policy interpretation, translation, enactment

This paper contributes to critical policy research by theorising one aspect of policy enactment, the meaning making work of a cohort of mid-level policy actors. Specifically, we propose that Basil Bernstein’s work on the structuring of pedagogic discourse, in particular, the concept of recontextualisation, may add to understandings of the policy work of interpretation and translation. Recontextualisation refers to the relational processes of selecting and moving knowledge from one context to another, as well as to the distinctive re-organisation of knowledge as an instructional and regulative or moral discourse. Processes of recontextualisation necessitate an analysis of power and control relations, and therefore add to the Foucauldian theorisations of power that currently dominate the critical policy literature. A process of code elaboration (decoding and recoding) takes place in various recontextualising agencies, responsible for the production of professional development materials, teaching guidelines and curriculum resources. We propose that mid-level policy actors are crucial to the work of policy interpretation and translation because they are engaged in elaborating the condensed codes of policy texts to an imagined logic of teachers’ practical work. To illustrate our theoretical points we draw on data; collected for an Australian research project on the accounts of mid-level policy actors responsible for the interpretation of child protection and safety policies for staff in Queensland schools.

Dancing to too many tunes : contemporary course design in teacher education

Contemporary course designers in schools and faculties of Education are finding themselves dancing to many tunes, arguably too many tunes, in order to have their initial teacher education courses accredited by external agencies whilst satisfying internal approval processes and, critically, maintaining the philosophical integrity of their programs and their institutional watermarks. The “tunes” here are the agendas driven by and the demands made by distinct independent agencies. The external agencies influencing Education include: TEQSA (Tertiary Education Quality and Standards Agency) which will assure alignment to the AQF (Australian Qualifications Framework); professional bodies such as AITSL (Australian Institute for Teaching and School Leadership) which now accredits all pre-service teacher Education courses across Australia and assures alignment with the Australian Professional Standards for Teachers; and the state and territory regulatory authorities that have an impact within a specific jurisdiction, for example, the Queensland College of Teachers (QCT) and the Teacher Registration Board of Western Australia (TRBWA). This paper – whose findings have been arrived at through a year-long OLT National Teaching Fellowship - will outline the complex and competing agendas currently at play and focus on the disjuncture evident in the fundamental defining of who is a “graduate.” It will also attempt to identify where there are synergies between the complex demands being made. It will argue that there are too many “tunes” and the task of finding a balance between compliance and delivering effective initial teacher education may not be possible because of the cacophony of their conflicting demands.

Vocational education in science technology, engineering and maths (STEM) : Curriculum innovation through industry school partnerships

Governments have recognised that the technological trades rely on knowledge embedded traditionally in science, technology, engineering and mathematics (STEM) disciplines. In this paper, we report preliminary findings on the development of two curricula that attempt to integrate science and mathematics with workplace knowledge and practices. We argue that these curricula provide educational opportunities for students to pursue their preferred career pathways. These curricula were co-developed by industry and educational personnel across two industry sectors, namely, mining and aerospace. The aim was to provide knowledge appropriate for students moving from school to the workplace in the respective industries. The analysis of curriculum and associated policy documents reveals that the curricula adopt applied learning orientations through teaching strategies and assessment practices which focus on practical skills. However, although key theoretical science and maths concepts have been well incorporated, the extent to which knowledge deriving from workplace practices is included varies across the curricula. Our findings highlight the importance of teachers having substantial practical industry experience and the role that whole school policies play in attempts to align the range of learning experiences with the needs of industry.