Kaleidoscope : Reframing learning and teaching evaluation through a shared stakeholder lens

Sector wide interest in Reframe: QUT’s Evaluation Framework continues with a number of institutions requesting finer details as QUT embeds the new approach to evaluation across the university in 2013. This interest, both nationally and internationally has warranted QUT’s collegial response to draw upon its experiences from developing Reframe into distilling and offering Kaleidoscope back to the sector. The word Reframe is a relevant reference for QUT’s specific re-evaluation, reframing and adoption of a new approach to evaluation; whereas Kaleidoscope reflects the unique lens through which any other institution will need to view their own cultural specificity and local context through an extensive user-led stakeholder engagement approach when introducing new approaches to learning and teaching evaluation. Kaleidoscope’s objectives are for QUT to develop its research-based stakeholder approach to distil the successful experience exhibited in the Reframe Project into a transferable set of guidelines for use by other tertiary institutions across the sector. These guidelines will assist others to design, develop, and deploy, their own culturally specific widespread organisational change informed by stakeholder engagement and organisational buy-in. It is intended that these guidelines will promote, support and enable other tertiary institutions to embark on their own evaluation projects and maximise impact. Kaleidoscope offers an institutional case study of widespread organisational change underpinned by Reframe’s (i) evidence-based methodology; (ii) research including published environmental scan, literature review (Alderman, et al., 2012), development of a conceptual model (Alderman, et al., in press 2013), project management principles (Alderman & Melanie, 2012) and national conference peer reviews; and (iii) year-long strategic project with national outreach to collaboratively engage the development of a draft set of National Guidelines. Kaleidoscope’s aims are to inform Higher Education evaluation policy development through national stakeholder engagement, the finalisation of proposed National Guidelines. In correlation with the conference paper, the authors will present a Draft Guidelines and Framework ready for external peer review by evaluation practitioners from the Higher Education sector, as part of Kaleidoscope’s dissemination strategy (Hinton & Gannaway, 2011) applying illuminative evaluation theory (Parlett & Hamilton, 1976), through conference workshops and ongoing discussions (Shapiro, et al., 1983; Jacobs, 2000). The initial National Guidelines will be distilled from the Reframe: QUT’s Evaluation Framework’s Policy, Protocols, and incorporated Business Rules. It is intended that the outcomes of Kaleidoscope are owned by and reflect sectoral engagement, including iterative evaluation through multiple avenues of dissemination and collaboration including the Higher Education sector. The dissemination strategy with the inclusion of Illuminative Evaluation methodology provides an inclusive opportunity for other institutions and stakeholders across the Higher Education sector to give voice through the information-gathering component of evaluating the draft Guidelines, providing a comprehensive understanding of the complex realities experienced across the Higher Education sector, and thereby ‘illuminating’ both the shared and unique lenses and contexts. This process will enable any final guidelines developed to have broader applicability, greater acceptance, enhanced sustainability and additional relevance benefiting the Higher Education sector, and the adoption and adaption by any single institution for their local contexts.

Framing student evaluations of university learning and teaching : discursive strategies and textual outcomes

Evaluation in higher education is an evolving social practice, that is, it involves what people, institutions and broader systems do and say, how they do and say it, what they value, the effects of these practices and values, and how meanings are ascribed. The textual products (verbal, written, visual, gestural) that inform and are produced by, for and through evaluative practices are important as they promulgate particular kinds of meanings and values in specific contexts. This paper reports on an exploratory study that sought to investigate, using discourse analysis, the types of evaluative practices that were ascribed value, and the student responses that ensued, in different evaluative instruments. Findings indicate that when a reflective approach is taken to evaluation, students’ responses are more considered, they interrogate their own engagement in the learning context and they are more likely to demonstrate reconstructive thought. These findings have implications for reframing evaluation as reflective learning.

The paradox of creative uncertainty in learning and teaching

In an ever-changing and globalised world there is a need for higher education to adapt and evolve its models of learning and teaching. The old industrial model has lost traction, and new patterns of creative engagement are required. These new models potentially increase relevancy and better equip students for the future. Although creativity is recognised as an attribute that can contribute much to the development of these pedagogies, and creativity is valued by universities as a graduate capability, some educators understandably struggle to translate this vision into practice. This paper reports on selected survey findings from a mixed methods research project which aimed to shed light on how creativity can be designed for in higher education learning and teaching settings. A social constructivist epistemology underpinned the research and data was gathered using survey and case study methods. Descriptive statistical methods and informed grounded theory were employed for the analysis reported here. The findings confirm that creativity is valued for its contribution to the development of students’ academic work, employment opportunities and life in general; however, tensions arise between individual educator’s creative pedagogical goals and the provision of institutional support for implementation of those objectives. Designing for creativity becomes, paradoxically, a matter of navigating and limiting complexity and uncertainty, while simultaneously designing for those same states or qualities.

A case study of teaching English and multimodality with ICTs: Constraints and possibilities

Current Australian policies and curricular frameworks demand that teachers and students use technology creatively and meaningfully in classrooms to develop students into 21C technological citizens. English teachers and students also have to learn new metalanguage around visual grammar since multimodal tasks often combine creative with critical General Capabilities (GC) with that of the of ICTs and literacy in the Australian Curriculum: English (AC:E). Both teachers and learners come to these tasks with varying degrees of techno-literacy, skills and access to technologies. This paper reports on case-study research following a technology based collaborative professional development (PD) program between a university Lecturer facilitator and English Teachers in a secondary Catholic school. The study found that the possibilities for creative and critical engagement are rich, but there are real grounded constraints such as lack of time, impeding teachers’ ability to master and teach new technologies in classrooms. Furthermore, pedagogical approaches are affected by technical skill levels and school infrastructure concerns which can militate against effective use of ICTs in school settings. The research project was funded by the Brisbane Catholic Education Office and focused on how teachers can be supported in these endeavours in educational contexts as they prepare students of English to be creative global citizens who use technology creatively.

The regional differences between countries in traffic safety: A cross-cultural study and Turkish Case

Road traffic accidents are a large problem everywhere in the world. However, regional differences in traffic safety between countries are considerable. For example, traffic safety records are much worse in Southern Europe and the Middle East than in Northern and Western Europe. Despite the large regional differences in traffic safety, factors contributing to different accident risk figures in different countries and regions have remained largely unstudied. The general aim of this study was to investigate regional differences in traffic safety between Southern European/Middle Eastern (i.e., Greece, Iran, Turkey) and Northern/Western European (i.e., Finland, Great Britain, The Netherlands) countries and to identify factors related to these differences. We conducted seven sub-studies in which I applied a traffic culture framework, including a multi-level approach, to traffic safety. We used aggregated level data (national statistics), surveys among drivers, and data on traffic accidents and fatalities in the analyses. In the first study, we investigated the influence of macro level factors (i.e., economic, societal, and cultural) on traffic safety across countries. The results showed that a high GNP per capita and conservatism correlated with a low number of traffic fatalities, whereas a high degree of uncertainty avoidance, neuroticism, and egalitarianism correlated with a high number of traffic fatalities. In the second, third, and fourth studies, we examined whether the conceptualisation of road user characteristics (i.e., driver behaviour and performance) varied across traffic cultures and how these factors determined overall safety, and the differences between countries in traffic safety. The results showed that the factorial agreement for driver behaviour (i.e., aggressive driving) and performance (i.e., safety skills) was unsatisfactory in Greece, Iran, and Turkey, where the lack of social tolerance and interpersonal aggressive violations seem to be important characteristics of driving. In addition, we found that driver behaviour (i.e., aggressive violations and errors) mediated the relationship between culture/country and accidents. Besides, drivers from "dangerous" Southern European countries and Iran scored higher on aggressive violations and errors than did drivers from "safe" Northern European countries. However, "speeding" appeared to be a "pan-cultural" problem in traffic. Similarly, aggressive driving seems largely depend on road users' interactions and drivers' interpretation (i.e., cognitive biases) of the behaviour of others in every country involved in the study. Moreover, in all countries, a risky general driving style was mostly related to being young and male. The results of the fifth and sixth studies showed that among young Turkish drivers, gender stereotypes (i.e., masculinity and femininity) greatly influence driver behaviour and performance. Feminine drivers were safety-oriented whereas masculine drivers were skill-oriented and risky drivers. Since everyday driving tasks involve not only erroneous (i.e., risky or dangerous driving) or correct performance (i.e., normal habitual driving), but also "positive" driver behaviours, we developed a reliable scale for measuring "positive" driver behaviours among Turkish drivers in the seventh study. Consequently, I revised Reason's model [Reason, J. T., 1990. Human error. Cambridge University Press: New York] of aberrant driver behaviour to represent a general driving style, including all possible intentional behaviours in traffic while evaluating the differences between countries in traffic safety. The results emphasise the importance of economic, societal and cultural factors, general driving style and skills, which are related to exposure, cognitive biases as well as age, sex, and gender, in differences between countries in traffic safety.

The association of faculty's teaching philosophy and teaching behaviour, and critical thinking and self- direction in learning

The intent in this study was to investigate in what ways teachers· beliefs about education and teaching are expressed in the specific teaching behaviours they employ, and whether teaching behaviours, as perceived by their students, are correlated with students· critical thinking and self-directed learning. To this end the relationships studied were: among faCUlty members· philosophy of teaching, locus of control orientation, psychological type, and observed teaching behaviour; and among students· psychological type, perceptions of teaching behaviour, self-directed learning readiness, and critical thinking. The overall purpose of the study was to investigate whether the implicit goals of higher education, critical thinking and self-direction, were actually accounted for in the university classroom. The research was set within the context of path-goal theory, adapted from the leadership literature. Within this framework, Mezirow·s work on transformative learning, including the influences of Habermas· writings, was integrated to develop a theoretical perspective upon which to base the research methodology. Both qualitative and quantitative methodologies were incorporated. Four faCUlty and a total of 142 students participated in the study. Philosophy of teaching was described through faCUlty interviews and completion of a repertory grid. Faculty completed a descriptive locus of control scale, and a psychological type test. Observations of their teaching behaviour were conducted. Students completed a Teaching Behaviour Assessment Scale, the Self-Directed Learning Readiness Scale, a psychological type test, and the Watson-Glaser Critical Thinking Appraisal. A small sample of students were interviewed. Follow-up discussions with faculty were used to validate the interview, observation, teaching behaviour, and repertory grid data. Results indicated that some discrepancies existed between faculty's espoused philosophy of teaching and their observed teaching behaviour. Instructors' teaching behaviour, however, was a function of their personal theory of practice. Relationships were found between perceived teaching behaviour and students· self-directed learning and critical thinking, but these varied across situations, as would be predicted from path-goal theory. Psychological type of students and instructor also accounted for some of the variability in the relationships studied. Student psychological type could be shown as a partial predictor of self-directed learning readiness. The results were discussed in terms of theory development and implications for further research and practice.

Calculators, mathematics and young children : a study of six children using calculators as part of the mathematics curriculum during their first two years of school

The thesis investigates the role a calculator can play in the developing number knowledge of three girls and three boys as part of their mathematics program, during their first two years at primary school. Random sampling was used initially to select six girls and six boys from the twenty-four children entering a 1993 prep class. These twelve children were interviewed on entrance to school and based on the performance of the twelve children on the initial interview, a girl and a boy were chosen from the higher, middle and lower achievers to take part in the full study. The class teachers involved were previously participants in the ‘Calculators in Primary Mathematics’ research program and were committed to the use of calculators in their mathematics program. A case study approach using qualitative methods within the activity theory framework is used to collect relevant data and information, an analysis of five interviews with each child and observations of the children in forty-one classroom lessons provides comprehensive data on the children's developing number knowledge during the two years. The analysis questionnaires establishes each teacher's perceptions of the children's number learning at the beginning and end of each year, compares teacher expectations with children's actual performance for the year and compares curriculum expectations with children's actual performance. A teacher interview established reasons for changes in teaching style; teacher expectations; children's number learning; and was used to confirm my research findings. An activity theory framework provides an appropriate means of co-coordinating perspectives within this research to enable a description of the child's number learning within a social environment. This framework allows for highlighting the mediation offered by the calculator supporting the children's number learning in the classroom. Levels of children's developing number knowledge reached when working with a calculator and as a result of calculator use are mapped against the levels recommended in ‘Mathematics in the National Curriculum’ (National Curriculum Council, December 1988), and the Curriculum and Standards Framework: Mathematics (Board of Studies 2000). Findings from this comparison illustrate that the six children's performance in number was enhanced when using a calculator and indicate that on-going development and understanding of number concepts occurred at levels of performance at least two years in advance of curriculum recommendations for the first two years of school.

Action research in higher education : the advancement of university learning and teaching

This thesis aims to contribute to the improvement and advancement of university learning, teaching, and staff development; to integrate educational theory and the practice of university teaching; and to contribute to the establishment of a new, emerging paradigm in higher education. The strategy towards achieving these aims comprises (1) an alternative research methodology in the interpretive, non-positivist paradigm; (2) an integrated framework drawing on a variety of previously unrelated theories to form an alternative model of university education; and (3) reference to the dialectical relationship between educational theory and teaching practice and their integration through action research in higher education. The thesis is not so much a critique of the traditional paradigm and of existing functionalist-structuralist approaches to higher education, but more a development and clarification of an alternative, dialectical, human action approach to higher education. The original contribution of this thesis to the theory and practice of higher education lies in the development (1) of an alternative model of university education based on an integration of previously unrelated domains of theory; (2) of a theoretical model of professional development as action research (the CRASP Model: Critical attitude, Research into teaching, Accountability, Self-evaluation, Professionalism); and (3) of action research projects in higher education. Action research is research by the university teachers themselves into their teaching practice, i.e. into problems of the curriculum and student learning. The case studies included in and appended to this thesis show that in one educational setting at least it was possible to improve and advance university learning and teaching through action research. The evidence for this advancement is provided in a number of previously published case studies compiled in the Appendix.

A cross-national study of teaching and learning in primary science classrooms

This presentation reports on the methodological issues confronting an Australian-German-Taiwanese team planning comparative video ethnographic research into primary science classrooms. The issues that will be canvassed include: the benefits of cross-cultural comparisons in providing perspectives on local practice, the theoretical justifications of such comparisons, selection of cases for comparison and possibilities for claiming cultural representativeness, the planning of appropriate data sets, the different comparative stories offered by different analytical frames, practical issues of communication and data sharing, and issues of entanglement of language and culture in the analysis.

African American and Teaching in Higher Education IS a Course on Race and Culture

Using theoretical applications, the authors present an overview of theories that highlight approaches for teaching culturally sensitive content, personal experiences as educator and colleague in a predominantly white college campus and strategies for addressing culturally insensitive experiences in and outside the classroom. Presenters focus on the recruitment and retention of people of color and stress the need for today's predominantly white institutions to become more knowledgeable, tolerant and sensitive about their environments in an effort to make them more accepting.

Public, Private and University Collaboration: Applying a Retrospective Study of the Central Massachusetts Bio-Medical Cluster, Cluster Project Summary for Mosakowski Institute

Three Clark faculty members are studying the development of the Worcester Biotechnology Cluster for any lessons it might hold for current efforts to catalyze the growth of a sustainable energy/green jobs cluster in Worcester or elsewhere. Mary Ellen Boyle (GSOM), Jennie Stephens (IDCE/ESP), and Jing Zhang (GSOM) have conducted extensive in-depth interviews and combed the literature of cluster development to produce several articles and working papers.