Reframing evaluation of learning and teaching: An approach to change

Evaluation practices in the Higher Education sector have been criticised for having unclear purpose and principles; ignoring the complexity and changing nature of learning and teaching and the environments in which they occur; relying almost exclusively on student ratings of teachers working in classroom settings; lacking reliability and validity; using data for inappropriate purposes; and focusing on accountability and marketing rather than the improvement of learning and teaching. In response to similar criticism from stakeholders, in 2011 Queensland University of Technology (QUT) began a project which aims to reframe the organisation’s approach to the evaluation of learning and teaching. This paper describes the existing evaluation system; the emergence and early development of the project; and formulation of a conceptual framework identifying key dimensions of evaluation. It then compares the draft framework with other conceptualisations and models of evaluation identified in the literature, to determine its validity and suitability for supporting QUT’s plans for the future. Overall, the paper represents a structured evaluation of the REFRAME project at a particular point in its lifecycle. Given that the project follows an evidence based, practice-led process and applies an ongoing action research cycle, the findings are presented in the belief that QUT’s experience is broadly applicable to other institutions which may be contemplating change in relation to evaluation of learning and teaching.

Australian learning and teaching council projects in the mathematical sciences : a retrospective

Since 2004, the Australian Learning and Teaching Council (ALTC) and its predecessor, the Carrick Institute for Learning and Teaching in Higher Education, have funded numerous teaching and educational research-based projects in the Mathematical Sciences. In light of the Commonwealth Government’s decision to close the ALTC in 2011, it is appropriate to take account of the ALTCs input into the Mathe- matical Sciences in higher education. Here we present an overview of ALTC projects in the Mathematical Sciences, as well as report on the contributions they have made to the Discipline.

Learning and teaching reflection : developing capacities for lifelong learning

Reflection is an essential part of being an effective learner and working as a productive teacher. It enables the learner or teacher to deliberate about the factors that lead to successful learning and teaching for them and/or their students, in a particular place and for a specific purpose, so they can make reasoned and effective choices. This chapter introduces important frameworks that cover a century of thinking around reflection in education, and illustrates how preservice teachers can use these ideas across three phases. First, becoming a reflective learner as a university student to enhance learning and assessment outcomes; second, becoming a reflective teacher to improve classroom teaching and learning outcomes; and third, developing the reflective capacities of primary students so they can enhance their skills for lifelong learning.

Establishing virtual production as a research and teaching activity at QUT

Located within the Creative Industries Faculty, the Animation team at the Queensland University of Technology (QUT) recently acquired a full-body inertial motion capture system. Our research to date has been predominantly concerned with interdisciplinary practice and the benefits this could bring to undergraduate teaching. From early experimental tests it was identified that there was a need to develop a framework for best practice and an efficient production workflow to ensure the system was being used to its full potential. Through our ongoing investigation we have identified at least three areas that stand to have long-term benefits from universities engaging in motion capture related research activity. This includes interdisciplinary collaborative research, undergraduate teaching and improved production processes. The following paper reports the early stages of our research, which explores the use of a full-body inertial motion capture (MoCap) solution in collaboration with performing artists.

A model for enhancing assessment and teaching practice at the coalface : Insights from a Fellow-In-Residence engagement program

Australian universities are currently engaging with new governmental policies and regulations that require them to demonstrate enhanced quality and accountability in teaching and research. The development of national academic standards for learning outcomes in higher education is one such instance of this drive for excellence. These discipline-specific standards articulate the minimum, or Threshold Learning Outcomes, to be addressed by higher education institutions so that graduating students can demonstrate their achievement to their institutions, accreditation agencies, and industry recruiters. This impacts not only on the design of Engineering courses (with particular emphasis on pedagogy and assessment), but also on the preparation of academics to engage with these standards and implement them in their day-to-day teaching practice on a micro level. This imperative for enhanced quality and accountability in teaching is also significant at a meso level, for according to the Australian Bureau of Statistics, about 25 per cent of teachers in Australian universities are aged 55 and above and more than 54 per cent are aged 45 and above (ABS, 2006). A number of institutions have undertaken recruitment drives to regenerate and enrich their academic workforce by appointing capacity-building research professors and increasing the numbers of early- and mid-career academics. This nationally driven agenda for quality and accountability in teaching permeates also the micro level of engineering education, since the demand for enhanced academic standards and learning outcomes requires both a strong advocacy for a shift to an authentic, collaborative, outcomes-focused education and the mechanisms to support academics in transforming their professional thinking and practice. Outcomes-focused education means giving greater attention to the ways in which the curriculum design, pedagogy, assessment approaches and teaching activities can most effectively make a positive, verifiable difference to students’ learning. Such education is authentic when it is couched firmly in the realities of learning environments, student and academic staff characteristics, and trustworthy educational research. That education will be richer and more efficient when staff works collaboratively, contributing their knowledge, experience and skills to achieve learning outcomes based on agreed objectives. We know that the school or departmental levels of universities are the most effective loci of changes in approaches to teaching and learning practices in higher education (Knight & Trowler, 2000). Heads of Schools are being increasingly entrusted with more responsibilities - in addition to setting strategic directions and managing the operational and sometimes financial aspects of their school, they are also expected to lead the development and delivery of the teaching, research and other academic activities. Guiding and mentoring individuals and groups of academics is one critical aspect of the Head of School’s role. Yet they do not always have the resources or support to help them mentor staff, especially the more junior academics. In summary, the international trend in undergraduate engineering course accreditation towards the demonstration of attainment of graduate attributes poses new challenges in addressing academic staff development needs and the assessment of learning. This paper will give some insights into the conceptual design, implementation and empirical effectiveness to date, of a Fellow-In-Residence Engagement (FIRE) program. The program is proposed as a model for achieving better engagement of academics with contemporary issues and effectively enhancing their teaching and assessment practices. It will also report on the program’s collaborative approach to working with Heads of Schools to better support academics, especially early-career ones, by utilizing formal and informal mentoring. Further, the paper will discuss possible factors that may assist the achievement of the intended outcomes of such a model, and will examine its contributions to engendering an outcomes-focussed thinking in engineering education.

HLA-DRB1 associations with disease susceptibility and clinical course in Australians with multiple sclerosis

Human leucocyte antigen (HLA)-DRB1*1501 and other class II alleles influence susceptibility to multiple sclerosis (MS), but their contribution if any to the clinical course of MS remains uncertain. Here, we have investigated DRB1 alleles in a large sample of 1230 Australian MS cases, with some enrichment for subjects with primary progressive (PPMS) disease (n = 246) and 1210 healthy controls. Using logistic regression, we found that DRB1*1501 was strongly associated with risk (P = 7 x 10-45), as expected, and after adjusting for DRB1*1501, a predisposing effect was also observed for DRB1*03 (P = 5 x 10-7). Individuals homozygous for either DRB1*15 or DRB1*03 were considerably more at risk of MS than heterozygotes and non-carriers. Both the DRB1*04 and the DRB1*01/DRB1*15 genotype combination, respectively, protected against PPMS in comparison to subjects with relapsing disease. Together, these data provide further evidence of heterogeneity at the DRB1 locus and confirm the importance of HLA variants in the phenotypic expression of MS.

Kaleidoscope : illuminating learning and teaching evaluation for a shared stakeholder lens

Within the Australian higher education sector, institutions are required to evaluate teaching, units and courses to assure the quality of the student learning experience, however with hardly any regulatory parameters guiding institutions, and with disparate practices, there are few opportunities to benchmakr across institutions or the sector. QUT has received interest and requests from national and international universities on accessing Reframe: QUT's Evaluation Framework.

A metastatic neuroendocrine anaplastic small cell tumor in a patient with multiple endocrine neoplasia type 1 syndrome : assessment of disease status and response to doxorubicin, cyclophosphamide, etoposide chemotherapy through scintigraphic imaging with 111In-pentetreotide

Extrapulmonary small cell and small cell neuroendocrine tumors of unknown primary site are, in general, aggressive neoplasms with a short median survival. Like small cell lung cancer (SCLC), they often are responsive to chemotherapy and radiotherapy. Small cell lung cancer and well differentiated neuroendocrine carcinomas of the gastrointestinal tract and pancreas tend to express somatostatin receptors. These tumors may be localized in patients by scintigraphic imaging using radiolabeled somatostatin analogues. A patient with an anaplastic neuroendocrine small cell tumor arising on a background of multiple endocrine neoplasia type 1 syndrome is reported. The patient had a known large pancreatic gastrinoma and previously treated parathyroid adenopathy. At presentation, there was small cell cancer throughout the liver and skeleton. Imaging with a radiolabeled somatostatin analogue, 111In- pentetreotide (Mallinckrodt Medical B. V., Petten, Holland), revealed all sites of disease detected by routine biochemical and radiologic methods. After six cycles of chemotherapy with doxorubicin, cyclophosphamide, and etoposide, there was almost complete clearance of the metastatic disease. 111In-pentetreotide scintigraphy revealed uptake consistent with small areas of residual disease in the liver, the abdomen (in mesenteric lymph nodes), and posterior thorax (in a rib). The primary gastrinoma present before the onset of the anaplastic small cell cancer showed no evidence of response to the treatment. The patient remained well for 1 year and then relapsed with brain, lung, liver, and skeletal metastases. Despite an initial response to salvage radiotherapy and chemotherapy with carboplatin and dacarbazine, the patient died 6 months later.

Objectifying the Subjective : Assessment and Feedback in Creative Arts Studio Learning and Teaching

This paper examines the Assessment and Feedback aspects of Studio Teaching as Creative Arts pedagogy. Prompted by USQ’s newly offered Bachelor of Creative Arts (BCA), the author has developed an Assessment Matrix specifically designed to satisfy a number of imperatives, including: • ‘objectifying’ the subjective aspects of creative practice as assessable coursework/research • providing the means by which accurate, detailed, personalised and confidential feedback may be provided to students individually • providing consistent, accurate, meaningful assessment records for student, lecturer, and institution • ensuring consistency, continuity, and transparency of assessment processes and records to satisfy quality audits • minimising marking and assessment time, whilst maximising assessment integrity and depth • requiring only basic level skills and knowledge of a computer application already in common use (Microsoft Excel) • adaptability to a range of creative courses ‐ across disciplines This Assessment Matrix has been in development (and trialled) since January 2009.

A state-wide systems approach to embedding the learning and teaching of sustainability in teacher education

Education systems have a key role to play in preparing future citizens to engage in sustainable living practices and help create a more sustainable world. Many schools throughout Australia have begun to develop whole-school approaches to sustainability education that are supported by national and state policies and curriculum frameworks. Preservice teacher education, however, lags behind in building the capacity of new teachers to initiate and implement such approaches (ARIES, 2010). This proposed project seeks to develop a state-wide systems approach to embedding Education for Sustainability (EfS) in teacher education that is aligned with the Australian National Curriculum and the aspirations for EfS in the Melbourne Declaration and other national documents. Representatives from all teacher education institutions and other agents of change in the Queensland education system will be engaged in a multilevel systems approach, involving collaboration at the state, institutional and course levels, to develop curriculum practices that reflect a shared vision of EfS.

The mouse model of Chlamydia genital tract infection: a review of infection, disease, immunity and vaccine development

Chlamydia trachomatis is the most common sexually transmitted bacterial infection worldwide. The impact of this pathogen on human reproduction has intensified research efforts to better understand chlamydial infection and pathogenesis. Whilst there are animal models available that mimic the many aspects of human chlamydial infection, the mouse is regarded as the most practical and widely used of the models. Studies in mice have greatly contributed to our understanding of the host-pathogen interaction and provided an excellent medium for evaluating vaccines. Here we explore the advantages and disadvantages of all animal models of chlamydial genital tract infection, with a focus on the murine model and what we have learnt from it so far.

Web based surveillance systems could improve disease detection and the response to emerging disease events

The Chinese government should be commended for its open, concerted, and rapid response to the recent H7N9 influenza outbreak. However, the first known case was not reported until 48 days after disease onset.1 Although the difficulties in detecting the virus and the lack of suitable diagnostic methods have been the focus of discussion,2 systematic limitations that may have contributed to this delay have hardly been discussed. The detection speed of surveillance systems is limited by the highly structured nature of information flow and hierarchical organisation of these systems. Flu surveillance usually relies on notification to a central authority of laboratory confirmed cases or presentations to sentinel practices for flu-like illness. Each step in this pathway presents a bottleneck at which information and time can be lost; this limitation must be dealt with...

Learning and teaching epidemiology with audience response technology

Background The learning and teaching of epidemiology is core to many public health programs. Many students find the content of epidemiology, and specifically risk of bias assessment, challenging to learn. Howbeit, learning is enhanced when knowledge is able to be acquired from an active-learning, hands-on experience. Methods The innovative use of wireless audience response technology “clickers” was incorporated into the lectures of the university’s post-graduate epidemiology units and the tailored epidemiological modules delivered for professional disciplines (e.g. optometry). Clickers were used to apply several pedagogical approaches of active learning including peer-instruction and real-world simulation. Students were also assessed for their gain in knowledge within the lecture (pre-post) and their perceptions of how the use of clickers helped them learn. The routine university-wide end of semester Insight Survey provided further information of the student’s satisfaction with the approach. Results The technology was useful in identifying deficits of knowledge of key concepts either before or after instruction. Where key concepts were re-tested post-lecture, as expected, knowledge increased significantly and provided immediate feed-back to students. Across the lecture series, typically 85% of students identified the technology helped them learn, increased their opportunity to interact with the lecturer, and recommend their use for future classes. The Insight Survey report identified 93% of respondents identified the unit in which clickers were consistently used provided good learning opportunities. Numerous student comments supported the teaching method. Conclusions Epidemiological subject matter lends itself to incorporation of audience response technology. The use of the technology to facilitate interactive voting provides an instant response and participation of everyone to enhance the classroom experience. The pedagogical approach increases students’ knowledge and increases their satisfaction with the unit.