Learner-focused evaluation cycles : facilitating learning using feedforward, concurrent and feedback evaluation

There is a growing trend to offer students learning opportunities that are flexible, innovative and engaging. As educators embrace student-centred agile teaching and learning methodologies, which require continuous reflection and adaptation, the need to evaluate students’ learning in a timely manner has become more pressing. Conventional evaluation surveys currently dominate the evaluation landscape internationally, despite recognition that they are insufficient to effectively evaluate curriculum and teaching quality. Surveys often: (1) fail to address the issues for which educators need feedback, (2) constrain student voice, (3) have low response rates and (4) occur too late to benefit current students. Consequently, this paper explores principles of effective feedback to propose a framework for learner-focused evaluation. We apply a three-stage control model, involving feedforward, concurrent and feedback evaluation, to investigate the intersection of assessment and evaluation in agile learning environments. We conclude that learner-focused evaluation cycles can be used to guide action so that evaluation is not undertaken simply for the benefit of future offerings, but rather to benefit current students by allowing ‘real-time’ learning activities to be adapted in the moment. As a result, students become co-producers of learning and evaluation becomes a meaningful, responsive dialogue between students and their instructors.

Examining workload models in online and blended teaching

Over the past decade, most Australian universities have moved increasingly towards online course delivery for both undergraduate and graduate programs. In almost all cases, elements of online teaching are part of routine teaching loads. Yet detailed and accurate workload data are not readily available. As a result, institutional policies on academic staff workload are often guided more by untested assumptions about reduction of costs per student unit, rather than being evidence-based, with the result that implementation of new technologies for online teaching has resulted in poorly defined workload expectations. While the academics in this study often revealed a limited understanding of their institutional workload formulas, which in Australia are negotiated between management and the national union through their local branches, the costs of various types of teaching delivery have become a critical issue in a time of increasing student numbers, declining funding, pressures to increase quality and introduce minimum standards of teaching and curriculum, and substantial expenditure on technologies to support e-learning. There have been relatively few studies on the costs associated with workload for online teaching, and even fewer on the more ubiquitous ‘blended’, ‘hybrid’ or ‘flexible’ modes, in which face-to-face teaching is supplemented by online resources and activities. With this in mind the research reported here has attempted to answer the following question: What insights currently inform Australian universities about staff workload when teaching online?

University students’ perspective on blended learning

This research project aimed to explore students’ perspective on an appropriate mix of online and-face-to-face activities in a master’s programme in library and information science at an Australian university. Identifying aspects that students evaluate as supportive, challenging and efficient in their learning is important for the design of an appropriate mix in blended learning courses. Twenty-three master’s students responded to a questionnaire containing 40 open-ended and closed questions. Applying both statistical and content analysis provides a deeper understanding of students’ responses. Students like the flexibility and the convenience of online learning, but also the possibilities of face-to-face interaction with teachers and peers for building personal learning networks. Students expect an equal quality of learning delivery and criticised the quality of online participation and lecture recordings. Blended learning is an approach that supports a range of learning styles and life styles.

Locally weighted learning model predictive control for nonlinear and time varying dynamics

This paper proposes an online learning control system that uses the strategy of Model Predictive Control (MPC) in a model based locally weighted learning framework. The new approach, named Locally Weighted Learning Model Predictive Control (LWL-MPC), is proposed as a solution to learn to control robotic systems with nonlinear and time varying dynamics. This paper demonstrates the capability of LWL-MPC to perform online learning while controlling the joint trajectories of a low cost, three degree of freedom elastic joint robot. The learning performance is investigated in both an initial learning phase, and when the system dynamics change due to a heavy object added to the tool point. The experiment on the real elastic joint robot is presented and LWL-MPC is shown to successfully learn to control the system with and without the object. The results highlight the capability of the learning control system to accommodate the lack of mechanical consistency and linearity in a low cost robot arm.

Combining recommender and reputation systems to produce better online advice

Although recommender systems and reputation systems have quite different theoretical and technical bases, both types of systems have the purpose of providing advice for decision making in e-commerce and online service environments. The similarity in purpose makes it natural to integrate both types of systems in order to produce better online advice, but their difference in theory and implementation makes the integration challenging. In this paper, we propose to use mappings to subjective opinions from values produced by recommender systems as well as from scores produced by reputation systems, and to combine the resulting opinions within the framework of subjective logic.

Online self-supervised multi-instance segmentation of dynamic objects

This paper presents a method for the continuous segmentation of dynamic objects using only a vehicle mounted monocular camera without any prior knowledge of the object’s appearance. Prior work in online static/dynamic segmentation is extended to identify multiple instances of dynamic objects by introducing an unsupervised motion clustering step. These clusters are then used to update a multi-class classifier within a self-supervised framework. In contrast to many tracking-by-detection based methods, our system is able to detect dynamic objects without any prior knowledge of their visual appearance shape or location. Furthermore, the classifier is used to propagate labels of the same object in previous frames, which facilitates the continuous tracking of individual objects based on motion. The proposed system is evaluated using recall and false alarm metrics in addition to a new multi-instance labelled dataset to evaluate the performance of segmenting multiple instances of objects.

Design Minds Tomorrow's Classroom Toolkit

The Design Minds Tomorrow’s Classroom Toolkit was one of six K7-12 secondary school design toolkits commissioned by the State Library of Queensland (SLQ) Asia Pacific Design Library (APDL), to facilitate the delivery of the Stage 1 launch of its Design Minds online platform (www.designminds.org.au) partnership initiative with Queensland Government Arts Queensland and the Smithsonian Cooper-Hewitt National Design Museum, on June 29, 2012. Design Minds toolkits are practical guides, underpinned by a combination of one to three of the Design Minds model phases of ‘Inquire’, ‘Ideate’ and ‘Implement’ (supported by at each stage with structured reflection), to enhance existing school curriculum and empower students with real life design exercises, within the classroom environment. Toolkits directly identify links to Naplan, National Curriculum, C2C and Professional Standards benchmarks, as well as the student capabilities of successful and creative 21st century citizens they seek to engender through design thinking. This toolkit explores, through four distinct exercises, different design tools and ways to approach the future design of environments (classrooms/schools) to facilitate the Reggio Emilia philosophy of learning, while addressing diverse and changing social, cultural, technological and environmental challenges. The Design Minds Tomorrow’s Classroom Toolkit encourages students to explore architecture and interior design, and to think about their (life-long) learning as a product of inspiring interactions with people and the environments around them, and that their potential role in contributing to both delightful and functional design solutions requires a deep understanding of the user experience. More generally, it aims to facilitate awareness in young people, of the role of design in society and the value of design thinking skills in generating strategies to solve basic to complex systemic challenges, as well as to inspire post-secondary pathways and idea generation for education. The toolkit encourages students and teachers to develop sketching, making, communication, presentation and collaboration skills to improve their design process, as well as explore further inquiry (background research) to enhance the ideation exercises. Exercise 1 focuses on the ‘Inquire’ and ‘Ideate’ phases, Exercise 2 on the ‘Inquire’, Exercise 3 builds on ideation skills, and Exercise 4 concentrates on the ‘Implement’ phase. Depending on the intensity of the focus, the unit of work could be developed over a 2-5 week program (approximately 4-10 x 60 minute lessons/workshops) or as smaller workshops treated as discrete learning experiences. The toolkit is available for public download from http://designminds.org.au/tomorrows-classroom/ on the Design Minds website. This toolkit inspired the authorship and facilitation of a 2-day design workshop entitled Learning Environment 2050 at John Paul College, Daisy Hill, Brisbane on the 15-16 August 2013. 120 Grade 7 students and their teachers, under the mentorship of two design academics, 3 QUT design students and a professional architect, as part of a QUT School of Design Project Week community engagement activity, explored the formulation of a participatory design brief for the redesign of the school’s Wesley Precinct (including classrooms, a sustainable farm and recreation areas).

Online teacher education : lessons from the cyber-trenches

This paper provides an overview of ‘lessons learned’ from the author’s decade long involvement in online teaching and learning, including eight years in the development, implementation, teaching and administration of a wholly online Master of Arts in Applied Linguistics coursework degree program, which attracted several hundred students annually from around the world, and has won awards for innovation, including being identified as a ‘flagship’ program during an external review of the university.

Peer assisted learning and generation Y : a case study

In recent years Australian Law Schools have implemented various forms of peer assisted learning or mentoring, including career mentoring by former students of final year students and orientation mentoring or tutoring by later year students of incoming first year students. The focus of these programs therefore is on the transition into or out of law school. There is not always as great an emphasis however, as part of this transition, on the use of law students belonging to the same unit cohort as a learning resource for each other within their degree. This is despite the claimed preference of Generation Y students for collaborative learning environments, authentic learning experiences and the development of marketable workplace skills. In the workplace, be it professional legal practice or otherwise, colleagues rely heavily on each other for information, support and guidance. In the undergraduate law degree at the Queensland University of Technology (‘QUT’) the Torts Student Peer Mentor Program aims to supplement a student’s understanding of the substantive law of torts with the development of life-long skills. As such it has the primary objective, albeit through discussion facilitated by more senior students, of encouraging first year students to develop for themselves the skills they need to be successful both as law students and as legal practitioners. Examples of such skills include those relevant to: preparation for assessment tasks; group work; problem solving, cognition and critical thinking; independent learning; and communication. Significantly, in this way, not only do the mentees benefit from involvement in the program, but the peer mentors, or program facilitators, themselves also benefit from their participation in the real world learning environment the program provides. This paper outlines the development and implementation of the above program, the pedagogy which influenced it, and its impact on student learning experiences

Does usability engineering matters for STEM education?

Technological maturity and the exponential growth of digital applications are contributing to lifestyle changes worldwide. Consequently, learning and teaching is demanding more effective sociotechnical interactions involving emerging technologies, as opposed to traditional, conventional face-to-face learning and teaching approaches. In this context, usability engineering is making significant contributions for improving computer and distance-based learning, both for learners and instructors, which have often been ignored when designing online learning and teaching applications. Usability testing is a central part of the human centered learning approach for developing sustainable STEM education from the socio-technological perspective. Our experiences with usability engineering and the impact of teaching low-cost rapid usability testing methods on knowledge translation from undergraduate to graduate courses to real-world practice (i.e. getting the methods out there in real use) are diverse and multi-modal. Our sample space has been hundreds of trained students who have learned how to do effective usability engineering in real-world situations at higher levels of realism (i.e. fidelity) and at a much lower cost than using traditional fixed usability labs. Furthermore, this low-cost rapid approach to usability engineering has been adopted by many of our graduates who are now managers, CIOs etc and who are using the methods routinely in their organizations in real world applications and scenarios. This knowledge has been used to improve design and implementation of a wide range of applications, including applications designed for teaching and learning.

Enhancing interdisciplinary learning with a learning management system

Interdisciplinary learning is a form of knowledge production that is increasingly being embraced as an educational approach in higher education. A role of information and communication technologies (ICT) is to enhance interdisciplinary learning. Issues surrounding the mix of interdisciplinary pedagogic methodologies and emerging digital technologies are worthy of investigation. In this paper, the authors report the findings of a study that examined student perceptions of an interdisciplinary course on information technology (IT) and visual design that utilized a learning management system. Using questionnaire instrumentation, the authors sought the perceptions of first-year university students enrolled in a newly formed interdisciplinary IT course. Results indicate that ICT-based interdisciplinary learners prefer a self-directed and collaborative instructional modality, as well as teacher presence and interventions in the online environment. The types of student participation can significantly influence how students perceive ICT-based interdisciplinary learning design.

Course evaluation : reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.

The dynamics of expertise acquisition in sport : the role of affective learning design

Objectives The aim of this position paper is to discuss the role of affect in designing learning experiences to enhance expertise acquisition in sport. The design of learning environments and athlete development programmes are predicated on the successful sampling and simulation of competitive performance conditions during practice. This premise is captured by the concept of representative learning design, founded on an ecological dynamics approach to developing skill in sport, and based on the individual-environment relationship. In this paper we discuss how the effective development of expertise in sport could be enhanced by the consideration of affective constraints in the representative design of learning experiences. Conclusions Based on previous theoretical modelling and practical examples we delineate two key principles of Affective Learning Design: (i) the design of emotion-laden learning experiences that effectively simulate the constraints of performance environments in sport; (ii) recognising individualised emotional and coordination tendencies that are associated with different periods of learning. Considering the role of affect in learning environments has clear implications for how sport psychologists, athletes and coaches might collaborate to enhance the acquisition of expertise in sport.

Studio teaching in Australian planning curriculum

Planning studio pedagogy has long been a part of planning education and has recently re-emerged as a topic of investigation. Scholarship has: 1) critically examined the fluctuating popularity of studio teaching and the changing role of studio teaching in contemporary planning curricula in the USA and New Zealand; 2) challenged conceptualizations of the traditional studio and considered how emerging strategies for blended and online learning, and ‘real world engagement’ are producing new modes of studio delivery; 3) considered the benefits and outcomes of studio teaching, and; 4) provided recommendations for teaching practice by critically analysing studio experiences in different contexts (Aitken-Rose & Dixon, 2009; Balassiano, 2011; Balassiano & West, 2012; Balsas, 2012; Dandekar, 2009; Heumann & Wetmore, 1984; Higgins, Thomas & Hollander, 2010; Lang, 1983; Long, 2012; Németh & Long, 2012; Winkler, 2013). Twenty-three universities in Australia offer accredited planning degrees, yet data about the use of studio teaching in planning programs are limited. How, when and why are studio pedagogies used? If it is not a part of the curriculum – why?, and has this had any impact on student outcomes? What are the opportunities and limitations of new models of studio teaching for student, academic, professional and institutional outcomes? This paper presents early ideas from a QUT seed grant on the use of studio teaching in Australian planning education to gain a better understanding of the different roles of studio teaching in planning curricula at a National level and opportunities and challenges for this pedagogical mode in the face of dilemmas facing planning education.

Do e-learning applications click with students globally? A study of the development and adoption of WriteCite.com

This paper traces the process of invention of adoption of WriteCite.com which is an internet based learning tool used by students to format citation lists. A number of theoretical issues relating to development and deployment of online learning tools are addressed in this paper. A significant finding in this paper is that adoption rate of new technologies amongst students, in this case for WriteCite.com, is not immediately global in nature despite the internet's ubiquity and the existence of internationally accepted bibliographic styles. Rather, and in contrast to neo theories of technology internationalisation, early-stage adoption tends to be geo-centric and concentrated in the US. In the case of WriteCite.com, which has not been sponsored by an academic institution, the findings reflect voluntary rates of globa adoption. From an applied perspective, the process of identifying globally contestable markets is relevant to developers of online learning tools.