Does the summative assessment of real world learning using criterion-referenced assessment need to be discipline specific?

This paper synthesises the existing literature on the contemporary conception of ‘real world’ and compares it with similar notions such as ‘authentic’ and ‘work integrated learning’. While the term ‘real world’ may be partly dependent on the discipline, it does not necessarily follow that the criterion-referenced assessment of ‘real world’ assessment must involve criteria and performance descriptors that are discipline specific. Two examples of summative assessment (court report and trial process exercise) from a final year core subject at the Queensland University of Technology, LWB432 Evidence, emphasise real world learning, are authentic, innovative and better prepare students for the transition into the workplace than more generic forms of assessment such as tutorial participation or oral presentations. The court report requires students to attend a criminal trial in a Queensland Court and complete a two page report on what they saw in practice compared with what they learned in the classroom. The trial process exercise is a 50 minute written closed book activity conducted in tutorials, where students plan questions that they would ask their witness in examination-in-chief, plan questions that they would ask their opponent’s witness in cross-examination, plan questions that they would ask in reexamination given what their opponent asked in cross-examination, and prepare written objections to their opponent’s questions. The trial process exercise simulates the real world, whereas the court report involves observing the real world, and both assessment items are important to the role of counsel. The design of the criterion-referenced assessment rubrics for the court report and trial process exercise is justified by the literature. Notably, the criteria and performance descriptors are not necessarily law specific and this paper highlights the parts that may be easily transferred to other disciplines.

Learning in the third space : a sociocultural perspective on learning with analogies

Research on analogies in science education has focussed on student interpretation of teacher and textbook analogies, psychological aspects of learning with analogies and structured approaches for teaching with analogies. Few studies have investigated how analogies might be pivotal in students’ growing participation in chemical discourse. To study analogies in this way requires a sociocultural perspective on learning that focuses on ways in which language, signs, symbols and practices mediate participation in chemical discourse. This study reports research findings from a teacher-research study of two analogy-writing activities in a chemistry class. The study began with a theoretical model, Third Space, which informed analyses and interpretation of data. Third Space was operationalized into two sub-constructs called Dialogical Interactions and Hybrid Discourses. The aims of this study were to investigate sociocultural aspects of learning chemistry with analogies in order to identify classroom activities where students generate Dialogical Interactions and Hybrid Discourses, and to refine the operationalization of Third Space. These aims were addressed through three research questions. The research questions were studied through an instrumental case study design. The study was conducted in my Year 11 chemistry class at City State High School for the duration of one Semester. Data were generated through a range of data collection methods and analysed through discourse analysis using the Dialogical Interactions and Hybrid Discourse sub-constructs as coding categories. Results indicated that student interactions differed between analogical activities and mathematical problem-solving activities. Specifically, students drew on discourses other than school chemical discourse to construct analogies and their growing participation in chemical discourse was tracked using the Third Space model as an interpretive lens. Results of this study led to modification of the theoretical model adopted at the beginning of the study to a new model called Merged Discourse. Merged Discourse represents the mutual relationship that formed during analogical activities between the Analog Discourse and the Target Discourse. This model can be used for interpreting and analysing classroom discourse centred on analogical activities from sociocultural perspectives. That is, it can be used to code classroom discourse to reveal students’ growing participation with chemical (or scientific) discourse consistent with sociocultural perspectives on learning.

Personalised ontology learning and mining for web information gathering

Over the last decade, the rapid growth and adoption of the World Wide Web has further exacerbated user needs for e±cient mechanisms for information and knowledge location, selection, and retrieval. How to gather useful and meaningful information from the Web becomes challenging to users. The capture of user information needs is key to delivering users' desired information, and user pro¯les can help to capture information needs. However, e®ectively acquiring user pro¯les is di±cult. It is argued that if user background knowledge can be speci¯ed by ontolo- gies, more accurate user pro¯les can be acquired and thus information needs can be captured e®ectively. Web users implicitly possess concept models that are obtained from their experience and education, and use the concept models in information gathering. Prior to this work, much research has attempted to use ontologies to specify user background knowledge and user concept models. However, these works have a drawback in that they cannot move beyond the subsumption of super - and sub-class structure to emphasising the speci¯c se- mantic relations in a single computational model. This has also been a challenge for years in the knowledge engineering community. Thus, using ontologies to represent user concept models and to acquire user pro¯les remains an unsolved problem in personalised Web information gathering and knowledge engineering. In this thesis, an ontology learning and mining model is proposed to acquire user pro¯les for personalised Web information gathering. The proposed compu- tational model emphasises the speci¯c is-a and part-of semantic relations in one computational model. The world knowledge and users' Local Instance Reposito- ries are used to attempt to discover and specify user background knowledge. From a world knowledge base, personalised ontologies are constructed by adopting au- tomatic or semi-automatic techniques to extract user interest concepts, focusing on user information needs. A multidimensional ontology mining method, Speci- ¯city and Exhaustivity, is also introduced in this thesis for analysing the user background knowledge discovered and speci¯ed in user personalised ontologies. The ontology learning and mining model is evaluated by comparing with human- based and state-of-the-art computational models in experiments, using a large, standard data set. The experimental results are promising for evaluation. The proposed ontology learning and mining model in this thesis helps to develop a better understanding of user pro¯le acquisition, thus providing better design of personalised Web information gathering systems. The contributions are increasingly signi¯cant, given both the rapid explosion of Web information in recent years and today's accessibility to the Internet and the full text world.

Digital kids, analogue students : a mixed methods study of students' engagement with a school-based Web 2.0 learning innovation

The inquiry documented in this thesis is located at the nexus of technological innovation and traditional schooling. As we enter the second decade of a new century, few would argue against the increasingly urgent need to integrate digital literacies with traditional academic knowledge. Yet, despite substantial investments from governments and businesses, the adoption and diffusion of contemporary digital tools in formal schooling remain sluggish. To date, research on technology adoption in schools tends to take a deficit perspective of schools and teachers, with the lack of resources and teacher ‘technophobia’ most commonly cited as barriers to digital uptake. Corresponding interventions that focus on increasing funding and upskilling teachers, however, have made little difference to adoption trends in the last decade. Empirical evidence that explicates the cultural and pedagogical complexities of innovation diffusion within long-established conventions of mainstream schooling, particularly from the standpoint of students, is wanting. To address this knowledge gap, this thesis inquires into how students evaluate and account for the constraints and affordances of contemporary digital tools when they engage with them as part of their conventional schooling. It documents the attempted integration of a student-led Web 2.0 learning initiative, known as the Student Media Centre (SMC), into the schooling practices of a long-established, high-performing independent senior boys’ school in urban Australia. The study employed an ‘explanatory’ two-phase research design (Creswell, 2003) that combined complementary quantitative and qualitative methods to achieve both breadth of measurement and richness of characterisation. In the initial quantitative phase, a self-reported questionnaire was administered to the senior school student population to determine adoption trends and predictors of SMC usage (N=481). Measurement constructs included individual learning dispositions (learning and performance goals, cognitive playfulness and personal innovativeness), as well as social and technological variables (peer support, perceived usefulness and ease of use). Incremental predictive models of SMC usage were conducted using Classification and Regression Tree (CART) modelling: (i) individual-level predictors, (ii) individual and social predictors, and (iii) individual, social and technological predictors. Peer support emerged as the best predictor of SMC usage. Other salient predictors include perceived ease of use and usefulness, cognitive playfulness and learning goals. On the whole, an overwhelming proportion of students reported low usage levels, low perceived usefulness and a lack of peer support for engaging with the digital learning initiative. The small minority of frequent users reported having high levels of peer support and robust learning goal orientations, rather than being predominantly driven by performance goals. These findings indicate that tensions around social validation, digital learning and academic performance pressures influence students’ engagement with the Web 2.0 learning initiative. The qualitative phase that followed provided insights into these tensions by shifting the analytics from individual attitudes and behaviours to shared social and cultural reasoning practices that explain students’ engagement with the innovation. Six indepth focus groups, comprising 60 students with different levels of SMC usage, were conducted, audio-recorded and transcribed. Textual data were analysed using Membership Categorisation Analysis. Students’ accounts converged around a key proposition. The Web 2.0 learning initiative was useful-in-principle but useless-in-practice. While students endorsed the usefulness of the SMC for enhancing multimodal engagement, extending peer-topeer networks and acquiring real-world skills, they also called attention to a number of constraints that obfuscated the realisation of these design affordances in practice. These constraints were cast in terms of three binary formulations of social and cultural imperatives at play within the school: (i) ‘cool/uncool’, (ii) ‘dominant staff/compliant student’, and (iii) ‘digital learning/academic performance’. The first formulation foregrounds the social stigma of the SMC among peers and its resultant lack of positive network benefits. The second relates to students’ perception of the school culture as authoritarian and punitive with adverse effects on the very student agency required to drive the innovation. The third points to academic performance pressures in a crowded curriculum with tight timelines. Taken together, findings from both phases of the study provide the following key insights. First, students endorsed the learning affordances of contemporary digital tools such as the SMC for enhancing their current schooling practices. For the majority of students, however, these learning affordances were overshadowed by the performative demands of schooling, both social and academic. The student participants saw engagement with the SMC in-school as distinct from, even oppositional to, the conventional social and academic performance indicators of schooling, namely (i) being ‘cool’ (or at least ‘not uncool’), (ii) sufficiently ‘compliant’, and (iii) achieving good academic grades. Their reasoned response therefore, was simply to resist engagement with the digital learning innovation. Second, a small minority of students seemed dispositionally inclined to negotiate the learning affordances and performance constraints of digital learning and traditional schooling more effectively than others. These students were able to engage more frequently and meaningfully with the SMC in school. Their ability to adapt and traverse seemingly incommensurate social and institutional identities and norms is theorised as cultural agility – a dispositional construct that comprises personal innovativeness, cognitive playfulness and learning goals orientation. The logic then is ‘both and’ rather than ‘either or’ for these individuals with a capacity to accommodate both learning and performance in school, whether in terms of digital engagement and academic excellence, or successful brokerage across multiple social identities and institutional affiliations within the school. In sum, this study takes us beyond the familiar terrain of deficit discourses that tend to blame institutional conservatism, lack of resourcing and teacher resistance for low uptake of digital technologies in schools. It does so by providing an empirical base for the development of a ‘third way’ of theorising technological and pedagogical innovation in schools, one which is more informed by students as critical stakeholders and thus more relevant to the lived culture within the school, and its complex relationship to students’ lives outside of school. It is in this relationship that we find an explanation for how these individuals can, at the one time, be digital kids and analogue students.

Teaching and learning in a context-based chemistry classroom

Many current chemistry programs privilege de-contextualised conceptual learning, often limited by a narrow selection of pedagogies that too often ignore the realities of studentse own lives and interests (e.g., Tytler, 2007). One new approach that offers hope for improving studentse engagement in learning chemistry and perceived relevance of chemistry is the context-based approach. This study investigated how teaching and learning occurred in one year 11 context-based chemistry classroom. Through an interpretive methodology using a case study design, the teaching and learning that occurred during one term (ten weeks) of a unit on Water Quality are described. The researcher was a participant observer in the study who co-designed the unit of work with the teacher. The research questions explored the structure and implementation of the context-based approach, the circumstances by which students connected concepts and context in the context-based classroom and the outcome of the approach for the students and the teacher. A dialectical sociocultural theoretical framework using the dialectics of structure | agency and agency | passivity was used as a lens to explore the interactions between learners in different fields, such as the field of the classroom and the field of the local community. The findings of this study highlight the difficulties teachers face when implementing a new pedagogical approach. Time constraints and opportunities for students to demonstrate a level of conceptual understanding that satisfied the teacher, hindered a full implementation of the approach. The study found that for high (above average) and sound (average) achieving students, connections between sanctioned science content of school curriculum and the studentse out-of-school worlds were realised when students actively engaged in fields that contextualised inquiry and gave them purpose for learning. Fluid transitions or the toing and froing between concepts and contexts occurred when structures in the classroom afforded students the agency to connect concepts and contexts. The implications for teaching by a context-based approach suggest that keeping the context central, by teaching content on a "need-to-know" basis, contextualises the chemistry for students. Also, if teachers provide opportunities for student-student interactions and written work student learning can improve.

How can flexible learning centres use social networking to improve their ongoing educational assessments?

Most online assessment systems now incorporate social networking features, and recent developments in social media spaces include protocols that allow the synchronisation and aggregation of data across multiple user profiles. In light of these advances and the concomitant fear of data sharing in secondary school education this papers provides important research findings about generic features of online social networking, which educators can use to make sound and efficient assessments in collaboration with their students and colleagues. This paper reports on a design experiment in flexible educational settings that challenges the dichotomous legacy of success and failure evident in many assessment activities for at-risk youth. Combining social networking practices with the sociology of education the paper proposes that assessment activities are best understood as a negotiable field of exchange. In this design experiment students, peers and educators engage in explicit, "front-end" assessment (Wyatt-Smith, 2008) to translate digital artefacts into institutional, and potentiality economic capital without continually referring to paper based pre-set criteria. This approach invites students and educators to use social networking functions to assess “work in progress” and final submissions in collaboration, and in doing so assessors refine their evaluative expertise and negotiate the value of student’s work from which new criteria can emerge. The mobile advantages of web-based technologies aggregate, externalise and democratise this transparent assessment model for most, if not all, student work that can be digitally represented.

MiLK. Mobile Learning Kit

The Mobile Learning Kit is a new digital learning application that allows students and teachers to compose, publish, discuss and evaluate their own mobile learning games and events. The research field was interaction design in the context of mobile learning. The research methodology was primarily design-based supported by collaboration between participating disciplines of game design, education and information technology. As such, the resulting MiLK application is a synthesis of current pedagogical models and experimental interaction design techniques and technologies. MiLK is a dynamic learning resource for incorporating both formal and informal teaching and learning practices while exploiting mobile phones and contemporary digital social tools in innovative ways. MiLK explicitly addresses other predominant themes in educational scholarship that relate to current education innovation and reform such as personalised learning, life-long learning and new learning spaces. The success of this project is indicated through rigorous trials and actual uptake of MiLK by international participants in Australia, UK, US and South Africa. MiLK was awarded for excellence in the use of emerging technologies for improved learning and teaching as a finalist (top 3) in the Handheld Learning and Innovation Awards in the UK in 2008. MiLK was awarded funding from the Australasian CRC for Interaction Design in 2008 to prepare the MiLK application for development. MiLK has been awarded over $230,000 from ACID since 2006. The resulting application and research materials are now being commercialised by a new company, ‘ACID Services’.

Curriculum design innovation in flexible science teaching

In this paper you will be introduced to a number of principles which can be used to inform good teaching practice and rigorous curriculum design. Principles relate to: * Application of a common sequence of events for how learners learn; * Accommodating different learning styles; * Adopting a purposeful approach to teaching and learning; * Using assessment as a central driving force in the curriculum and as an organising structure leading to coherence of teaching and learning approach; and * The increasing emphasis that is being placed on the development of generic graduate competencies over and above discipline content knowledge. The principles are particularly significant in relation to adult learning. The paper will use three specific applications as illustrations to help you to learn how these principles can be applied. The illustrations are taken from a second year subject in supercomputing that uses scientific case studies. The subject has been developed (with support from Silicon Graphics Inc. and Intel) to be taught entirely via the Internet.

Towards an intelligent learning system for the natural born cyborg

We propose to design a Custom Learning System that responds to the unique needs and potentials of individual students, regardless of their location, abilities, attitudes, and circumstances. This project is intentionally provocative and future-looking but it is not unrealistic or unfeasible. We propose that by combining complex learning databases with a learner’s personal data, we could provide all students with a personal, customizable, and flexible education. This paper presents the initial research undertaken for this project of which the main challenges were to broadly map the complex web of data available, to identify what logic models are required to make the data meaningful for learning, and to translate this knowledge into simple and easy-to-use interfaces. The ultimate outcome of this research will be a series of candidate user interfaces and a broad system logic model for a new smart system for personalized learning. This project is student-centered, not techno-centric, aiming to deliver innovative solutions for learners and schools. It is deliberately future-looking, allowing us to ask questions that take us beyond the limitations of today to motivate new demands on technology.

Milk : students building mobile learning games in everyday places

This paper explores how mobile games can transform everyday places into dynamic learning spaces filled with information and inspiration. It discusses the motivation inherent in playing games and creating games for others, and how this stimulates an iterative process of creation and reflection and evokes a natural desire to engage in learning. The use of MiLK at the Adelaide Botanic Gardens is offered as a case in point. MiLK is an authoring tool that allows students and teachers to create and share SMS games for mobile phones. A group of South Australian high school students used MiLK to play a game, create their own games and play each other’s games during a day at the gardens. This paper details the learning processes involved in these activities and how the students reflected on their learning, conducted peer assessment, and engaged in a two-way discussion with their teacher about new technologies and their implications for learning. The paper concludes with a discussion of the needs and requirements of 21st Century learners and how MiLK can support constructivist and connectivist teaching methods that engage learners and may produce an appropriately skilled future workforce.

How can we exploit mobile data to customise learning based on a student’s environmental surroundings?

User-Based intelligent systems are already commonplace in a student’s online digital life. Each time they browse, search, buy, join, comment, play, travel, upload, download, a system collects, analyses and processes data in an effort to customise content and further improve services. This panel session will explore how intelligent systems, particularly those that gather data from mobile devices, can offer new possibilities to assist in the delivery of customised, personal and engaging learning experiences. The value of intelligent systems for education lies in their ability to formulate authentic and complex learner profiles that bring together and systematically integrate a student’s personal world with a formal curriculum framework. As we well know, a mobile device can collect data relating to a student’s interests (gathered from search history, applications and communications), location, surroundings and proximity to others (GPS, Bluetooth). However, what has been less explored is the opportunity for a mobile device to map the movements and activities of a student from moment to moment and over time. This longitudinal data provides a holistic profile of a student, their state and surroundings. Analysing this data may allow us to identify patterns that reveal a student’s learning processes; when and where they work best and for how long. Through revealing a student’s state and surroundings outside of schools hour, this longitudinal data may also highlight opportunities to transform a student’s everyday world into an inventory for learning, punctuating their surroundings with learning recommendations. This would in turn lead to new ways to acknowledge and validate and foster informal learning, making it legitimate within a formal curriculum.

A constraints-led perspective to understanding skill acquisition and game play: A basis for integration of motor learning theory and physical education praxis?

Background: In order to design appropriate environments for performance and learning of movement skills, physical educators need a sound theoretical model of the learner and of processes of learning. In physical education, this type of modelling informs the organization of learning environments and effective and efficient use of practice time. An emerging theoretical framework in motor learning, relevant to physical education, advocates a constraints-led perspective for acquisition of movement skills and game play knowledge. This framework shows how physical educators could use task, performer and environmental constraints to channel acquisition of movement skills and decision making behaviours in learners. From this viewpoint, learners generate specific movement solutions to satisfy the unique combination of constraints imposed on them, a process which can be harnessed during physical education lessons. Purpose: In this paper the aim is to provide an overview of the motor learning approach emanating from the constraints-led perspective, and examine how it can substantiate a platform for a new pedagogical framework in physical education: nonlinear pedagogy. We aim to demonstrate that it is only through theoretically valid and objective empirical work of an applied nature that a conceptually sound nonlinear pedagogy model can continue to evolve and support research in physical education. We present some important implications for designing practices in games lessons, showing how a constraints-led perspective on motor learning could assist physical educators in understanding how to structure learning experiences for learners at different stages, with specific focus on understanding the design of games teaching programmes in physical education, using exemplars from Rugby Union and Cricket. Findings: Research evidence from recent studies examining movement models demonstrates that physical education teachers need a strong understanding of sport performance so that task constraints can be manipulated so that information-movement couplings are maintained in a learning environment that is representative of real performance situations. Physical educators should also understand that movement variability may not necessarily be detrimental to learning and could be an important phenomenon prior to the acquisition of a stable and functional movement pattern. We highlight how the nonlinear pedagogical approach is student-centred and empowers individuals to become active learners via a more hands-off approach to learning. Summary: A constraints-based perspective has the potential to provide physical educators with a framework for understanding how performer, task and environmental constraints shape each individual‟s physical education. Understanding the underlying neurobiological processes present in a constraints-led perspective to skill acquisition and game play can raise awareness of physical educators that teaching is a dynamic 'art' interwoven with the 'science' of motor learning theories.

Using concept mapping to scaffold learning for students who experience learning difficulties in science classes

In order to develop scientific literacy students need the cognitive tools that enable them to read and evaluate science texts. One cognitive tool that has been widely used in science education to aid the development of conceptual understanding is concept mapping. However, it has been found some students experience difficulty with concept map construction. This study reports on the development and evaluation of an instructional sequence that was used to scaffold the concept-mapping process when middle school students who were experiencing difficulty with science learning used concept mapping to summarise a chapter of a science text. In this study individual differences in working memory functioning are suggested as one reason that students experience difficulty with concept map construction. The study was conducted using a design-based research methodology in the school’s learning support centre. The analysis of student work samples collected during the two-year study identified some of the difficulties and benefits associated with the use of scaffolded concept mapping with these students. The observations made during this study highlight the difficulty that some students experience with the use of concept mapping as a means of developing an understanding of science concepts and the amount of instructional support that is required for such understanding to develop. Specifically, the findings of the study support the use of multi-component, multi-modal instructional techniques to facilitate the development of conceptual understanding with students who experience difficulty with science learning. In addition, the important roles of interactive dialogue and metacognition in the development of conceptual understanding are identified.

Scaffolding an EFL (English as a foreign language) ‘effective writing’ class in a hybrid learning community

This study sought to establish and develop innovative instructional procedures, in which scaffolding can be expanded and applied, in order to enhance learning of English as a Foreign Language (EFL) writing skills in an effective hybrid learning community (a combination of face-to-face and online modes of learning) at the university where the researcher is working. Many educational experts still believe that technology has not been harnessed to its potential to meet the new online characteristics and trends. There is also an urgency to reconsider the pedagogical perspectives involved in the utilisation of online learning systems in general and the social interactions within online courses in particular that have been neglected to date. An action research design, conducted in two cycles within a duration of four months, was utilised throughout this study. It was intended not only to achieve a paradigm shift from transmission-absorption to socio-constructivist teaching/learning methodologies but also to inform practice in these technology-rich environments. Five major findings emerged from the study. First, the scaffolding theory has been extended. Two new scaffolding types (i.e., quasi-transcendental scaffolding and transcendental scafolding), two scaffolding aspects (i.e., receptive and productive) and some scaffolding actions (e.g., providing a stimulus, awareness, reminder, or remedy) for EFL writing skills in an effective hybrid learning community have been identified and elaborated on. Second, the EFL ‘Effective Writing’ students used the scaffolds implemented in a hybrid environment to enhance and enrich their learning of writing of English essays. The online activities, conducted after the F2F sessions most of the time, gave students greater opportunities to both reinforce and expand the knowledge they had acquired in the F2F mode. Third, a variety of teaching techniques, different online tasks and discussion topics utilised in the two modes bolstered the students’ interests and engagement in their knowledge construction of how to compose English-language essays. Fourth, through the scaffolded activities, the students learned how to scaffold themselves and thus became independent learners in their future endeavours of constructing knowledge. Fifth, the scaffolding-to-scaffold activities provided the students with knowledge on how to effectively engage in transcendental scaffolding actions and facilitate the learning of English writing skills by less able peers within the learning community. Thus, the findings of this current study extended earlier understandings of scaffolding in an EFL hybrid learning environment and will contribute to the advancement of future ICT-mediated courses in terms of their scaffolding pedagogical aspects.