Assessment by interview and portfolio in a Graduate School Program

Assessment plays an integral role in teaching and learning in Higher Education and teachers have a strong interest in debates and commentaries on assessment as and for learning. In a one-year graduate entry teacher preparation program, the temptation is to emphasize assessment in an attempt to ensure students “cover” everything as part of a robust preparation for the profession. The risk is that, for students, assessment drives curriculum, and time spent in the completion of assignments is no guarantee of either effective learning or authentic preparation for teaching. Interviews as assessment provide an opportunity for a learning experience as well as an authentic task, since students will shortly be interviewing for employment in a “real world” situation. This paper reports on a project experimenting with interview panels as authentic assessment with pre-service early childhood teachers. At the end of their first semester of study, students enrolled in the Graduate Diploma of Education program at the Queensland University of Technology in Australia were required to participate in a panel interview where they were graded by a panel made up of three faculty staff and one undergraduate student enrolled in the four-year Bachelor of Education program. Students and panel members completed a questionnaire on their experience after the interview. Results indicated that both students and staff valued the experience and felt it was authentic. Results are discussed in terms of how the assessment interview and portfolio presentation supports graduating students in their preparation for employment interviews, and how this authentic assessment task has benefits for both students and teaching staff.

Patterns of engagement in workplace learning amongst employees from social sectors

Continuous learning and development has become increasingly important in the information age. However, employees with limited formal education in lower status occupations may be disadvantaged in their opportunities for development, as their jobs tend to require more limited knowledge and skills. In mature age, such workers may be subject to cumulative disadvantage with respect to work related learning and development, as well as negative stereotyping. This thesis concerns work related learning and development from a lifespan development psychology perspective. Development across the lifespan is grounded in biocultural co-constructivism. That is, the reciprocal influences of the individual and environment produce change in the individual. Existing theories and models of adaptive development attempt to explain how developmental resources are allocated across the lifespan. These included the Meta- theory of Selective Optimisation with Compensation, Dual Process Model of Self Regulation, and Developmental Regulation via Optimisation and Primary and Secondary Control. These models were integrated to create the Model of Adaptive Development for Work Related Learning. The Learning and Development Survey (LDS) was constructed to measure the hypothesised processes of adaptive development for work related learning, which were individual goal selection, individual goal engagement, individual goal disengagement, organisational opportunities (selection and engagement), and organisational constraints. Data collection was undertaken in two phases: the pilot study and the main study. The objective of the pilot study was to test the LDS on a target population of 112 employees from a local government organisation. Exploratory factor analysis reduced the pilot version of the survey to 38 items encompassing eight constructs which covered the processes of the model of adaptive development for work related learning. In the main study, the Revised Learning and Development Survey (R-LDS) was administered to another group of 137 employees from the local government organisation, as well as 110 employees from a private healthcare organisation. The purpose of the main study was to validate the R-LDS on two different groups to provide evidence of stability, and compare survey scores according to age and occupational status to determine construct validity. Findings from the main study indicated that only four constructs of the R-LDS were stable, which were organisational opportunities – selection, individual goal engagement, organisational constraints – disengagement and organisational opportunities – engagement. In addition, MANOVA studies revealed that the demographic variables affected organisational opportunities and constraints in the workplace, although individual goal engagement was not influenced by age. The findings from the pilot and main study partially supported the model of adaptive development for work related learning. Given that only four factors displayed adequate reliability in terms of internal consistency and stability, the findings suggest that individual goal selection and individual goal disengagement are less relevant to work related learning and development. Some recent research which emerged during the course of the current study has suggested that individual goal selection and individual goal disengagement are more relevant when goal achievement is impeded by biological constraints such as ageing. However, correlations between the retained factors support the model of adaptive development for work related learning, and represent the role of biocultural co-constructivism in development. Individual goal engagement was positively correlated with both opportunity factors (selection and engagement), while organisational constraints – disengagement was negatively correlated with organisational opportunities – selection. Demographic findings indicated that higher occupational status was associated with more opportunities for development. Age was associated with fewer opportunities or greater constraints for development, especially for lower status workers.

A constraints-led perspective to understanding skill acquisition and game play: A basis for integration of motor learning theory and physical education praxis?

Background: In order to design appropriate environments for performance and learning of movement skills, physical educators need a sound theoretical model of the learner and of processes of learning. In physical education, this type of modelling informs the organization of learning environments and effective and efficient use of practice time. An emerging theoretical framework in motor learning, relevant to physical education, advocates a constraints-led perspective for acquisition of movement skills and game play knowledge. This framework shows how physical educators could use task, performer and environmental constraints to channel acquisition of movement skills and decision making behaviours in learners. From this viewpoint, learners generate specific movement solutions to satisfy the unique combination of constraints imposed on them, a process which can be harnessed during physical education lessons. Purpose: In this paper the aim is to provide an overview of the motor learning approach emanating from the constraints-led perspective, and examine how it can substantiate a platform for a new pedagogical framework in physical education: nonlinear pedagogy. We aim to demonstrate that it is only through theoretically valid and objective empirical work of an applied nature that a conceptually sound nonlinear pedagogy model can continue to evolve and support research in physical education. We present some important implications for designing practices in games lessons, showing how a constraints-led perspective on motor learning could assist physical educators in understanding how to structure learning experiences for learners at different stages, with specific focus on understanding the design of games teaching programmes in physical education, using exemplars from Rugby Union and Cricket. Findings: Research evidence from recent studies examining movement models demonstrates that physical education teachers need a strong understanding of sport performance so that task constraints can be manipulated so that information-movement couplings are maintained in a learning environment that is representative of real performance situations. Physical educators should also understand that movement variability may not necessarily be detrimental to learning and could be an important phenomenon prior to the acquisition of a stable and functional movement pattern. We highlight how the nonlinear pedagogical approach is student-centred and empowers individuals to become active learners via a more hands-off approach to learning. Summary: A constraints-based perspective has the potential to provide physical educators with a framework for understanding how performer, task and environmental constraints shape each individual‟s physical education. Understanding the underlying neurobiological processes present in a constraints-led perspective to skill acquisition and game play can raise awareness of physical educators that teaching is a dynamic 'art' interwoven with the 'science' of motor learning theories.

Teaching the economic way of thinking : a new approach to teaching introductory economics in a business degree

In teaching introductory economics there has been a tendency to put a lot of emphasis on imparting abstract models and technical skills to students (Stilwell, 2005; Voss, Blais, Greens, & Ahwesh, 1986). This model building approach has the merit of preparing the grounding for students 10 pursue further studies in economics. However, in a business degree with only a small proportion of students majoring in economics, such an approach tend to alienate the majority of students transiting from high school in to university. Surveys in Europe and Australia found that students complained about the lack of relevance of economics courses to the real world and the over-reliance of abstract mathematical modelling (Kirman, 2001; Lewis and Norris, 1997; Siegfried & Round, 2000). BSB112 Economics 1 is one of the eight faculty core units in the Faculty of Business at QUT, with over 1000 students in each semester. In semester I 2008, a new approach to teaching this unit was designed aiming to achieve three inter-related objectives: (1) to provide business students with a first insight into economic thinking and language, (2) to integrate economic analysis with current Australian social, environmental and political issues, and (3) to cater for students with a wide range of academic needs. Strategies used to achieve these objectives included writing up a new text which departs from traditional economics textbooks in important ways, integrating students' cultures in teaching and learning activities, and devising a new assessment format to encourage development of research skills and applications rather than reproduction of factual knowledge. This paper will document the strategies used in this teaching innovation, present quantitative and qualitative evidence to evaluate this new approach and suggest ways of further improvement.

Learning approach and perception of learning context in economics education : a causality test

In the study of student learning literature, the traditional view holds that when students are faced with heavy workload, poor teaching, and content that they cannot relate to – important aspects of the learning context, they will more likely utilise the surface approach to learning due to stresses, lack of understanding and lack of perceived relevance of the content (Kreber, 2003; Lizzio, Wilson, & Simons, 2002; Ramdsen, 1989; Ramsden, 1992; Trigwell & Prosser, 1991; Vermunt, 2005). For example, in studies involving health and medical sciences students, courses that utilised student-centred, problem-based approaches to teaching and learning were found to elicit a deeper approach to learning than the teacher-centred, transmissive approach (Patel, Groen, & Norman, 1991; Sadlo & Richardson, 2003). It is generally accepted that the line of causation runs from the learning context (or rather students’ self reported data on the learning context) to students’ learning approaches. That is, it is the learning context as revealed by students’ self-reported data that elicit the associated learning behaviour. However, other research studies also found that the same teaching and learning environment can be perceived differently by different students. In a study of students’ perceptions of assessment requirements, Sambell and McDowell (1998) found that students “are active in the reconstruction of the messages and meanings of assessment” (p. 391), and their interpretations are greatly influenced by their past experiences and motivations. In a qualitative study of Hong Kong tertiary students, Kember (2004) found that students using the surface learning approach reported heavier workload than students using the deep learning approach. According to Kember if students learn by extracting meanings from the content and making connections, they will more likely see the higher order intentions embodied in the content and the high cognitive abilities being assessed. On the other hand, if they rote-learn for the graded task, they fail to see the hierarchical relationship in the content and to connect the information. These rote-learners will tend to see the assessment as requiring memorising and regurgitation of a large amount of unconnected knowledge, which explains why they experience a high workload. Kember (2004) thus postulate that it is the learning approach that influences how students perceive workload. Campbell and her colleagues made a similar observation in their interview study of secondary students’ perceptions of teaching in the same classroom (Campbell et al., 2001). The above discussions suggest that students’ learning approaches can influence their perceptions of assessment demands and other aspects of the learning context such as relevance of content and teaching effectiveness. In other words, perceptions of elements in the teaching and learning context are endogenously determined. This study attempted to investigate the causal relationships at the individual level between learning approaches and perceptions of the learning context in economics education. In this study, students’ learning approaches and their perceptions of the learning context were measured. The elements of the learning context investigated include: teaching effectiveness, workload and content. The authors are aware of existence of other elements of the learning context, such as generic skills, goal clarity and career preparation. These aspects, however, were not within the scope of this present study and were therefore not investigated.

Portfolio assessment

At a time when global uncertainty is paramount and when a new form or re-form of curriculum is emerging – with content displaced by skills and knowledge acquisition by learning - assessment, too, begins to take on a new from or re-form. The focus for assessment has shifted to that which engages and promotes learning as s process rather than an assessment that focuses solely on measuring and reporting learning as product or score. The use of the portfolio for assessment offers the potential for the process and progress – integral to learning - to be included.

Academic rigour in science assessment tasks

The critical problem of student disengagement and underachievement in the middle years of schooling (Years 4 . 9) has focussed attention on the quality of educational programs in schools, in Australia and elsewhere. The loss of enthusiasm for science in the middle years is particularly problematic given the growing demand for science professionals. Reshaping middle years programs has included an emphasis on integrating Information and Communication Technologies (ICTs) and improving assessment practices to engage students in higher cognitive processes and enhance academic rigour. Understanding the nature of academic rigour and how to embed it in students. science assessment tasks that incorporate the use of ICTs could enable teachers to optimise the quality of the learning environment. However, academic rigour is not clearly described or defined in the literature and there is little empirical evidence upon which researchers and teachers could draw to enhance understandings. This study used a collective case study design to explore teachers' understandings of academic rigour within science assessment tasks. The research design is based on a conceptual framework that is underpinned by socio-cultural theory. Three methods were used to collect data from six middle years teachers and their students. These methods were a survey, focus group discussion with teachers and a group of students and individual semi-structured interviews with teachers. Findings of the case study revealed six criteria of academic rigour, namely, higher order thinking, alignment, building on prior knowledge, scaffolding, knowledge construction and creativity. Results showed that the middle years teachers held rich understandings of academic rigour that led to effective utilisation of ICTs in science assessment tasks. Findings also indicated that teachers could further enhance their understandings of academic rigour in some aspects of each of the criteria. In particular, this study found that academic rigour could have been further optimised by: promoting more thoughtful discourse and interaction to foster higher order thinking; increasing alignment between curriculum, pedagogy, and assessment, and students. prior knowledge; placing greater emphasis on identifying, activating and building on prior knowledge; better differentiating the level of scaffolding provided and applying it more judiciously; fostering creativity throughout tasks; enhancing teachers‟ content knowledge and pedagogical content knowledge, and providing more in-depth coverage of fewer topics to support knowledge construction. Key contributions of this study are a definition and a model which clarify the nature of academic rigour.

Culture-fair assessment : challenging Indigenous students through effortful mathematics teaching

This paper reports on a mathematics education research project centred on teachers’ pedagogical practices and capacity to assess Indigenous Australian students in a culture-fair manner. The project has been funded by the Australian Research Council Linkage program and is being conducted in seven Catholic and Independent primary schools in north Queensland. Our Industry Partners are Catholic Education and the Association of Independent Schools, Queensland. The study aims to provide greater understanding about how to build more equitable assessment practices to address the issue of underperforming Aboriginal and Torres Strait Islander (ATSI) students in regional and remote Australia. The goal is to identify ways forward by attending to culture-fair assessment practice. The research is exploring the attitudes, beliefs and responses of Indigenous students to assessment in the context of mathematics learning with particular focus on teacher knowledge in these educational settings in relation to the design of assessment tasks that are authentic and engaging for these students in an accountability context. This approach highlights how teachers need to distinguish the ‘funds of knowledge’ (González, Moll, Floyd Tenery, Rivera, Rendón, Gonzales & Amanti, 2008) that Indigenous students draw on and how teachers need to be culturally responsive in their pedagogy to open up curriculum and assessment practice to allow for different ways of knowing and being

Strategies to engage engineering students in group project work

Project focused group work is significant in developing social and personal skills as well as extending the ability to identify, formulate and solve engineering problems. As a result of increasing undergraduate class sizes, along with the requirement for many students to work part-time, group projects, peer and collaborative learning are seen as a fundamental part of engineering education. Group formation, connection to learning objectives and fairness of assessment has been widely reported as major issues that leave students dissatisfied with group project based units. Several strategies were trialled including a study of formation of groups by different methods across two engineering disciplines over the past 2 years. Other strategies involved a more structured approach to assessment practices of civil and electrical engineering disciplines design units. A confidential online teamwork management tool was used to collect and collate student self and peer assessment ratings and used for both formative feedback as well as assessment purposes. Student satisfaction and overall academic results in these subjects have improved since the introduction of these interventions. Both student and staff feedback highlight this approach as enhancing student engagement and satisfaction, improved student understanding of group roles, reducing number of dysfunctional groups whilst requiring less commitment of academic resources.

Talking the talk : oracy demands in first year university assessment tasks

With more constructivist approaches to learning in higher education and more value on teamwork skills, students’ oracy (speaking and listening) features more prominently in curriculum, pedagogy and assessment. The paper reports on a study of two first-year Australian university courses in disciplines with explicit industry orientations and high proportions of international students. Drawing on classroom observations and interviews with the lecturers, this paper investigates their pedagogical designs on oracy and the oracy demands of their assessment tasks. The study found that talk-based assessment tasks (a group project and a group oral presentation) featured in both courses but the two courses treated students’ oracy differently: as product or process. The contrast between the two assessment designs explicates issues around EAL student needs, authentic links to industry, the provenance of criteria used to assess performance, perceptions about the relevance of talk and the ‘hidden assessment’ of oracy.

An assessment system for evaluation of driving competencies

Automobiles have deeply impacted the way in which we travel but they have also contributed to many deaths and injury due to crashes. A number of reasons for these crashes have been pointed out by researchers. Inexperience has been identified as a contributing factor to road crashes. Driver’s driving abilities also play a vital role in judging the road environment and reacting in-time to avoid any possible collision. Therefore driver’s perceptual and motor skills remain the key factors impacting on road safety. Our failure to understand what is really important for learners, in terms of competent driving, is one of the many challenges for building better training programs. Driver training is one of the interventions aimed at decreasing the number of crashes that involve young drivers. Currently, there is a need to develop comprehensive driver evaluation system that benefits from the advances in Driver Assistance Systems. A multidisciplinary approach is necessary to explain how driving abilities evolves with on-road driving experience. To our knowledge, driver assistance systems have never been comprehensively used in a driver training context to assess the safety aspect of driving. The aim and novelty of this thesis is to develop and evaluate an Intelligent Driver Training System (IDTS) as an automated assessment tool that will help drivers and their trainers to comprehensively view complex driving manoeuvres and potentially provide effective feedback by post processing the data recorded during driving. This system is designed to help driver trainers to accurately evaluate driver performance and has the potential to provide valuable feedback to the drivers. Since driving is dependent on fuzzy inputs from the driver (i.e. approximate distance calculation from the other vehicles, approximate assumption of the other vehicle speed), it is necessary that the evaluation system is based on criteria and rules that handles uncertain and fuzzy characteristics of the driving tasks. Therefore, the proposed IDTS utilizes fuzzy set theory for the assessment of driver performance. The proposed research program focuses on integrating the multi-sensory information acquired from the vehicle, driver and environment to assess driving competencies. After information acquisition, the current research focuses on automated segmentation of the selected manoeuvres from the driving scenario. This leads to the creation of a model that determines a “competency” criterion through the driving performance protocol used by driver trainers (i.e. expert knowledge) to assess drivers. This is achieved by comprehensively evaluating and assessing the data stream acquired from multiple in-vehicle sensors using fuzzy rules and classifying the driving manoeuvres (i.e. overtake, lane change, T-crossing and turn) between low and high competency. The fuzzy rules use parameters such as following distance, gaze depth and scan area, distance with respect to lanes and excessive acceleration or braking during the manoeuvres to assess competency. These rules that identify driving competency were initially designed with the help of expert’s knowledge (i.e. driver trainers). In-order to fine tune these rules and the parameters that define these rules, a driving experiment was conducted to identify the empirical differences between novice and experienced drivers. The results from the driving experiment indicated that significant differences existed between novice and experienced driver, in terms of their gaze pattern and duration, speed, stop time at the T-crossing, lane keeping and the time spent in lanes while performing the selected manoeuvres. These differences were used to refine the fuzzy membership functions and rules that govern the assessments of the driving tasks. Next, this research focused on providing an integrated visual assessment interface to both driver trainers and their trainees. By providing a rich set of interactive graphical interfaces, displaying information about the driving tasks, Intelligent Driver Training System (IDTS) visualisation module has the potential to give empirical feedback to its users. Lastly, the validation of the IDTS system’s assessment was conducted by comparing IDTS objective assessments, for the driving experiment, with the subjective assessments of the driver trainers for particular manoeuvres. Results show that not only IDTS was able to match the subjective assessments made by driver trainers during the driving experiment but also identified some additional driving manoeuvres performed in low competency that were not identified by the driver trainers due to increased mental workload of trainers when assessing multiple variables that constitute driving. The validation of IDTS emphasized the need for an automated assessment tool that can segment the manoeuvres from the driving scenario, further investigate the variables within that manoeuvre to determine the manoeuvre’s competency and provide integrated visualisation regarding the manoeuvre to its users (i.e. trainers and trainees). Through analysis and validation it was shown that IDTS is a useful assistance tool for driver trainers to empirically assess and potentially provide feedback regarding the manoeuvres undertaken by the drivers.

Communicating assessment criteria is not sufficient for influencing students' approaches to assessment tasks – perspectives from a differential equations class

This report presents the findings of an exploratory study into the perceptions held by students regarding the use of criterion-referenced assessment in an undergraduate differential equations class. Students in the class were largely unaware of the concept of criterion referencing and of the various interpretations that this concept has among mathematics educators. Our primary goal was to investigate whether explicitly presenting assessment criteria to students was useful to them and guided them in responding to assessment tasks. Quantitative data and qualitative feedback from students indicates that while students found the criteria easy to understand and useful in informing them as to how they would be graded, the manner in which they actually approached the assessment activity was not altered as a result of the use of explicitly communicated grading criteria.

The link between age, career goals, and adaptive development for work-related learning among local government employees

More recently, lifespan development psychology models of adaptive development have been applied to the workforce to investigate ageing worker and lifespan issues. The current study uses the Learning and Development Survey (LDS) to investigate employee selection and engagement of learning and development goals and opportunities and constraints for learning at work in relation to demographics and career goals. It was found that mature age was associated with perceptions of preferential treatment of younger workers with respect to learning and development. Age was also correlated with several career goals. Findings suggest that younger workers’ learning and development options are better catered for in the workplace. Mature aged workers may compensate for unequal learning opportunities at work by studying for an educational qualification or seeking alternate job opportunities. The desire for a higher level job within the organization or educational qualification was linked to engagement in learning and development goals at work. It is suggested that an understanding of employee perceptions in the workplace in relation to goals and activities may be important in designing strategies to retain workers.

Raising the stakes : the challenges for teacher assessment

These are changing times for teachers and their students in Australia with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to make more use of assessment to support and improve learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the alleged purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are key questions that will be addressed in this presentation with reference to innovative research from global networks that have maintained the assessment focus on learning.