Evaluation of the year 1 literacy and numeracy checkpoints assessments trial - 2010

The draft Year 1 Literacy and Numeracy Checkpoints Assessments were in open and supported trial during Semester 2, 2010. The purpose of these trials was to evaluate the Year 1 Literacy and Numeracy Checkpoints Assessments (hereafter the Year 1 Checkpoints) that were designed in 2009 as a way to incorporate the use of the Year 1 Literacy and Numeracy Indicators as formative assessment in Year 1 in Queensland Schools. In these trials there were no mandated reporting requirements. The processes of assessment were related to future teaching decisions. As such the trials were trials of materials and the processes of using those materials to assess students, plan and teach in year 1 classrooms. In their current form the Year 1 Checkpoints provide assessment resources for teachers to use in February, June and October. They aim to support teachers in monitoring children's progress and making judgments about their achievement of the targeted P‐3 Literacy and Numeracy Indicators by the end of Year 1 (Queensland Studies Authority, 2010 p. 1). The Year 1 Checkpoints include support materials for teachers and administrators, an introductory statement on assessment, work samples, and a Data Analysis Assessment Record (DAAR) to record student performance. The Supported Trial participants were also supported with face‐to‐face and on‐line training sessions, involvement in a moderation process after the October Assessments, opportunities to participate in discussion forums as well as additional readings and materials. The assessment resources aim to use effective early years assessment practices in that the evidence is gathered from hands‐on teaching and learning experiences, rather than more formal assessment methods. They are based in a model of assessment for learning, and aim to support teachers in the “on‐going process of determining future learning directions” (Queensland Studies Authority, 2010 p. 1) for all students. Their aim is to focus teachers on interpreting and analysing evidence to make informed judgments about the achievement of all students, as a way to support subsequent planning for learning and teaching. The Evaluation of the Year 1 Literacy and Numeracy Checkpoints Assessments Supported Trial (hereafter the Evaluation) aimed to gather information about the appropriateness, effectiveness and utility of the Year 1 Checkpoints Assessments from early years’ teachers and leaders in up to one hundred Education Queensland schools who had volunteered to be part of the Supported Trial. These sample schools represent schools across a variety of Education Queensland regions and include schools with:  - A high Indigenous student population; - Urban, rural and remote school locations; - Single and multi‐age early phase classes; - A high proportion of students from low SES backgrounds. The purpose of the Evaluation was to: Evaluate the materials and report on the views of school‐based staff involved in the trial on the process, materials, and assessment practices utilised. The Evaluation has reviewed the materials, and used surveys, interviews, and observations of processes and procedures to collect relevant data to help present an informed opinion on the Year 1 Checkpoints as assessment for the early years of schooling. Student work samples and teacher planning and assessment documents were also collected. The evaluation has not evaluated the Year 1 Checkpoints in any other capacity than as a resource for Year 1 teachers and relevant support staff.

An evaluation framework for sustaining the impact of educational development

Notwithstanding significant efforts by international aid agencies, aid ineffectiveness became apparent in 1990s as the impact of continued development intervention did not endure the expected outcomes. Conventional monitoring and evaluation by those agencies is critiqued for focusing on measuring project outcomes and giving little attention to aspects of sustainability. As a result, devising a rigorous evaluation framework for educational development has been sought in light of recent paradigm shifts in international development. This paper reports on a case study of an Egyptian educational development project highlighting the importance of transforming the evaluation procedures to process evaluation so as to enhance project impact and longevity. This requires building evaluation capacity of the aid recipient country.

Assessment for learning in the accountability era : Queensland, Australia

Developments in school education in Australia over the past decade have witnessed the rise of national efforts to reform curriculum, assessment and reporting. Constitutionally the power to decide on curriculum matters still resides with the States. Higher stakes in assessment, brought about by national testing and international comparative analyses of student achievement data, have challenged State efforts to maintain the emphasis on assessment to promote learning while fulfilling accountability demands. In this article lessons from the Queensland experience indicate that it is important to build teachers' assessment capacity and their assessment literacy for the promotion of student learning. It is argued that teacher assessment can be a source of dependable results through moderation practice. The Queensland Studies Authority has recognised and supported the development of teacher assessment and moderation practice in the context of standards-driven, national reform. Recent research findings explain how the focus on learning can be maintained by avoiding an over-interpretation of test results in terms of innate ability and limitations and by encouraging teachers to adopt more tailored diagnosis of assessment data to address equity through focus on achievement for all. Such efforts are challenged as political pressures related to the Australian government’s implementation of national testing and national partnership funding arrangements tied to the performance of students at or below minimum standards become increasingly apparent.

Integrating a sustainable academic career around scholarly learning and teaching activities

To be scholarly in learning and teaching is rigorous academic work. It demands: currency and command of both discipline subject matter and educational theory; inquiring, methodical, and reflective approaches; the collection, evaluation and documentation of evidence of learning and teaching efficacy; and, optimally, entails participation in and communication among a community of teaching professionals. This chapter examines the author’s own practice in this regard to explicate the ‘how’ and ‘why’ of scholarly and scholarship approaches, as much as the ‘what’ and ‘where’ of that endeavour. In doing so, this meta‐analysis is made ‘community property’, in the same way that Shulman (1993: 6) exhorted we ‘change the status of teaching from private to community property’ so that teaching might be more greatly valued in the academy.

Wellness education : an integrated theoretical framework for fostering transformative learning

Optimum Wellness involves the development, refinement and practice of lifestyle choices which resonate with personally meaningful frames of reference. Personal transformations are the means by which our frames of reference are refined across the lifespan. It is through critical reflection, supportive relationships and meaning making of our experiences that we construct and reconstruct our life paths. When individuals are able to be what they are destined to be or reach their higher purpose, then they are able to contribute to the world in positive and meaningful ways. Transformative education facilitates the changes in perspective that enable one to contemplate and travel a path in life that leads to self-actualisation. This thesis argues for an integrated theoretical framework for optimum Wellness Education. It establishes a learner centred approach to Wellness education in the form of an integrated instructional design framework derived from both Wellness and Transformative education constructs. Students’ approaches to learning and their study strategies in a Wellness education context serve to highlight convergences in the manner in which students can experience perspective transformation. As they learn to critically reflect, pursue relationships and adapt their frames of reference to sustain their pursuit of both learning and Wellness goals, strengthening the nexus between instrumental and transformative learning is a strategically important goal for educators. The aim of this exploratory research study was to examine those facets that serve to optimise the learning experiences of students in a Wellness course. This was accomplished through three research issues: 1) What are the relationships between Wellness, approaches to learning and academic success? 2) How are students approaching learning in an undergraduate Wellness subject? Why are students approaching their learning in the ways they do? 3) What sorts of transformations are students experiencing in their Wellness? How can transformative education be formulated in the context of an undergraduate Wellness subject? Subsequent to a thorough review of the literature pertaining to Wellness education, a mixed method embedded case study design was formulated to explore the research issues. This thesis examines the interrelationships between student, content and context in a one semester university undergraduate unit (a coherent set of learning activities which is assigned a unit code and a credit point value). The experiences of a cohort of 285 undergraduate students in a Wellness course formed the unit of study and seven individual students from a total of sixteen volunteers whose profiles could be constructed from complete data sets were selected for analysis as embedded cases. The introductory level course required participants to engage in a personal project involving a behaviour modification plan for a self-selected, single dimension of Wellness. Students were given access to the Standard Edition Testwell Survey to assess and report their Wellness as a part of their personal projects. To identify relationships among the constructs of Self-Regulated Learning (SRL), Wellness and Student Approaches to Learning (SAL) a blend of quantitative and qualitative methods to collect and analyse data was formulated. Surveys were the primary instruments for acquiring quantitative data. Sources included the Wellness data from Testwell surveys, SAL data from R-SPQ surveys, SRL data from MSLQ surveys and student self-evaluation data from an end of semester survey. Students’ final grades and GPA scores were used as indicators of academic performance. The sources of qualitative data included subject documentation, structured interview transcripts and open-ended responses to survey items. Subsequent to a pilot study in which survey reliability and validity were tested in context, amendments to processes for and instruments of data collection were made. Students who adopted meaning oriented (deep/achieving) approaches tended to assess their Wellness at a higher level, seek effective learning strategies and perform better in formal study. Posttest data in the main study revealed that there were significant positive statistical relationships between academic performance and total wellness scores (rs=.297, n=205, p<.01). Deep (rs=.343, n=137, p<.01) and achieving (rs=.286, n=123, p<.01) approaches to learning also significantly correlated with Wellness whilst surface approaches had negative correlations that were not significant. SRL strategies including metacognitive selfregulation, effort, help-seeking and critical thinking were increasingly correlated with Wellness. Qualitative findings suggest that while all students adopt similar patterns of day to day activities for example attending classes, taking notes, working on assignments the level of care with which these activities is undertaken varies considerably. The dominant motivational trigger for students in this cohort was the personal relevance and associated benefits of the material being learned and practiced. Students were inclined to set goals that had a positive impact on affect and used “sense of happiness” to evaluate their achievement status. Students who had a higher drive to succeed and/or understand tended to have or seek a wider range of strategies. Their goal orientations were generally learning rather than performance based and barriers presented a challenge which could be overcome as opposed to a blockage which prevented progress. Findings from an empirical analysis of the Testwell data suggest that a single third order Wellness construct exists. A revision of the instrument is necessary in order to juxtapose it with the chosen six dimensional Wellness model that forms the foundation construct in the course. Further, redevelopment should be sensitive to the Australian context and culture including choice of language, examples and scenarios used in item construction. This study concludes with an heuristic for use in Wellness education. Guided by principles of Transformative education theory and behaviour change theory, and informed by this representative case study the “CARING” heuristic is proposed as an instructional design tool for Wellness educators seeking to foster transformative learning. Based upon this study, recommendations were made for university educators to provide authentic and personal experiences in Wellness curricula. Emphasis must focus on involving students and teachers in a partnership for implementing Wellness programs both in the curriculum and co-curricularly. The implications of this research for practice are predicated on the willingness of academics to embrace transformative learning at a personal level and a professional one. To explore students’ profiles in detail is not practical however teaching students how to guide us in supporting them through the “pain” of learning is a skill which would benefit them and optimise the learning and teaching process. At a theoretical level, this research contributes to an ecological theory of Wellness education as transformational change. By signposting the wider contexts in which learning takes place, it seeks to encourage changing paradigms to ones which harness the energy of each successive contextual layer in which students live. Future research which amplifies the qualities of individuals and groups who are “Well” and seeks the refinement and development of instruments to measure Wellness constructs would be desirable for both theoretical and applied knowledge bases. Mixed method Wellness research derived and conducted by teams that incorporate expertise from multiple disciplines such as psychology, anthropology, education, and medicine would enable creative and multi-perspective programs of investigation to be designed and implemented. Congruences and inconsistencies in health promotion and education would provide valuable material for strengthening the nexus between transformational learning and behaviour change theories. Future development of and research on the effectiveness of the CARING heuristic would be valuable in advancing the understanding of pedagogies which advance rather than impede learning as a transformative process. Exploring pedagogical models that marry with transformative education may render solutions to the vexing challenge of teaching and learning in diverse contexts.

Assessment for learning - Why the theory needs the practice

Assessment for Learning is a pedagogical practice with anticipated gains of increased student motivation, mastery and autonomy as learners develop their capacity to monitor and plan their own learning progress. Assessment for Learning (AfL) differs from Assessment of learning in its timing, occurring within the regular flow of learning rather than end point, in its purpose of improving student learning rather than summative grading and in the ownership of the learning where the student voice is heard in judging quality. Since Black and Wiliam (1998) highlighted the achievement gains that AfL practices seem to bring to all learners in classrooms, it has become part of current educational policy discourse in Australia, yet teacher adoption of the practices is not a straightforward implementation of techniques within an existing classroom repertoire. As can be seen from the following meta-analysis, recent research highlights a more complex interrelationship between teacher and student beliefs about learning and assessment, and the social and cultural interactions in and contexts of the classroom. More research is needed from a sociocultural perspective that allows meaning to emerge from practice. Before another policy push, we need to understand better the many factors within the assessment relationship. We need to hear from teachers and students through long-term AfL case studies both to inform AfL theory and to shed light on the complexities of pedagogical change for enhancing learner autonomy.

The challenge of assessing professional competence in work integrated learning

A fundamental aspect of work integrated learning (WIL) is the development of professional competence, the ability of students to perform in the work place. Alignment theory therefore suggests that the assessment of WIL should include an assessment of students’ demonstration of professional competence in the workplace. The assessment of professional competence in WIL is, however, problematic. It may be impractical for the academic supervisor to directly assess professional competence if there is a large number of students in external placements. If evidence of professional competence is provided by the student, the student’s ability to articulate his or her own capabilities will interfere with the validity of the assessment. If evidence of professional competency is provided by the supervisor then the assessment is heavily dependent on the individual supervisor and may be unreliable. This paper will examine the literature relating to the assessment of professional competence in WIL. The paper will be informed by the author’s experience in coordinating a WIL subject in an undergraduate law course. It will recommend that a mix of evidence provided by the student, the workplace supervisor and the academic supervisor should be used to assess professional competence in WIL.

Hunters & gatherers : strategies for curriculum mapping and data collection for assurance of learning

Assurance of learning is a predominant feature in both quality enhancement and assurance in higher education. Assurance of learning is a process that articulates explicit program outcomes and standards, and systematically gathers evidence to determine the extent to which performance matches expectations. Benefits accrue to the institution through the systematic assessment of whole of program goals. Data may be used for continuous improvement, program development, and to inform external accreditation and evaluation bodies. Recent developments, including the introduction of the Tertiary Education and Quality Standards Agency (TEQSA) will require universities to review the methods they use to assure learning outcomes. This project investigates two critical elements of assurance of learning: 1. the mapping of graduate attributes throughout a program; and 2. the collection of assurance of learning data. An audit was conducted with 25 of the 39 Business Schools in Australian universities to identify current methods of mapping graduate attributes and for collecting assurance of learning data across degree programs, as well as a review of the key challenges faced in these areas. Our findings indicate that external drivers like professional body accreditation (for example: Association to Advance Collegiate Schools of Business (AACSB)) and TEQSA are important motivators for assuring learning, and those who were undertaking AACSB accreditation had more robust assurance of learning systems in place. It was reassuring to see that the majority of institutions (96%) had adopted an embedding approach to assuring learning rather than opting for independent standardised testing. The main challenges that were evident were the development of sustainable processes that were not considered a burden to academic staff, and obtainment of academic buy in to the benefits of assuring learning per se rather than assurance of learning being seen as a tick box exercise. This cultural change is the real challenge in assurance of learning practice.

Affiliation, autonomy and assessment for learning

Assessment for Learning (AfL) is an international assessment area of interest, yet, during 20 years of AfL research, the desired outcome of increased learner autonomy remains elusive. This article analyses AfL practices in classrooms as students negotiated identities as autonomous learners within a classroom community of practice. A sociocultural theoretical framework in formed the analysis of three case studies conducted in Queensland middle school classrooms. Key findings include the importance of the teacher–student relationship, viewing AfL as patterns of participation that develop expertise, and learner autonomy as a negotiated learner identity within each classroom context.

Good Practice Report : Blended Learning

In early 2011, the Australian Learning and Teaching Council Ltd (ALTC) commissioned a series of Good Practice Reports on completed ALTC projects and fellowships. This report will: • Provide a summative evaluation of the good practices and key outcomes for teaching and learning from completed ALTC projects and fellowships relating to blended learning • Include a literature review of the good practices and key outcomes for teaching and learning from national and international research • Identify areas in which further work or development are appropriate. The literature abounds with definitions; it can be argued that the various definitions incorporate different perspectives, but there is no single, collectively accepted definition. Blended learning courses in higher education can be placed somewhere on a continuum, between fully online and fully face-to-face courses. Consideration must therefore be given to the different definitions for blended learning presented in the literature and by users and stakeholders. The application of this term in these various projects and fellowships is dependent on the particular focus of the team and the conditions and situations under investigation. One of the key challenges for projects wishing to develop good practice in blended learning is the lack of a universally accepted definition. The findings from these projects and fellowships reveal the potential of blended learning programs to improve both student outcomes and levels of satisfaction. It is clear that this environment can help teaching and learning engage students more effectively and allow greater participation than traditional models. Just as there are many definitions, there are many models and frameworks that can be successfully applied to the design and implementation of such courses. Each academic discipline has different learning objectives and in consequence there can’t be only one correct approach. This is illustrated by the diversity of definitions and applications in the ALTC funded projects and fellowships. A review of the literature found no universally accepted guidelines for good practice in higher education. To inform this evaluation and literature review, the Seven Principles for Good Practice in Undergraduate Education, as outlined by Chickering and Gamson (1987), were adopted: 1. encourages contacts between students and faculty 2. develops reciprocity and cooperation among students 3. uses active learning techniques 4. gives prompt feedback 5. emphasises time on task 6. communicates high expectations 7. respects diverse talents and ways of learning. These blended learning projects have produced a wide range of resources that can be used in many and varied settings. These resources include: books, DVDs, online repositories, pedagogical frameworks, teaching modules. In addition there is valuable information contained in the published research data and literature reviews that inform good practice and can assist in the development of courses that can enrich and improve teaching and learning.

Learning as identity and practice through involvement in online moderation

For teachers working in a standards-based assessment system, professional conversations through organised social moderation meetings are a vital element. This qualitative research investigated the learning that occurred as a result of online moderation discussions. Findings illustrate how participating in social moderation meetings in an online context can support teachers to understand themselves as assessors, and can provide opportunities for teachers to imagine possibilities for their teaching that move beyond the moderation practice.

Professional conversations : a new approach to professional learning in ECEC

In 2009, Australia celebrated the introduction of a national Early Years Learning Framework. This is a critical component in a series of education reforms designed to support quality pedagogy and practice in early childhood education and care (ECEC) and successful transition to school. As with any policy change, success in real terms relies upon building shared understanding and the capacity of educators to apply new knowledge and support change and improved practice within their service. With these outcomes in mind, a collaborative research project is investigating the efficacy of a new approach to professional learning in ECEC: the professional conversation. This paper reports on the trial and evaluation of a series of professional conversations on the Early Years Learning Framework, and their capacity to promote collaborative reflective practice, shared understanding, and improved practice in ECEC. The paper details the professional conversation approach, key challenges and critical success factors, and the learning outcomes for conversation participants. Findings support the efficacy of this approach to professional learning in ECEC, and its capacity to support policy reform and practice change in ECEC.

Teaching and Learning in Maritime Security: A Literature Review

With the rise in attacks and attempted attacks on marine‐based critical infrastructure, maritime security is an issue of increasing importance worldwide. However, there are three significant shortfalls in the efforts to overcome potential threats to maritime security: the need for greater understanding of whether current standards of best practice are truly successful in combating and reducing the risks of terrorism and other security issues, the absence of a collective maritime security best practice framework and the need for improved access to maritime security specific graduate and postgraduate (long) courses. This paper presents an overview of existing international, regional national standards of best practice and shows that literature concerning the measurement and/ or success of standards is virtually non‐existent. In addition, despite the importance of maritime workers to ensuring the safety of marine based critical infrastructure, a similar review of available Australian education courses shows a considerable lack of availability of maritime security‐specific courses other than short courses that cover only basic security matters. We argue that the absence of an Australian best practice framework informed by evaluation of current policy responses – particularly in the post 9/11 environment – leaves Australia vulnerable to maritime security threats. As this paper shows, the reality is that despite the security measures put in place post 9/11, there is still considerable work to be done to ensure Australia is equipped to overcome the threats posed to maritime security.

Learning or legitimacy? an exploration of business planning amongst Australian social enterprises

In recent years, the value of business planning for new business ventures and small firms has been the subject of debate amongst entrepreneurship researchers (Brinckmann et al 2010: 24). Drawing on institutional theory, a number of writers suggest that business planning is primarily used to confer symbolic legitimacy on businesses seeking investment and engagement from external stakeholders ( Karlsson & Honig 2009; Zimmerman & Zeitz 2002; Delmar & Shane 2004). In this sense, business planning may not have any significant effects on firm learning, but may be used as evidence of good business operations in order to attract external resources. Meta-evaluation of the available empirical literature contests this proposition, finding that both the symbolic and organisational learning effects of business planning influence small firm performance (Brinckmann et al 2010: 36) While social enterprise – which we define as organisations that exist for a public or community benefit and trade to fulfill their mission - the study of social enterprise is a nascent and pre-paradigmatic area of inquiry (Nicholls 2010). As a consequence, there has been relatively little empirical analysis of the nature or effects of business planning amongst social enterprises (for two exceptions, see exploratory studies by Hynes 2009 and Bull & Crompton 2006). In this paper, we examine business planning practices amongst Australian social enterprises. Drawing on a survey of 365 social enterprises conducted in 2010 and in-depth interviews with 11 social entrepreneurs and managers from eight social enterprises, we find that social enterprises report being more actively engaged in business planning activities than their mainstream business counterparts. Our exploratory research suggests that both legitimacy and learning drive business planning amongst social enterprises, although legitimacy is the stronger driver. Our results also suggest that, as multi-stakeholder businesses led by mission, business planning can serve unique communicative and relational functions for this business type.