Evaluation of the Whitten Project : final report

Evaluation of The Whitten Project. A pilot project aimed at developing emerging young community leaders across Melbourne’s western region. Final Report

A formative evaluation of the Stronger Smarter Learning Communities Project. 2011 report.

This report is a formative evaluation of the operations of the DEEWR funded Stronger Smarter Learning Community (SSLC) project from September 2009 to July 2011. It is undertaken by an independent team of researchers from Queensland University of Technology, the University of Newcastle and Harvard University. It reports on findings from: documentary analysis; qualitative case studies of SSLC Hub schools; descriptive, multivariate and multilevel analysis of survey data from school leaders and teachers from SSLC Hub and Affiliate schools and from a control group of non-SSLC schools; and multilevel analysis of school-level data on SSLC Hubs, Affiliates and ACARA like-schools. Key findings from this work are that: • SSLC school leaders and teachers are reporting progress in changing school ethos around issues of: recognition of Indigenous identity, Indigenous leadership, innovative approaches to staffing and school models, Indigenous community engagement and high expectations leadership; • Many Stronger Smarter messages are reportedly having better uptake in schools with high percentages of Indigenous students; • There are no major or consistent patterns of differences between SSLC and non-SSLC schools in teacher and school leader self-reports of curriculum and pedagogy practices; and • There is no evidence to date that SSLC Hubs and Affiliates have increased attendance or increased achievement gains compared to ACARA like-schools. Twenty-one months is relatively early in this school reform project. Hence the major focus of subsequent reports will be on the documentation of comparative longitudinal gains in achievement tests and improved attendance. The 2011 and 2012 research also will model the relationships between change in school ethos/climate, changed Indigenous community relations, improved curriculum/pedagogy, and gains in Indigenous student achievement, attendance and outcomes. The key challenge for SSLC and the Stronger Smarter approach will be whether it can systematically generate change and reform in curriculum and pedagogy practices that can be empirically linked to improved student outcomes.

Ensuring quality during monitoring of the learning environment of government primary schools achieved through EUI-PDP for teachers and head teachers

A positive change in the learning environment in schools is visible through ongoing professional development of teachers and administrators. Monitoring the professional development program and providing support to teachers and administrators to transfer their learnings into the school environment ensures some measures of quality. Quality issues led to the launching of the Professional Development Program (PDP) for Primary School Teachers (PSTs) of Sindh by the United Educational Initiative (UEI), a consortium of five Governmental and Non-Governmental Organizations, working under the supervision of Education Sector Reform Assistance (ESRA). Implementation of the UEI-PDP in four districts of Sindh, is ensured by a team of professionals in each district. Recognising that capacity building of district education employees would improve the educational system in the country, 130 Master Trainers were selected, on merit, from the District Education Office for the training of 17,000 teachers and 3000 Head teachers/administrators over a period of two years. This paper developed the design of a Monitoring Process for a Professional Development Program for Primary School Teachers and Administrators. Data was collected through Pre/Post observations, Interviews, Questionnaires and Reports. Such tools make it possible for the monitoring teams to observe, to inquire further, and, along with the Managers, Master Trainers and School Support Team, seek to explain the progress of the program and take corrective action where indicated. Both formative evaluations as well as summative  evaluation techniques are utilized for evaluating the program. The monitoring process that assisted in formative evaluations is described. In order to assist in summative evaluation, data collected through the monitoring process was further developed to categorize the schools where teachers and head teachers are trained. It is hoped that the categorization of the schools may lead to further improvements in those schools which fall in the group for need improvement. It may also initiate further research as to reasons behind why some schools are in the good category and why others fall in the average category.

Looking Yonder : Longitudinal Research and Evaluation of the Yonder Participative Arts-based Learning Model Report 2010-2012

Since 2007, KITE Arts Education Program @ QPAC has been engaged in a series of arts and drama-based experiences for students in selected primary schools on the edges of Brisbane and in regional Queensland. The in-school workshop experiences of the program have culminated in a performance by the children for their school community, parents and carers at the Queensland Performing Arts Centre or a regional cultural venue. In conducting an analysis of the Yonder project, the researcher aimed to provide evidence of outcomes brought about through participation by schools, school staff, students and their communities in the Yonder project. To develop longitudinal data project initiators, participants were interviewed at six-monthly intervals to establish patterns of engagement and participation. The report analyses arts-based workshops conducted by the teacher artist in edge-city Brisbane and a regional centre; interviews with teachers and school administrators from the participating schools; interviews with teacher artist and professional artists; interviews with community partners; teacher professional development workshops; community-based workshops; performance outcomes that were the culminating events of the workshop program; student work samples and student reflections on the program. This document covers data and project outputs from February 2010 to July 2012. There have been five iterations of the Yonder project since its commencement in mid-2009 — three in regional Queensland (February–April 2010; February–May 2011; February–May 2012) and two in edge-city1 Brisbane (July–September 2010; August–October 2011). This report is a result of a research partnership between Queensland Performing Arts Centre and Queensland University of Technology (QUT) Creative Industries Faculty(Drama).

Evaluation of the "reconceptualising early mathematics learning" project

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure enables mathematical thinking and simple forms of generalisation from an early age. The project aims to promote a strong foundation for mathematical development by focusing on critical, underlying features of mathematics learning. This paper provides an overview of key aspects of the assessment and intervention, and analyses of the impact of PASMAP on students’ representation, abstraction and generalisation of mathematical ideas. A purposive sample of four large primary schools, two in Sydney and two in Brisbane, representing 316 students from diverse socio-economic and cultural contexts, participated in the evaluation throughout the 2009 school year and a follow-up assessment in 2010. Two different mathematics programs were implemented: in each school, two Kindergarten teachers implemented the PASMAP and another two implemented their regular program. The study shows that both groups of students made substantial gains on the ‘I Can Do Maths’ assessment and a Pattern and Structure Assessment (PASA) interview, but highly significant differences were found on the latter with PASMAP students outperforming the regular group on PASA scores. Qualitative analysis of students’ responses for structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a relatively short period of time.

Evaluation of the Implementation of CLIL (Content and Language Integrated Learning) Methodology in the Didactics of the English Language in Preschool Education Course Taught in the Preschool Education Teacher Undergraduate Program at the University of Alicante

Although it is known that the Spanish current Educative System promotes using the Communicate Approach to teach foreign languages in schools, other recently designed approaches are also used to help students improve their skills when communicating in a foreign language. One of these approaches is Content and Language Integrated Learning, also known as CLIL, which is used to teach content courses using the English language as the language of instruction. This approach improves the students’ skills in English as the same time as they learn content from other areas. The goal of this thesis is to present a research project carried out at the University of Alicante during the academic year 2011-2012. With this research we obtained results that provide quantitative and qualitative data which explains how the use of the CLIL methodology affects the English level of students in the “Didactics of the English Language in Preschool Education” course in Preschool Education Teacher Undergraduate Program as students acquire the contents of the course.

Evaluation of the quality of the teaching-learning process in undergraduate courses in Nursing

Objective: to identify aspects of improvement of the quality of the teaching-learning process through the analysis of tools that evaluated the acquisition of skills by undergraduate students of Nursing. Method: prospective longitudinal study conducted in a population of 60 second-year Nursing students based on registration data, from which quality indicators that evaluate the acquisition of skills were obtained, with descriptive and inferential analysis. Results: nine items were identified and nine learning activities included in the assessment tools that did not reach the established quality indicators (p<0.05). There are statistically significant differences depending on the hospital and clinical practices unit (p<0.05). Conclusion: the analysis of the evaluation tools used in the article "Nursing Care in Welfare Processes" of the analyzed university undergraduate course enabled the detection of the areas for improvement in the teaching-learning process. The challenge of education in nursing is to reach the best clinical research and educational results, in order to provide improvements to the quality of education and health care.

Evaluation of the year 1 literacy and numeracy checkpoints assessments trial - 2010

The draft Year 1 Literacy and Numeracy Checkpoints Assessments were in open and supported trial during Semester 2, 2010. The purpose of these trials was to evaluate the Year 1 Literacy and Numeracy Checkpoints Assessments (hereafter the Year 1 Checkpoints) that were designed in 2009 as a way to incorporate the use of the Year 1 Literacy and Numeracy Indicators as formative assessment in Year 1 in Queensland Schools. In these trials there were no mandated reporting requirements. The processes of assessment were related to future teaching decisions. As such the trials were trials of materials and the processes of using those materials to assess students, plan and teach in year 1 classrooms. In their current form the Year 1 Checkpoints provide assessment resources for teachers to use in February, June and October. They aim to support teachers in monitoring children's progress and making judgments about their achievement of the targeted P‐3 Literacy and Numeracy Indicators by the end of Year 1 (Queensland Studies Authority, 2010 p. 1). The Year 1 Checkpoints include support materials for teachers and administrators, an introductory statement on assessment, work samples, and a Data Analysis Assessment Record (DAAR) to record student performance. The Supported Trial participants were also supported with face‐to‐face and on‐line training sessions, involvement in a moderation process after the October Assessments, opportunities to participate in discussion forums as well as additional readings and materials. The assessment resources aim to use effective early years assessment practices in that the evidence is gathered from hands‐on teaching and learning experiences, rather than more formal assessment methods. They are based in a model of assessment for learning, and aim to support teachers in the “on‐going process of determining future learning directions” (Queensland Studies Authority, 2010 p. 1) for all students. Their aim is to focus teachers on interpreting and analysing evidence to make informed judgments about the achievement of all students, as a way to support subsequent planning for learning and teaching. The Evaluation of the Year 1 Literacy and Numeracy Checkpoints Assessments Supported Trial (hereafter the Evaluation) aimed to gather information about the appropriateness, effectiveness and utility of the Year 1 Checkpoints Assessments from early years’ teachers and leaders in up to one hundred Education Queensland schools who had volunteered to be part of the Supported Trial. These sample schools represent schools across a variety of Education Queensland regions and include schools with:  - A high Indigenous student population; - Urban, rural and remote school locations; - Single and multi‐age early phase classes; - A high proportion of students from low SES backgrounds. The purpose of the Evaluation was to: Evaluate the materials and report on the views of school‐based staff involved in the trial on the process, materials, and assessment practices utilised. The Evaluation has reviewed the materials, and used surveys, interviews, and observations of processes and procedures to collect relevant data to help present an informed opinion on the Year 1 Checkpoints as assessment for the early years of schooling. Student work samples and teacher planning and assessment documents were also collected. The evaluation has not evaluated the Year 1 Checkpoints in any other capacity than as a resource for Year 1 teachers and relevant support staff.

Exploratory factor analysis and psychometric evaluation of the Teachers Reporting Attitude Scale for Child Sexual Abuse

This paper presents an evaluation of an instrument to measure teachers’ attitudes towards reporting child sexual abuse and discusses the instrument’s merit for research into reporting practice. Based on responses from 444 Australian teachers, the Teachers’ Reporting Attitude Scale for Child Sexual Abuse (TRAS - CSA) was evaluated using exploratory factor analysis. The scale isolated three dimensions: commitment to the reporting role; confidence in the system’s response to reports; and concerns about reporting. These three factors accounted for 37.5% of the variance in the 14-item measure. Alpha coefficients for the subscales were 0.769 (commitment), 0.617 (confidence), and 0.661 (concerns). The findings provide insights into the complexity of studying teachers’ attitudes towards reporting of child sexual abuse, and have implications for future research.

An evaluation of the Australian ‘reconceptualising early mathematics learning’ project : key findings and implications

The Pattern and Structure Mathematics Awareness Project (PASMAP) has investigated the development of patterning and early algebraic reasoning among 4 to 8 year olds over a series of related studies. We assert that an awareness of mathematical pattern and structure (AMPS) enables mathematical thinking and simple forms of generalization from an early age. This paper provides an overview of key findings of the Reconceptualizing Early Mathematics Learning empirical evaluation study involving 316 Kindergarten students from 4 schools. The study found highly significant differences on PASA scores for PASMAP students. Analysis of structural development showed increased levels for the PASMAP students; those categorised as low ability developed improved structural responses over a short period of time.

Telephone survey evaluation of the educational impact of a distance learning course

The educational impact of a distance learning (DL) course entitled ''Health Screening for Health Promotion, was investigated using a telephone questionnaire survey. An introduction to the DL course was distributed to all community pharmacists in England (16,400); the main body of the course, on which pharmacists were examined, was distributed free of charge to all pharmacists who requested it (1,485). Pharmacists participating in the survey (868) were organized by random selection into groups and stratified according to age, sex and postcode. A matched control group was randomly drawn from those pharmacists who had not participated in the course. The DL course improved pharmacists' knowledge about health screening/health promotion issues (e.g., mean score of 66 percent achieved by a group who had completed the course; 51 percent achieved by the control group; P<0.001). Factors influencing score achieved included sex and year of registration. Males performed better than females (P<0.008) while performance decreased with number of years on the register (P<0.001).

An evaluation of the Socrative (personal response system) app for increasing student engagement and learning in an undergraduate psychology curriculum.

Personal response systems using hardware such as 'clickers' have been around for some time, however their use is often restricted to multiple choice questions (MCQs) and they are therefore used as a summative assessment tool for the individual student. More recent innovations such as 'Socrative' have removed the need for specialist hardware, instead utilising web-based technology and devices common to students, such as smartphones, tablets and laptops. While improving the potential for use in larger classrooms, this also creates the opportunity to pose more engaging open-response questions to students who can 'text in' their thoughts on questions posed in class. This poster will present two applications of the Socrative system in an undergraduate psychology curriculum which aimed to encourage interactive engagement with course content using real-time student responses and lecturer feedback. Data is currently being collected and result will be presented at the conference.
The first application used Socrative to pose MCQs at the end of two modules (a level one Statistics module and level two Individual Differences Psychology module, class size N≈100), with the intention of helping students assess their knowledge of the course. They were asked to rate their self-perceived knowledge of the course on a five-point Likert scale before and after completing the MCQs, as well as their views on the value of the revision session and any issues that had with using the app. The online MCQs remained open between the lecture and the exam, allowing students to revisit the questions at any time during their revision.
This poster will present data regarding the usefulness of the revision MCQs, the metacognitive effect of the MCQs on student's judgements of learning (pre vs post MCQ testing), as well as student engagement with the MCQs between the revision session and the examination. Student opinions on the use of the Socrative system in class will also be discussed.
The second application used Socrative to facilitate a flipped classroom lecture on a level two 'Conceptual Issues in Psychology' module, class size N≈100). The content of this module requires students to think critically about historical and contemporary conceptual issues in psychology and the philosophy of science. Students traditionally struggle with this module due to the emphasis on critical thinking skills, rather than simply the retention of concrete knowledge. To prepare students for the written examination, a flipped classroom lecture was held at the end of the semester. Students were asked to revise their knowledge of a particular area of Psychology by assigned reading, and were told that the flipped lecture would involve them thinking critically about the conceptual issues found in this area. They were informed that questions would be posed by the lecturer in class, and that they would be asked to post their thoughts using the Socrative app for a class discussion. The level of preparation students engaged in for the flipped lecture was measured, as well as qualitative opinions on the usefulness of the session. This poster will discuss the level of student engagement with the flipped lecture, both in terms of preparation for the lecture, and engagement with questions posed during the lecture, as well as the lecturer's experience in facilitating the flipped classroom using the Socrative platform.

Older and wiser?: a comparison of the learning and study strategies of mature age and younger teacher education students

'Non-traditional' students are increasingly a part of university populations. This study examined differences between mature age and younger university students in their learning and study strategies as measured by the Learning and Study Strategies Inventory (LASSI). Subjects were 21 mature age and 104 younger teacher education students enrolled in The Bachelor of Teaching (Primary) course at the Royal Melbourne Institute of Technology. Significant correlations were found between the students' LASSI scores and both their self-reported level of global skill and their perceptions of how difficult the course was. There were significant differences between the two groups in terms of their learning and study strategies, with mature age students reporting themselves to be using effective strategies more often, on average, than younger students. The validity and implications of these findings in terms of student learning, support and instruction in study and learning and in predicting academic success are discussed.