Analysing lexical richness in French learner language: what frequency lists and teacher judgements can tell us about basic and advanced words

We show how teacher judgements can be used to assess the quality of vocabulary used by L2 learners of French.

Towards an understanding of teacher judgement in the context of social moderation

Social moderation involves teachers gathering together to discuss their judgements of the quality of student work and to reach agreement regarding the standard awarded. This qualitative study conducted over a three-year period investigated the social practice of moderation and the influence on teachers’ judgements of students work. An initial survey of teachers’ understandings of moderation and standards, pre-interviews of teachers who participated in the moderation meetings, observations of these meetings with a particular focus on one teacher (focus teachers) comprised the data collection methods. Data analysis involved organising, matching, coding, identifying patterns and themes using a constant comparative method. Socio-cultural theories of learning and assessment underpinned the approach to data analysis and proved helpful in explaining the diverse influences on teachers’ judgements beyond the task criteria, and the progressive development of shared understandings through engaging in professional discussions of students’ work. The study revealed that the process is not clear and linear and is influenced by factors such as the representation of the standards and the knowledge base of the teachers.

Do teacher stereotypes about school tracks function as expectations at the collective level and do they relate to the perception of obstacles in the classroom and to teachers’ self-efficacy beliefs?

The effects of individual teacher expectations have been the subject of intensive research. Results indicate that teachers use their expectations to adapt their interactions with their students to some degree (as summarized in a review by Jussim & Harber, 2005). This can in turn lead to expectancy-confirming student developments. While there are studies on the Pygmalion effect on individual students, there is only little research on teacher judgements of whole classes and schools. Our study aims to extend the perspective of teacher judgements at the collective level to stereotypes within the context of school tracking. The content and structure of teachers’ school track stereotypes are investigated as well as the question of whether these stereotypical judgements are related to teachers’ perception of obstacles to their teaching and their teaching self-efficacy beliefs. Cross-sectional data on 341 teachers at two different school types from the Panel Study at the Research School „Education and Capabilities“ in North Rhine-Westphalia (PARS) (see Bos et al., 2016) were used for two purposes: First, the structure of teachers’ stereotypes was identified via an exploratory factor analysis. Second, in follow-up regression analyses, the stereotype dimensions extracted were used to predict teachers’ perceptions of obstacles to their classroom work and their individual and collective teacher self-efficacy beliefs. Results showed that – after controlling for the average cognitive abilities and the average cultural capital of the students – teacher stereotypes were indeed related to perceived obstacles concerning their classroom work and their self-efficacy beliefs. After a discussion of the strengths and limitations of the present research, the article closes with a short proposal of a future research framework for collective Pygmalion effects. (DIPF/Orig.)

Designing online assessment tools for disengaged youth

This article reports on the development of online assessment tools for disengaged youth in flexible learning environments. Sociocultural theories of learning and assessment and Bourdieu’s sociological concepts of capital and exchange were used to design a purpose-built content management system. This design experiment engaged participants in assessment that led to the exchange of self, peer and teacher judgements for credentialing. This collaborative approach required students and teachers to adapt and amend social networking practices for students to submit and judge their own and others’ work using comments, ratings, keywords and tags. Students and teachers refined their evaluative expertise across contexts, and negotiated meanings and values of digital works, which gave rise to revised versions and emergent assessment criteria. By combining social networking tools with sociological models of capital, assessment activities related to students’ digital productions were understood as valuations and judgements within an emergent, negotiable social field of exchange.

The development of shared understandings of assessment policy : travelling between global and local contexts

In this paper, teachers’ enactment of assessment policy within demands for accountability and consistency of teacher judgements is considered. Evidence is drawn from a qualitative study involving 50 middle school teachers from Queensland, Australia, who participated in online social moderation meetings with teachers located in dispersed areas around the state. The study presents how travelling policy is embedded in local histories and cultures, in particular within systems of accountability; and the different layers of what may be considered ‘local’. The paper examines the intersections of travelling and embedded policy, and global and local contexts as these are enacted through online moderation meetings.

Transition: improved literacy outcomes research study

This paper reports on the Transition: Improved Literacy Outcomes Research Study established by the Department of Education and Training (DET). The purpose of the study was to identify the most useful and usable data for educators to access in order to ensure that student literacy learning is maximised during the transition from Primary to Secondary school.1. The research followed a mixed-methods approach, combining a statistically defensible sample of schools selected to represent school types (primary, secondary, combined), locations (rural, regional, urban) and all regions with a survey and follow- up interviews.2. The results are presented as a mixture of statistics, case studies and discussion. The case studies were developed from both survey data and interview data.3. Leaders and teachers overwhelmingly valued teacher judgement (through AusVELS) as the most useful data in understanding students’ strengths and weaknesses, followed by On-Demand testing.4. NAPLAN results were frequently transferred between schools, but were found to be less useful than other data by teachers.5. School leaders and teachers are currently using data for ability streaming, or for tailoring curriculum for individual student needs. A number of schools use the Literacy data for achieving a balance of student abilities within classes.6. Support for access to, and use of, data is mainly manual, with some use of spreadsheets.7. As the number of feeder Primary schools to a single Secondary school increases, data management becomes a much larger burden on the Secondary staff concerned.8. In Secondary schools, there was a clear demand for Literacy data, but these data were not always those that were being provided by their feeder Priamary schools. This mis-match appears to lead to some frustration among Literacy transition staff at both levels of schooling.9. A gap that needs filling is around the nature of the pedagogies at the two school levels. Not knowing, or misunderstanding, the approaches and needs of each level of schooling leads to the passing on of irrelevant Literacy data in some instances.10. The need for a common template for transition data was expressed by many Transition staff, both Primary and Secondary. In some instances the secondary school had set up a local template for this11. It would be advantageous for data to be directly up-loaded or transferred into a system via electronic tools already in use, such as an Excel spreadsheet.12. We would recommend that the capacity to provide some visualisation of data, and the compilation of internal Literacy data such as teacher judgements and AusVELS levels, or additional testing carried out by the school.13. Student background data was seen by many Secondary transition staff as being equally as valuable as more formal Literacy data for determining student needs.

Double standards in cultural judgements

The author, a teacher of television and film from a cultural studies perspective, endeavours to persuade his students to give up snobbery when they judge culture. The author has found that most students enter university with a series of middle class value judgements very strongly in place. Essentially, the judgements are that commercial culture is 'bad' and non-commercial culture is 'good'.

Moderation and consistency of teacher judgement : teachers’ views

Major curriculum and assessment reforms in Australia have generated research interest in issues related to standards, teacher judgement and moderation. This article is based on one related inquiry of a large-scale Australian Research Council Linkage project conducted in Queensland. This qualitative study analysed interview data to identify teachers’ views on standards and moderation as a means to achieving consistency of teacher judgement. A complementary aspect of the research involved a blind review that was conducted to determine the degree of teacher consistency without the experience of moderation. Empirical evidence was gained that most teachers, of the total interviewed, articulated a positive attitude towards the use of standards in moderation and perceived that this process produces consistency in teachers’ judgements. Context was identified as an important influential factor in teachers’ judgements and it was concluded that teachers’ assessment beliefs, attitudes and practices impact on their perceptions of the value of moderation practice and the extent to which consistency can be achieved.

Editorial - Assessment of EAL/D learners : the validity of teachers' professional judgements

Assessment is a recurring theme in this issue of TESOL in Context. It is the focus of a themed section; addressed in some of the award-winning thesis research we feature; and one of several topics in an article on EFL teacher preparation in an Australian university. In the first section of the issue we are pleased to publish précis of the theses produced by the joint winners of the Penny McKay Memorial Award for Best Thesis in Language Education. Penny was a major figure in the development of EAL/D education in Australia (see Dooley & Moore, 2009). Her work is remembered and continued through an Award established by the Australian Council of TESOL Associations (ACTA) and the Applied Linguistics Association of Australia (ALAA). The award recognises doctoral research which makes an outstanding contribution to second/additional language education in Australian schools (http://www.tesol.org.au/About-ACTA/PENNY-MCKAY-MEMORIAL-FUND)...

Researching teacher education : the research we have to have

Increasingly, politicians, bureaucrats, the business community, members of our communities and even members of the teaching profession, are asking questions about the professional preparation of teachers: What is the value of teacher education? What should beginning teachers know and be able to do? How can judgements be made about what beginning teachers know and are able to do?

Teacher educators are dangerously close to having little or no impact on the policies being driven by various responses to these questions. Research in teacher education, while copious, is under-funded and consequently teacher education researchers focus on their own programs and conduct small-scale, context specific, studies that are not readily generalisable. As a result, policy makers largely ignore it. I argue that if teacher educators are to retain their responsibility for the preparation of new professionals and for the development of new knowledge for the profession, we need to focus our research on key policy issues regarding the preparation of teachers and we need to ensure that our research connects with and extends the body of relevant research and the questions being asked about the value of teacher education.

Using professional atandards : assessing work integrated learning in initial teacher education

All Australian teacher education programs must include practical experience--the practicum. It is a critical part of learning to become a teacher.  One of the major challenges in initial teacher education is to provide good quality assessment of the practicum.  Assessing the practicum is filled with tension for both the individual supervisor as well as the pre-service teacher. In 2011 the Australian National Professional Standards for Teachers were established.  On completion of teacher education programs, graduate teachers will have gained the knowledge and practice to meet the seven national standards.  For teacher preparation programs, the successful implementation of the standards will rely on the opportunities for preservice teachers to gather evidence of achieving the standards. This project focussed specifically on evidence of achievements of these standards through assessment practices during practicum.
The overall aim of this project was to enhance the academic and school-based teacher educators' and preservice teachers' capacities and understandings of assessing the practicum.  To achieve this aim, four outcomes were developed to provide professional leaning for improving the assessment practices of the practicum: a website resource, a collaborative partnership process, a professional learning model (PLM) and a developmental 'inventory' of evidence of achievement of the first five national standards.  The website resource provides materials and activities for staff involved in the design of professional experience in initial teacher education programs, to work with partner schools and preservice teachers to facilitate high quality supervision and assessment in practicum sites.  The collaborateive partnership process used for achieving these soutcomes -- communities of reflective practitioners--is integral to the professional learning focus of the project.  It guides the use of the resource in future teacher education sites of practice.  The professional learning model and website materials emphasise the critical role that evidence-informed judgements play at school sites in learning and assessment of future teachers.