UNDERSTANDING ESSAY RATING AS A SOCIALLY MEDIATED ACTIVITY: THE CASE OF A HIGH-STAKES ENGLISH TEST IN CHINA

Most essay rating research in language assessment has examined human raters’ essay rating as a cognitive process, thus overlooking or oversimplifying the interaction between raters and sociocultural contexts. Given that raters are social beings, their practices have social meanings and consequences. Hence it is important to situate essay rating within its sociocultural context for a more meaningful understanding. Drawing on Engeström’s (1987, 2001) cultural-historical activity theory (CHAT) framework with a sociocultural perspective, this study reconceptualized essay rating as a socially mediated activity with both cognitive (individual raters’ goal-directed decision-making actions) and social layers (raters’ collective object-oriented essay rating activity at related settings). In particular, this study explored raters’ essay rating at one provincial rating centre in China within the context of a high-stakes university entrance examination, the National Matriculation English Test (NMET). This study adopted a multiple-method multiple-perspective qualitative case study design. Think-aloud protocols, stimulated recalls, interviews, and documents served as the data sources. This investigation involved 25 participants at two settings (rating centre and high schools), including rating centre directors, team leaders, NMET essay raters who were high school teachers, and school principals and teaching colleagues of these essay raters. Data were analyzed using Strauss and Corbin’s (1990) open and axial coding techniques, and CHAT for data integration. The findings revealed the interaction between raters and the NMET sociocultural context. Such interaction can be understood through a surface structure (cognitive layer) and a deep structure (social layer) concerning how raters assessed NMET essays, where the surface structure reflected the “what” and the deep structure explained the “how” and “why” in raters’ decision-making. This study highlighted the roles of goals and rules in rater decision-making, rating tensions and raters’ solutions, and the relationship between essay rating and teaching. This study highlights the value of a sociocultural view to essay rating research, demonstrates CHAT as a sociocultural approach to investigate essay rating, and proposes a direction for future washback research on the effect of essay rating. This study also provides support for NMET rating practices that can potentially bring positive washback to English teaching in Chinese high schools.

A Minor Change?: A Case Study of Carleton Place's Integration of Minor Variance Procedures into Ontario's Development Permit System

The Development Permit System has been introduce with minimal directives for establishing a decision making process. This is in opposition to the long established process for minor variances and suggests that the Development Permit System does not necessarily incorporate all of Ontario’s fundamental planning principles. From this concept, the study aimed to identify how minor variances are incorporated into the Development Permit System. In order to examine this topic, the research was based around the following research questions: • How are ‘minor variance’ applications processed within the DPS? • To what extent do the four tests of a minor variance influence the outcomes of lower level applications in the DPS approval process? A case study approach was used for this research. The single-case design employed both qualitative and quantitative research methods including a review of academic literature, court cases, and official documents, as well as a content analysis of Class 1, 1A, and 2 Development Permit application files from the Town of Carleton Place that were decided between 2011 and 2015. Upon the completion of the content analysis, it was found that minor variance issues were most commonly assigned to Class 1 applications. Planning staff generally met approval timelines and embraced their delegated approval authority, readily attaching conditions to applications in order to mitigate off-site impacts. While staff met the regulatory requirements of the DPS, ‘minor variance’ applications were largely decided on impact alone, demonstrating that the principles established by the four tests, the defining quality of the minor variance approval process, had not transferred to the Development Permit System. Alternatively, there was some evidence that the development community has not fully adjusted to the requirements of the new approvals process, as some applications were supported using a rationale containing the four tests. Subsequently, a set of four recommendations were offered which reflect the main themes established by the findings. The first two recommendations are directed towards the Province, the third to municipalities and the fourth to developers and planning consultants: 1) Amend Ontario Regulation 608/06 so that provisions under Section 4(3)(e) fall under Section 4(2). 2) Change the rhetoric from “combining elements of minor variances” to “replacing minor variances”. 3) Establish clear evaluation criteria. 4) Understand the evaluative criteria of the municipality in which you are working.