The stakes of politics and electoral administration : a comparative study of Southeast Asian democracies

Elections play a crucial role in political stability in post-democratization, and electoral administrations are the key to the electoral process. However, not all newly democratized countries have established reliable electoral administration. New democracies in Southeast Asia, such as the Philippines, Indonesia and Thailand, have independent election commissions which have different characteristics, especially in terms of neutrality. Based on three cases, this paper claims that the stakes of politics are the major determinant of the variations in neutrality. The high stakes of politics in Thailand brought about the partisan election commission, while the low stakes in Indonesia made the electoral system relatively neutral. Like Thailand, the high stakes of politics in the Philippines also cause political intervention in the electoral administration.

Factors associated with prompt difficulty in automated essay scoring

This study explores factors related to the prompt difficulty in Automated Essay Scoring. The sample was composed of 6,924 students. For each student, there were 1-4 essays, across 20 different writing prompts, for a total of 20,243 essays. E-rater® v.2 essay scoring engine developed by the Educational Testing Service was used to score the essays. The scoring engine employs a statistical model that incorporates 10 predictors associated with writing characteristics of which 8 were used. The Rasch partial credit analysis was applied to the scores to determine the difficulty levels of prompts. In addition, the scores were used as outcomes in the series of hierarchical linear models (HLM) in which students and prompts constituted the cross-classification levels. This methodology was used to explore the partitioning of the essay score variance.^ The results indicated significant differences in prompt difficulty levels due to genre. Descriptive prompts, as a group, were found to be more difficult than the persuasive prompts. In addition, the essay score variance was partitioned between students and prompts. The amount of the essay score variance that lies between prompts was found to be relatively small (4 to 7 percent). When the essay-level, student-level-and prompt-level predictors were included in the model, it was able to explain almost all variance that lies between prompts. Since in most high-stakes writing assessments only 1-2 prompts per students are used, the essay score variance that lies between prompts represents an undesirable or "noise" variation. Identifying factors associated with this "noise" variance may prove to be important for prompt writing and for constructing Automated Essay Scoring mechanisms for weighting prompt difficulty when assigning essay score.^

Proving My Competency One Test at a Time: Internationally Educated Nurses and the Canadian English Language Benchmark Assessment for Nurses

Abstract Professional language assessment is a new concept that has great potential to benefit Internationally Educated Professionals and the communities they serve. This thesis reports on a qualitative study that examined the responses of 16 Canadian English Language Benchmark Assessment for Nurses (CELBAN) test-takers on the topic of their perceptions of the CELBAN test-taking experience in Ontario in the winter of 2015. An Ontario organization involved in registering participants distributed an e-mail through their listserv. Thematic analyses of focus group and interview transcripts identified 7 themes from the data. These themes were used to inform conclusions to the following questions: (1) How do IENs characterize their assessment experience? (2) How do IENs describe the testing constructs measured by the CELBAN? (3) What, if any, potential sources of construct irrelevant variance (CIV) do the test-takers describe based on their assessment experience? (4) Do IENs feel that the CELBAN tasks provide a good reflection of the types of communicative tasks required of a nurse? Overall, participants reported positive experiences with the CELBAN as an assessment of their language skills, and noted some instances in which they felt some factors external to the assessment impacted their demonstration of their knowledge and skill. Lastly, some test-takers noted the challenge of completing the CELBAN where the types of communicative nursing tasks included in the assessment differed from nursing tasks typical of an IENs country or origin. The findings are discussed in relation to literature on high-stakes large-scale assessment and IEPs, and a set of recommendations are offered to future CELBAN administration. These recommendations include (1) the provision of a webpage listing all licensure requirements (2) monitoring of CELBAN location and dates in relation to the wider certification timeline for applicants (3) The provision of additional CELBAN preparatory materials (4) Minor changes to the CELBAN administrative protocols. Given that the CELBAN is a relatively new assessment format and its widespread use for high-stakes decisions (a component of nursing certification and licensure), research validating IEN-test-taker responses to construct representation and construct irrelevant variance is critical to our understanding of the role of competency testing for IENs.

Evaluating the efficiency of community-based HIV testing and counseling strategies in sub-Saharan Africa

Thesis (Ph.D.)--University of Washington, 2016-08

School leaders as both colonized and colonizers: understanding professional identity in an era of no child left behind

This study positioned the federal No Child Left Behind (NCLB) Act of 2002 as a reified colonizing entity, inscribing its hegemonic authority upon the professional identity and work of school principals within their school communities of practice. Pressure on educators and students intensifies each year as the benchmark for Adequate Yearly Progress under the NCLB policy is raised, resulting in standards-based reform, scripted curriculum and pedagogy, absence of elective subjects, and a general lack of autonomy critical to the work of teachers as they approach each unique class and student (Crocco & Costigan, 2007; Mabry & Margolis, 2006). Emphasis on high stakes standardized testing as the indicator for student achievement (Popham, 2005) affects educators’ professional identity through dramatic pedagological and structural changes in schools (Day, Flores, & Viana, 2007). These dramatic changes to the ways our nation conducts schooling must be understood and thought about critically from school leaders’ perspectives as their professional identity is influenced by large scale NCLB school reform. The author explored the impact No Child Left Behind reform had on the professional identity of fourteen, veteran Illinois principals leading in urban, small urban, suburban, and rural middle and elementary schools. Qualitative data were collected during semi-structured interviews and focus groups and analyzed using a dual theoretical framework of postcolonial and identity theories. Postcolonial theory provided a lens from which the author applied a metaphor of colonization to principals’ experiences as colonized-colonizers in a time of school reform. Principal interview data illustrated many examples of NCLB as a colonizing authority having a significant impact on the professional identity of school leaders. This framework was used to interpret data in a unique and alternative way and contributed to the need to better understand the ways school leaders respond to district-level, state-level, and national-level accountability policies (Sloan, 2000). Identity theory situated principals as professionals shaped by the communities of practice in which they lead. Principals’ professional identity has become more data-driven as a result of NCLB and their role as instructional leaders has intensified. The data showed that NCLB has changed the work and professional identity of principals in terms of use of data, classroom instruction, Response to Intervention, and staffing changes. Although NCLB defines success in terms of meeting or exceeding the benchmark for Adequate Yearly Progress, principals’ view AYP as only one measurement of their success. The need to meet the benchmark for AYP is a present reality that necessitates school-wide attention to reading and math achievement. At this time, principals leading in affluent, somewhat homogeneous schools typically experience less pressure and more power under NCLB and are more often labeled “successful” school communities. In contrast, principals leading in schools with more heterogeneity experience more pressure and lack of power under NCLB and are more often labeled “failing” school communities. Implications from this study for practitioners and policymakers include a need to reexamine the intents and outcomes of the policy for all school communities, especially in terms of power and voice. Recommendations for policy reform include moving to a growth model with multi-year assessments that make sense for individual students rather than one standardized test score as the measure for achievement. Overall, the study reveals enhancements and constraints NCLB policy has caused in a variety of school contexts, which have affected the professional identity of school leaders.

Multiple-choice versus short-response items: Differences in omit behavior

The overall rate of omission of items for 28,331 17 year old Australian students on a high stakes test of achievement in the common elements or cognitive skills of the senior school curriculum is reported for a subtest in multiple choice format and a subtest in short response format. For the former, the omit rates were minuscule and there was no significant difference by gender or by type of school attended. For the latter, where an item can be 'worth' up to five times that of a single multiple choice item, the omit rates were between 10 and 20 times that for multiple choice and the difference between male and female omit rate was significant as was the difference between students from government and non-government schools. For both formats, females from single sex schools omitted significantly fewer items than did females from co-educational schools. Some possible explanations of omit behaviour are alluded to.

Assessment in education : principles, policy and practice

The English examination system provides a market in which a limited number of providers are accredited to offer curriculum-based examinations in many subject areas and at several levels. The most significant are the General Certificate of Secondary Education (GCSE) and the General Certificate of Education, Advanced (A level). Because these examinations are used for high-stakes purposes, including higher education and employment selection for individuals and programme evaluation for institutions, it is desired that scores from various exams be ‘comparable’ in several respects: across syllabuses and examination boards within a subject area, across years, and even across subject areas. Just how to accomplish this goal has been a topic of continual research and debate for over 50 years, through many changes of examination and institutional structures. But ever year, tens of thousands of scores must be reported, and every year, users expect them to ‘be comparable’ and use them as if they are.

Social studies disciplinary knowledge: Tensions between state curriculum and national assessment requirements

In this chapter, we are particularly concerned with making visible the general principles underlying the transmission of Social Studies curriculum knowledge, and considering it in light of a high-stakes mandated national assessment task. Specifically, we draw on Bernstein’s theoretical concept of pedagogic models as a tool for analysing orientations to teaching and learning. We introduce a case in point from the Australian context: one state Social Studies curriculum vis-a-vis one part of the Year Three national assessment measure for reading. We use our findings to consider the implications for the disciplinary knowledge of Social Studies in the communities in which we are undertaking our respective Australian Research Council Linkage project work (Glasswell et al.; Woods et al.). We propose that Social Studies disciplinary knowledge is being constituted, in part, through power struggles between different agencies responsible for the production and relay of official forms of state curriculum and national literacy assessment. This is particularly the case when assessment instruments are used to compare and contrast school results in highly visible web based league tables (see, for example, http://myschoolaustralia.ning.com/).

Engaging data-literate beginning teachers

Gen Y beginning teachers have an edge: they’ve grown up in an era of educational accountability, so when their students have to sit a high-stakes test, they can relate.

Parents’ risky choices around the IB Diploma

In recent years there has been a rapid growth in the International Baccalaureate Diploma(IBD), a secondary curriculum administered by the International Baccalaureate Organisation(IBO), as an alternative to the local curriculum in Australian schools in some schools. This growth is indicative of an increasing demand from Australian families for new educational structures, practices and processes. With more curriculum options and pathways such as the IBD available in the secondary education system, parents are faced with a more complex high stakes decision when it comes to choosing the optimal education path for their offspring, one which requires a careful assessment of potential outcomes and risks. This paper reports on the responses of 184 parents to an online survey conducted in 26 Australian schools that offer the IBD as a curricular alternative. It examines which parents either chose, or chose not to, enrol their children in the program, why, and what risks they perceived to be associated with that choice. The paper will compare the choice behaviour of the two groups of parents from a sociological perspective, framing the enquiry with reference to globalisation and neo-liberal education policy and its effect on parental choice of schooling. This paper will make evident how parental choice of educational alternatives has become a more complicated process for Australian families.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

What Could Socially-just Literacy Instruction Look Like?

Schools in Australia today are expected to improve student outcomes in an increasingly constrained context focused on accountability. At the same time as teachers work within these constraints, they are also working to ensure that all students are able to achieve quality outcomes from schooling while dealing with increasing diversity of the student population. So what is a teacher’s role in providing equitable possibilities for all of the students in their care? Teachers are left with the difficult task of balancing the many mandatory requirements placed on them, and on their students, such as gaining improvements on high stakes test scores, with the important work of dealing with individual students and student cohorts in equitable and socially just ways. This impacts on the work of all teachers, however for those teachers working in schools in contexts of complexity and duress these pressures are never greater.

The ascendancy of oracy in university courses: Implications for teachers and second language users

This chapter reports on a study of oracy in a first-year university Business course, with particular interest in the oracy demands for second language-using international students. The research is relevant at a time when Higher Education is characterised by the confluence of increased international enrolments, more dialogic teaching and learning, and imperatives for teamwork and collaboration. Data sources for the study included videotaped lectures and tutorials, course documents, student surveys, and an interview with the lecturer. The findings pointed to a complex, oracy-laden environment where interactive talk fulfilled high-stakes functions related to social inclusion, the co-construction of knowledge, and the accomplishment of assessment tasks. The salience of talk posed significant challenges for students negotiating these core functions in their second language. The study highlights the oracy demands in university courses and foregrounds the need for university teachers, curriculum writers and speaking test developers to recognise these demands and explicate them for the benefit of all students.

…Where did I lose you? Accessing the literacy demands of assessment

In this study we sought to find out how teachers could make assessment fairer for Indigenous students in learning mathematics, given the context of the high stakes of the National Assessment Program Literacy and Numeracy (NAPLAN). Today, teachers are experiencing the full range of demands from their own students who require individual attention, through to system level expectations of improved performances for all students. Many staff experience reform fatigue with limited time for critical reflection and a reduction in support for the use and the analysis of the overwhelming amount of data that has become available in recent years. Over the past three years we worked with teachers in seven schools to gradually refine our research focus to centre on how we might best support teachers in this demanding context with the important outcome of improved teaching and learning of mathematics with particular consideration of how to respond to the cultural needs of Indigenous (Aboriginal and Torres Strait Islander) students.

Low SES primary school students engaging in an after-school robotics program

Recent attention in education within many western contexts has focused on improved outcomes for students, with a particular focus on closing the gap between those who come from disadvantaged backgrounds and the rest of the student population. Much of this attention has supported a set of simplistic solutions to improving scores on high stakes standardized tests. The collateral damage (Nichols & Berliner, 2007) of such responses includes a narrowing of the curriculum, plateaus in gain scores on the tests, and unproductive blame games aimed by the media and politicians at teachers and communities (Nichols & Berliner, 2007; Synder, 2008). Alternative approaches to improving the quality and equity of schooling remain as viable alternatives to these measures. As an example in a recent study of school literacy reform in low SES schools, Luke, Woods and Dooley (2011) argued for the increase of substantive content and intellectual quality of the curriculum as a necessary means to re-engaging middle school students, improving outcomes of schooling and achieving a high quality, high equity system. The MediaClub is an afterschool program for students in years 4 to 7 (9-12 year old) at a primary school in a low SES area of a large Australian city. It is run as part of an Australian Research Council funded research project. The aim of the program has been to provide an opportunity for students to gain expertise in digital technologies and media literacies in an afterschool setting. It was hypothesized that this expertise might then be used to shift the ways of being literate that these students had to call on within classroom teaching and learning events. Each term, there is a different focus on digital media, and information and communication technology (ICT) activities in the MediaClub. The work detailed in this chapter relates to a robotics program presented as one of the modules within this afterschool setting. As part of the program, the participants were challenged to find creative solutions to problems in a constructivist-learning environment.