An Analysis of the Implementation and Perceived Effectiveness of the SchoolMAX Family Portal

ABSTRACT Title of Document: AN ANALYSIS OF THE IMPLEMENTATION AND PERCEIVED EFFECTIVENESS OF THE SCHOOLMAX FAMILY PORTAL Warren Wesley Watts, Doctor of Education, 2015 Directed By: Margaret J. McLaughlin, Ph.D. Department of Counseling, Higher Education and Special Education School districts have spent millions of dollars implementing student information systems that offer family portals with web-based access to parents and students. One of the main purposes of these systems is to improve school-to-home communication. Research has shown that when school-to-home communication is implemented effectively, parent involvement improves and student achievement increases (Epstein, 2001). The purpose of the study was to (a) understand why parents used or refrained from using the family portal and (b) determine what barriers to use might exist. To this end, this descriptive study identified the information parent users accessed in the SchoolMAX family portal, determined how frequently parents accessed the portal, and ascertained whether parents perceived an increase in communication with their children about academic matters after they began accessing the portal. Finally, the study sought to identify whether barriers existed that prevented parents from using the family portal. The inquiry employed three data sources to answer the aforementioned queries. These sources included (a) a survey sent electronically to 19,108 parents who registered online for the SchoolMAX family portal; (b) SchoolMAX portal usage data from the student information system for system usage between January 1, 2015 and June 30, 2015; and (c) a paper survey sent to 691 parents of students that had never used the SchoolMAX family portal in one elementary school, one middle school and one high school that were representative of other schools in the district. Survey results indicated that parents at all grade levels used the family portal. Usage data also confirmed that approximately 19% of the students had parents who monitored their progress through the family portal. Usage data also showed that parents were monitoring approximately 25% of students in secondary schools (6th – 12th grade) and 16% of students in elementary schools. Of the wide menu of resources available through the SchoolMAX family portal, parents used three areas most frequently: attendance, daily grades, and report cards. Approximately 70% of parents responded that their communication had improved with their children about academic matters since they started using the SchoolMAX family portal, and 90% of parents responded that the SchoolMAX family portal was an effective or somewhat effective tool. Parents also expressed interest in the addition of additional information to the SchoolMAX family portal. Specifically, the top three additions parents wanted to see included homework assignments, high stakes test scores, and graduation requirements. Parents also reported that 92% of them spoke to their children at least 2 to 3 times per week about academics. Due to the low response rate of the parent non-user survey, potential barriers to using the SchoolMAX family portal could not be addressed in this study. However, this issue may be a useful research topic in a future study. Keywords: school to home communication, student information systems, family portal, parent portal

Assessment and curriculum reform in China : the college English test and tertiary English as a foreign language education.

Educational assessment was a worldwide commonplace practice in the last century. With the theoretical underpinnings of education shifting from behaviourism and social efficiency to constructivism and cognitive theories in the past two decades, the assessment theories and practices show a widespread changing movement. The emergent assessment paradigm, with a futurist perspective, indicates a deviation away from the prevailing large scale high-stakes standardised testing and an inclination towards classroom-based formative assessment. Innovations and reforms initiated in attempts to achieve better education outcomes for a sustainable future via more developed learning and assessment theories have included the 2007 College English Reform Program (CERP) in Chinese higher education context. This paper focuses on the College English Test (CET) - the national English as a Foreign Language (EFL) testing system for non-English majors at tertiary level in China. It seeks to explore the roles that the CET played in the past two College English curriculum reforms, and the new role that testing and assessment assumed in the newly launched reform. The paper holds that the CET was operationalised to uplift the standards. However, the extended use of this standardised testing system brings constraints as well as negative washback effects on the tertiary EFL education. Therefore in the newly launched reform -CERP, a new assessment model which combines summative and formative assessment approaches is proposed. The testing and assessment, assumed a new role - to engender desirable education outcomes. The question asked is: will the mixed approach to formative and summative assessment provide the intended cure to the agony that tertiary EFL education in China has long been suffering - spending much time, yet achieving little effects? The paper reports the progresses and challenges as informed by the available research literature, yet asserts a lot needs to be explored on the potential of the assessment mix in this examination tradition deep-rooted and examination-obsessed society.

A controlled rapid approach to curriculum change : addressing the pressures of increased test-based accountability on schools

Increasingly, large amounts of public and private money are being invested in education and as a result, schools are becoming more accountable to stakeholders for this financial input. In terms of the curriculum, governments worldwide are frequently tying school funding to students‟ and schools‟ academic performances, which are monitored through high-stakes testing programs. To accommodate the resultant pressures from these testing initiatives, many principals are re-focussing their school‟s curriculum on the testing requirements. Such a re-focussing, which was examined critically in this thesis, constituted an externally facilitated rapid approach to curriculum change. In line with previously enacted change theories and recommendations from these, curriculum change in schools has tended to be a fairly slow, considered, collaborative process that is facilitated internally by a deputy-principal (curriculum). However, theoretically based research has shown that such a process has often proved to be difficult and very rarely successful. The present study reports and theorises the experiences of an externally facilitated process that emerged from a practitioner model of change. This case study of the development of the controlled rapid approach to curriculum change began by establishing the reasons three principals initiated curriculum change and why they then engaged an outsider to facilitate the process. It also examined this particular change process from the perspectives of the research participants. The investigation led to the revision of the practitioner model as used in the three schools and challenged the current thinking about the process of school curriculum change. The thesis aims to offer principals and the wider education community an alternative model for consideration when undertaking curriculum change. Finally, the thesis warns that, in the longer term, the application of study‟s revised model (the Controlled Rapid Approach to Curriculum Change [CRACC] Model) may have less then desirable educational consequences.

Interactional competence in a paired speaking test : features salient to raters

Paired speaking tests are now commonly used in both high-stakes testing and classroom assessment contexts. The co-construction of discourse by candidates is regarded as a strength of paired speaking tests, as candidates have the opportunity to display a wider range of interactional competencies, including turn taking, initiating topics and engaging in extended discourse with a partner, rather than an examiner. However, the impact of the interlocutor in such jointly negotiated discourse and the implications for assessing interactional competence are areas of concern. This article reports on the features of interactional competence that were salient to four trained raters of 12 paired speaking tests through the analysis of rater notes, stimulated verbal recalls and rater discussions. Findings enabled the identification of features of the performance noted by raters when awarding scores for interactional competence, and the particular features associated with higher and lower scores. A number of these features were seen by the raters as mutual achievements, which raises the issue of the extent to which it is possible to assess individual contributions to the co-constructed performance. The findings have implications for defining the construct of interactional competence in paired speaking tests and operationalising this in rating scales.

Interaction in a Paired Speaking Test : the Rater's Perspective

Paired speaking tests are increasingly used in both low-and high-stakes second language assessment contexts. Until recently, very little was known about the way in which raters interpret and apply descriptors relating to interactional competence to a performance that is co-constructed. This book presents a study which explores the interactional features of a paired speaking test that were sailient to raters and the extent to which raters viewed the performance as separable. The study shows that raters use their own frames of reference to interpret descriptors and that they viewed certain features of the performance as mutual accomplishments. The book takes us 'beyond scores', and in doing so, contributes to the growing body of research on paired speaking tests.

The Relationship of Student Demographics to 10th Grade MCAS Test Anxiety

The intensification of consequential testing situations is associated with an increase in anxiety among American students (Casbarro, 2005). Test anxiety can have negative effects on student test performance (Everson, Millsap, & Rodriguez, 1991). If test anxiety has the potential to decrease students’ test scores, it becomes a factor that can threaten the validity of any inferences drawn between test scores and student progress (Cizek & Burg, 2006). There are several factors that relate closely to test anxiety (Cizek & Burg, 2006). Variables of key influence include gender, socioeconomic status, and teacher-manifested anxiety (Hembree, 1988). Another influence upon test anxiety is students’ participation in academic support programs to prepare them for exit examinations. The purpose of this study was to examine the relationship between 10th grade high school student gender, socioeconomic status, perceived teacher anxiety, and student preparedness with levels of the Massachusetts Comprehensive Assessment System (MCAS) test anxiety. It appears that few studies have examined levels of high school test anxiety in regards to this specific high-stakes MCAS exit exam required for high school graduation. A two-phase sequential mixed-methods research design was used to survey (N=156) 10th grade students represented by a sampling of (n=80) students with low socioeconomic status and (n=76) students with high socioeconomic status regarding their levels of test anxiety in relation to upcoming MCAS testing. A multiple regression analysis was used to measure the relationship between the predictor variables (gender, socioeconomic status, perceived teacher anxiety, and student preparedness) with the criterion variable of student test anxiety using the Test Anxiety Inventory (TAI). Personal interviews with (n=20) volunteer students provided rich explanations of students’ academic self-efficacy, their perceptions of their performance on the upcoming MCAS exam, and their use of strategies to reduce their levels of test anxiety. Personal interviews with (n=12) volunteer school administrators and teachers provided descriptions of their perceptions of how test anxiety affected their students’ performance. A major quantitative finding of this study was that the variables of student socioeconomic status and student ratings of teacher anxiety accounted for the variance in students’ levels of surveyed test anxiety (R2 = .06, p = .033, small to medium effect size). These results indicate that different student populations vary in their readiness skills to successfully participate in consequential testing situations. Consequently, high-test anxious students would require emotional preparation as well as academic preparation when confronting high-stakes testing. The results have the potential to re-shape the format of schools’ MCAS test preparation efforts.

Uncovering the root cause of ethnic difference in ability testing:differential test functioning, test familiarity and trait optimism as explanations of ethnic group differences

The present research represents a coherent approach to understanding the root causes of ethnic group differences in ability test performance. Two studies were conducted, each of which was designed to address a key knowledge gap in the ethnic bias literature. In Study 1, both the LR Method of Differential Item Functioning (DIF) detection and Mixture Latent Variable Modelling were used to investigate the degree to which Differential Test Functioning (DTF) could explain ethnic group test performance differences in a large, previously unpublished dataset. Though mean test score differences were observed between a number of ethnic groups, neither technique was able to identify ethnic DTF. This calls into question the practical application of DTF to understanding these group differences. Study 2 investigated whether a number of non-cognitive factors might explain ethnic group test performance differences on a variety of ability tests. Two factors – test familiarity and trait optimism – were able to explain a large proportion of ethnic group test score differences. Furthermore, test familiarity was found to mediate the relationship between socio-economic factors – particularly participant educational level and familial social status – and test performance, suggesting that test familiarity develops over time through the mechanism of exposure to ability testing in other contexts. These findings represent a substantial contribution to the field’s understanding of two key issues surrounding ethnic test performance differences. The author calls for a new line of research into these performance facilitating and debilitating factors, before recommendations are offered for practitioners to ensure fairer deployment of ability testing in high-stakes selection processes.

Proving My Competency One Test at a Time: Internationally Educated Nurses and the Canadian English Language Benchmark Assessment for Nurses

Abstract Professional language assessment is a new concept that has great potential to benefit Internationally Educated Professionals and the communities they serve. This thesis reports on a qualitative study that examined the responses of 16 Canadian English Language Benchmark Assessment for Nurses (CELBAN) test-takers on the topic of their perceptions of the CELBAN test-taking experience in Ontario in the winter of 2015. An Ontario organization involved in registering participants distributed an e-mail through their listserv. Thematic analyses of focus group and interview transcripts identified 7 themes from the data. These themes were used to inform conclusions to the following questions: (1) How do IENs characterize their assessment experience? (2) How do IENs describe the testing constructs measured by the CELBAN? (3) What, if any, potential sources of construct irrelevant variance (CIV) do the test-takers describe based on their assessment experience? (4) Do IENs feel that the CELBAN tasks provide a good reflection of the types of communicative tasks required of a nurse? Overall, participants reported positive experiences with the CELBAN as an assessment of their language skills, and noted some instances in which they felt some factors external to the assessment impacted their demonstration of their knowledge and skill. Lastly, some test-takers noted the challenge of completing the CELBAN where the types of communicative nursing tasks included in the assessment differed from nursing tasks typical of an IENs country or origin. The findings are discussed in relation to literature on high-stakes large-scale assessment and IEPs, and a set of recommendations are offered to future CELBAN administration. These recommendations include (1) the provision of a webpage listing all licensure requirements (2) monitoring of CELBAN location and dates in relation to the wider certification timeline for applicants (3) The provision of additional CELBAN preparatory materials (4) Minor changes to the CELBAN administrative protocols. Given that the CELBAN is a relatively new assessment format and its widespread use for high-stakes decisions (a component of nursing certification and licensure), research validating IEN-test-taker responses to construct representation and construct irrelevant variance is critical to our understanding of the role of competency testing for IENs.

Warning Against Faking: Boon or Bane? Effects on Test Score Validity

Previous research found personality test scores to be inflated on average among individuals who were motivated to present themselves in a desirable fashion in high stakes situations, such as during the employee selection process. One apparently effective way to reduce the undesirable test score inflation in such situations was to warn participants against faking. This research set out to investigate whether warning against faking would indeed affect personality test scores in the theoretically expected fashion. Contrary to expectations, the results did not support the hypothesized causal chain. Results across three studies show that while a warning may lower test scores in participants motivated to respond desirably (i.e., to fake), the effect of warning on test scores was not fully mediated by: a reduction in motivation to do well and self-reports of exaggerated responses in the personality test. Theoretical and practical implications are discussed.

Testing for accountability

Current graduates in education are entering a very different profession to the one in which most of their “baby-boomer” colleagues started. It is a profession in which accountability and national high-stakes testing (e.g. NAPLAN) have become catch-cries, and where the interpretation and use of educational data is an additional challenge. This has led to schools focusing on performance, and teachers now have to analyse test data and apply the findings to their teaching.

Raising the stakes : the challenges for teacher assessment

These are changing times for teachers and their students in Australia with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to make more use of assessment to support and improve learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the alleged purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are key questions that will be addressed in this presentation with reference to innovative research from global networks that have maintained the assessment focus on learning.

Assessment policy change in relation to English language teaching and learning in China : a study of perspectives from two universities

In response to concerns about the quality of English Language Learning (ELL) education at tertiary level, the Chinese Ministry of Education (CMoE) launched the College English Reform Program (CERP) in 2004. By means of a press release (CMoE, 2005) and a guideline document titled College English Curriculum Requirements (CECR) (CMoE, 2007), the CERP proposed two major changes to the College English assessment policy, which were: (1) the shift to optional status for the compulsory external test, the College English Test Band 4 (CET4); and (2) the incorporation of formative assessment into the existing summative assessment framework. This study investigated the interactions between the College English assessment policy change, the theoretical underpinnings, and the assessment practices within two Chinese universities (one Key University and one Non-Key University). It adopted a sociocultural theoretical perspective to examine the implementation process as experienced by local actors of institutional and classroom levels. Systematic data analysis using a constant comparative method (Merriam, 1998) revealed that contextual factors and implementation issues did not lead to significant differences in the two cases. Lack of training in assessment and the sociocultural factors such as the traditional emphasis on the product of learning and hierarchical teacher/students relationship are decisive and responsible for the limited effect of the reform.

The ascendancy of oracy in university courses: Implications for teachers and second language users

This chapter reports on a study of oracy in a first-year university Business course, with particular interest in the oracy demands for second language-using international students. The research is relevant at a time when Higher Education is characterised by the confluence of increased international enrolments, more dialogic teaching and learning, and imperatives for teamwork and collaboration. Data sources for the study included videotaped lectures and tutorials, course documents, student surveys, and an interview with the lecturer. The findings pointed to a complex, oracy-laden environment where interactive talk fulfilled high-stakes functions related to social inclusion, the co-construction of knowledge, and the accomplishment of assessment tasks. The salience of talk posed significant challenges for students negotiating these core functions in their second language. The study highlights the oracy demands in university courses and foregrounds the need for university teachers, curriculum writers and speaking test developers to recognise these demands and explicate them for the benefit of all students.

Raising the stakes: the challenges for teacher assessment

These are changing times in Australia for teachers and their students, with the introduction of a national curriculum and standards driven reform. While countries in Europe such as England, and in Asia such as Singapore, are changing policy to use assessment in the support of and improvement of learning it appears that we in Australia are moving towards creating policy that will raise the assessment stakes for the putative purposes of transparency, accountability and fairness. What can be learnt from countries that have had years of high stakes testing? How can Australia avoid the mistakes of past curriculum and assessment reform efforts? And how can Australian teachers build their capacity to maximise their use of the learning power of assessment? These are the questions that are addressed in this article, with reference to innovative research from global networks that have maintained the assessment focus on learning rather than accountability practices.

Data analysis skills for teachers

Although accountability in the form of high stakes testing is in favour in the contemporary Australian educational context, this practice remains a highly contested source of debate. Proponents for high stakes tests claim that higher standards in teaching and learning result from their implementation, whereas others believe that this type of testing regime is not required and may even in fact be counterproductive. Regardless of what side of the debate you sit on, the reality is that at present, high stakes testing appears to be here to stay. It could therefore be argued it is essential that teachers understand accountability and possess the specific skills to interpret and use test data beneficially.