Design, development and assessment of the Java Intelligent Tutoring System

The "Java Intelligent Tutoring System" (JITS) research project focused on designing, constructing, and determining the effectiveness of an Intelligent Tutoring System for beginner Java programming students at the postsecondary level. The participants in this research were students in the School of Applied Computing and Engineering Sciences at Sheridan College. This research involved consistently gathering input from students and instructors using JITS as it developed. The cyclic process involving designing, developing, testing, and refinement was used for the construction of JITS to ensure that it adequately meets the needs of students and instructors. The second objective in this dissertation determined the effectiveness of learning within this environment. The main findings indicate that JITS is a richly interactive ITS that engages students on Java programming problems. JITS is equipped with a sophisticated personalized feedback mechanism that models and supports each student in his/her learning style. The assessment component involved 2 main quantitative experiments to determine the effectiveness of JITS in terms of student performance. In both experiments it was determined that a statistically significant difference was achieved between the control group and the experimental group (i.e., JITS group). The main effect for Test (i.e., pre- and postiest), F( l , 35) == 119.43,p < .001, was qualified by a Test by Group interaction, F( l , 35) == 4.98,p < .05, and a Test by Time interaction, F( l , 35) == 43.82, p < .001. Similar findings were found for the second experiment; Test by Group interaction revealed F( 1 , 92) == 5.36, p < .025. In both experiments the JITS groups outperformed the corresponding control groups at posttest.

Student perceptions of the writing process

Based on the critical research paradigm and using a mix of methodologies, this study examined student perceptions of the process approach used to teach writing. A class of 19 ESL students in an academic writing class at a small university paliicipated in the study. As collaborators in the study, they assessed their personality types using the PET Type Check (Crantoll & Knoop, 1995) and tlleir learning styles using Kolb'sLearning Styles Inventory (1976). Interviews, classroom observations, and journals provided a data base for case studies llilQ teacher reflection. Results indicated that students perceived the prewriting step of brainstonning and peer review as most useful. Student perceptions of the tasks and course and implications for theory and practice are examined.

Writing and science connections : integrated instruction and assessment

Learning to write is a daunting task for many young children. The purpose of this study was to examine the impact of a combined approach to writing instruction and assessment on the writing performance of students in two grade 3 classes. Five forms and traits of writing were purposefully connected during writing lessons while exhibiting links to the four strands of the grade 3 Ontario science curriculum. Students then had opportunities to engage in the writing process and to self-assess their compositions using either student-developed (experimental group/teacher-researcher's class) or teachercreated (control group/teacher-participant's class) rubrics. Paired samples t-tests revealed that both the experimental and control groups exhibited statistically significant growth from pretest to posttest on all five integrated writing units. Independent samples t-tests showed that the experimental group outperformed the control group on the persuasive + sentence fluency and procedure + word choice writing tasks. Pearson product-moment correlation r tests revealed significant correlations between the experimental group and the teacher-researcher on the recount + ideas and report + organization tasks, while students in the control group showed significant correlations with the teacher-researcher on the narrative + voice and procedure + word choice tasks. Significant correlations between the control group and the teacher-participant were evident on the persuasive + sentence fluency and procedure + word choice tasks. Qualitative analyses revealed five themes that highlighted how students' self-assessments and reflections can be used to guide teachers in their instructional decision making. These findings suggest that educators should adopt an integrated writing program in their classrooms, while working with students to create and utilize purposeful writing assessment tools.

Investigating Multiple-Choice Assessment: Secondary School Teachers' Perceptions and Practices

Multiple-choice assessment is used within nearly all levels of education and is often heavily relied upon within both secondary and postsecondary institutions in determining a student’s present and future success. Understanding why it is effective or ineffective, how it is developed, and when it is or is not used by teachers can further inform teachers’ assessment practices, and subsequently, improve opportunities for student success. Twenty-eight teachers from 3 secondary schools in southern Ontario were interviewed about their perceptions and use of multiple-choice assessment and participated in a single-session introductory workshop on this topic. Perceptions and practices were revealed, discussed, and challenged through the use of a qualitative research method and examined alongside existing multiple-choice research. Discussion centered upon participants’ perspectives prior to and following their participation in the workshop. Implications related to future assessment practices and research in this field of assessment were presented. Findings indicated that many teachers utilized the multiple-choice form of assessment having had very little teacher education coursework or inservice professional development in the use of this format. The findings also revealed that teachers were receptive to training in this area but simply had not been exposed to or been given the opportunity to further develop their understanding. Participants generally agreed on its strengths (e.g., objectivity) and weaknesses (e.g., development difficulty). Participants were particularly interested in the potential for this assessment format to assess different levels of cognitive difficulty (i.e., levels beyond remembering of Bloom’s revised taxonomy), in addition to its potential to perhaps provide equitable means for assessing students of varying cultures, disabilities, and academic streams.

A Needs Assessment: Language Instruction Stakeholders Engaging in Project Based Learning Outside the Classroom for Effective Strategic Problem Solving

Abstract A noted benefit of Project Based Learning (PBL) as a teaching strategy is how it engages the student and enhances learning outcomes as a result of working through challenges intended to depict dilemmas outside the classroom. PBL has seldom been applied outside the parameters of the classroom curriculum. The current needs assessment carried out in this research project examined current practices of language instruction and International Administrative Professionals of both the private and public Language Industry. Participants responded to survey questions on their current administrative practices, strategies, and program characteristics. The study investigated the usefulness of a handbook on the procedure of assisting administrative service teams in language instruction settings to an engaged approach to PBL for student service issues. The diverse opinions, beliefs, and ideas, along with institutional policy, can provide beneficial framework ideas for future tools.

Globalization and Its Impact on Assessment: Moving Toward a New Story

Globalization has resulted in large-scale international and local assessments closely tied to notions of accountability and competitiveness in a globalized economy. Although policy makers seek to ensure citizens meet the demands of a global knowledge-based economy, such assessments may also impede the development of requisite 21st century skills. While standardization currently is viewed as the most effective measurement of student achievement, several Canadian and international jurisdictions are moving toward assessment for learning (AfL). This conceptual study sought to identify whether AfL or standardized assessment most effectively meets 21st century learning goals in the wake of rapid global change. It applies a Story Model theoretical framework to understand the current, the new emerging, and the future ideal story of education from a personal, cultural, and global lens. The study examines the main critiques and/or challenges of standardized testing, the benefits of AfL for student learning, and new teaching and assessment approaches to the development of 21st century learning goals. The study applies the Story Model’s inside-outside/past-future approach to determine the future direction of assessment. Results show that the new story of assessment will most likely entail a model that integrates both standardized testing and in-class assessments in the form of AfL and PBL.

Investigating Formative Assessment: Exploring the Impact on the Self-Efficacy and Motivation of Mathematics Students

This is a study of the implementation and impact of formative assessment strategies on the motivation and self-efficacy of secondary school mathematics students. An explanatory sequential mixed methods design was implemented where quantitative and qualitative data were collected and analyzed sequentially in 2 different phases. The first phase involved quantitative data from student questionnaires and the second phase involved qualitative data from individual student and teacher interviews. The findings of the study suggest that formative assessment is implemented in practice in diverse ways and is a process where the strategies are interconnected. Teachers experience difficulty in incorporating peer and self-assessment and perceive a need for exemplars. Key factors described as influencing implementation include teaching philosophies, interpretation of ministry documents, teachers’ experiences, leadership in administration and department, teacher collaboration, misconceptions of teachers, and student understanding of formative assessment. Findings suggest that overall, formative assessment positively impacts student motivation and self-efficacy, because feedback is provided which offers encouragement and recognition by highlighting the progress that has been made and what steps need to be taken to improve. However, students are impacted differently with some considerations including how students perceive mistakes and if they fear judgement. Additionally, the impact of formative assessment is influenced by the connection between self-efficacy and motivation, namely how well a student is doing is a source of both concepts.