Assessing the effectiveness of two-year banking and insurance technology programs of junior colleges in Taiwan, R.O.C.: An application of the Context Input Process Product (CIPP) evaluation model

The purpose of this descriptive study was to evaluate the banking and insurance technology curriculum at ten junior colleges in Taiwan. The study focused on curriculum, curriculum materials, instruction, support services, student achievement and job performance. Data was collected from a diverse sample of faculty, students, alumni, and employers. ^ Questionnaires on the evaluation of curriculum at technical junior colleges were developed for use in this specific case. Data were collected from the sample described above and analyzed utilizing ANOVA, T-Tests and crosstabulations. Findings are presented which indicate that there is room for improvement in terms of meeting individual students' needs. ^ Using Stufflebeam's CIPP model for curriculum evaluation it was determined that the curriculum was adequate in terms of the knowledge and skills imparted to students. However, students were dissatisfied with the rigidity of the curriculum and the lack of opportunity to satisfy the individual needs of students. Employers were satisfied with both the academic preparation of students and their on the job performance. ^ In sum, the curriculum of the two-year banking and insurance technology programs of junior college in Taiwan was shown to have served adequately preparing a work force to enter businesses. It is now time to look toward the future and adapt the curriculum and instruction for the future needs of the ever evolving high-tech society. ^

An analysis of student and instructor perceptions of students' writing skills at the university level

This study examined university students' writing skills as perceived by university students and their English instructors. The goal of the study was to provide English instructors with objective, quantified information about writing perceptions from both the students' and instructors' viewpoints. ^ A survey instrument was developed based on a survey instrument created by Newkirk, Cameron, and Selfe (1977) to identify instructors' perceived knowledge of student writing skills. The present study used a descriptive statistical design. It examined five writing skill areas: attitude, content, grammar and mechanics, literary considerations, and the writing process through a questionnaire completed by a convenience sample of summer and fall admitted freshmen who were enrolled in Essay Writing and Freshman Composition courses and English Department instructors at a large South Florida public university. ^ The study consisted of five phases. The first phase was modifying of the Newkirk, Cameron, and Selfe (1977) questionnaire. Two versions of the revised survey were developed - one for instructors and one for students. The second phase was pilot testing the questionnaire for evaluation of administration and scoring. The third phase was administering the questionnaire to 1,280 students and 48 instructors. The fourth phase was analyzing the data. The study found a significant difference in the perceptions of students and instructors in all areas of writing skills examined by the survey. Responses to 29 of 30 questions showed that students felt they had better attitudes toward writing and better writing skills than instructors thought. ^ The final phase was developing recommendations for practice. Based on findings and theory and empirical evidence drawn from the fields of adult education and composition research, learner-centered, self-directed curriculum guidelines are offered. ^ By objectively quantifying student and instructor perceptions of students' writing skills, this study contributes to a growing body of literature that: (a) encourages instructors to acknowledge the perception disparities between instructors and students; (b) gives instructors a better understanding of how to communicate with students; and (c) recommends the development of new curriculum, placement tests, and courses that meet the needs of students and enables English instructors to provide meaningful instruction. ^

GRE as a predictor of graduate student success at a Hispanic serving institution of higher education

Accurately predicting the success of graduate students is an important aspect of determining which students should be admitted into graduate programs. The GRE is a pivotal factor to examine since it is one of the most widely used criteria for graduate school admission. Even though the GRE is advertised as an accurate tool for predicting first year graduate GPA, there is a lack of research on long term success factors such as time to degree and graduate rate (Luthy, 1996; Powers, 2004). Furthermore, since most studies have low minority sample sizes, the validity of the GRE may not be the same across all groups (ETS, 2008b; Kuncel, Hezlett, & Ones, 2001). Another gap in GRE studies is that few researchers analyze student characteristics, which may alter or moderate the prediction validity of the GRE. Thus, student characteristics such as degree of academic involvement, mentorship interactions, and other academic and social experiences have not been widely examined in this context. These gaps in the analysis of GRE validity are especially relevant given the high attrition rates within of some graduate programs (e.g., an estimated 68% of doctoral student never complete their programs in urban universities; Lovitts, 2001). A sequential mixed methods design was used to answer the research questions in two phases. The quantitative phase used student data files to analyze the relationship of two success variables (graduation rate and graduate GPA) to the GRE scores as well as other academic and demographic graduate student characteristics. The qualitative phase served to complement the first phase by describing a wider range of characteristics from the 11 graduate students who were interviewed. Both proximal and distal moderators influence student behaviors and success in graduate school. In the first phase of the study, the GRE was the distal facilitator under analysis. Findings suggested that both the GRE Quantitative and the GRE Verbal were predictors of success for master’s students, but the GRE Quantitative was not predictive of success for doctoral students. Other student characteristics such as demographic variables and disciplinary area were also predictors of success for the population of students studied. In the second phase of the study, it was inconclusive whether the GRE was predictive of graduate student success; though it did influence access to graduate programs. Furthermore, proximal moderators such as student involvement, faculty/peer interactions, motivational factors, and program structure were perceived to be facilitators and/or detractors for success.

Developing a process to create and validate an instrument assessing student attainment of competencies at an intercultural university in Mexico

This study took place at one of the intercultural universities (IUs) of Mexico that serve primarily indigenous students. The IUs are pioneers in higher education despite their numerous challenges (Bertely, 1998; Dietz, 2008; Pineda & Landorf, 2010; Schmelkes, 2009). To overcome educational inequalities among their students (Ahuja, Berumen, Casillas, Crispín, Delgado et al., 2004; Schmelkes, 2009), the IUs have embraced performance-based assessment (PBA; Casillas & Santini, 2006). PBA allows a shared model of power and control related to learning and evaluation (Anderson, 1998). While conducting a review on PBA strategies of the IUs, the researcher did not find a PBA instrument with valid and reliable estimates. The purpose of this study was to develop a process to create a PBA instrument, an analytic general rubric, with acceptable validity and reliability estimates to assess students' attainment of competencies in one of the IU's majors, Intercultural Development Management. The Human Capabilities Approach (HCA) was the theoretical framework and a sequential mixed method (Creswell, 2003; Teddlie & Tashakkori, 2009) was the research design. IU participants created a rubric during two focus groups, and seven Spanish-speaking professors in Mexico and the US piloted using students' research projects. The evidence that demonstrates the attainment of competencies at the IU is a complex set of actual, potential and/or desired performances or achievements, also conceptualized as "functional capabilities" (FCs; Walker, 2008), that can be used to develop a rubric. Results indicate that the rubric's validity and reliability estimates reached acceptable estimates of 80% agreement, surpassing minimum requirements (Newman, Newman, & Newman, 2011). Implications for practice involve the use of PBA within a formative assessment framework, and dynamic inclusion of constituencies. Recommendations for further research include introducing this study's instrument-development process to other IUs, conducting parallel mixed design studies exploring the intersection between HCA and assessment, and conducting a case study exploring assessment in intercultural settings. Education articulated through the HCA empowers students (Unterhalter & Brighouse, 2007; Walker, 2008). This study aimed to contribute to the quality of student learning assessment at the IUs by providing a participatory process to develop a PBA instrument.

TEACHNJ: An Evaluation of Two Years Implementation

The Teacher Effectiveness and Accountability for the Children of New Jersey (TEACHNJ) Act was adopted by the New Jersey legislature in August 2012 with the intent to raise student achievement by improving the overall quality of instruction. As a result of this act, new teacher evaluation systems are being introduced in school districts across the state in an effort to more accurately assess teacher performance. The new teacher evaluations will be based on multiple classroom observations as well as the academic achievement of their students as measured on standardized tests. In addition, professional development opportunities are likely to change under this legislation, with schools customizing professional development programs to more effectively meet the needs of their teachers. The overarching question that informs our research is what impact will TEACH NJ have on the overall value of teacher evaluations and the quality of professional development opportunities offered to teachers. Data collected through survey research presents the pre-implementation practices (2011-2012 school year) as well as one year post-implementation practices (2013-2014) taking place in school districts throughout New Jersey. The findings reflect teachers’ perceptions of the value of the current teacher evaluation practices, the quality of the current professional development opportunities and the value the school administration places on teacher evaluations.

A study of the effects of a service-learning experience on student success at an urban community college

The purpose of this study was to explore the effects of a service-learning experience on student success as measured by class attendance, course completion, final course grades, and end-of-term evaluation data. Though many outcomes of service-learning experiences have been studied, including ethical values, self-esteem, student personal development, and career preparation, relatively few studies have been conducted on the effects of such experiences on academic achievement, and the studies that have been done have primarily studied students at traditional, four-year, residential universities. The study consisted of 286 students enrolled in six paired courses taught by five instructors at a community college in the Fall term 1996. One section of each pair (the control group) was taught using traditional subject matter and course materials and the other section of each pair (the treatment group) participated in a 20-hour required service- learning activity in addition to the regular course curriculum. The courses in the study included American History, Sociology, College Preparatory English, and Introduction to English Composition. The results of this study indicate that, overall, students who participated in a class in which service-learning was a requirement, achieved higher final course grades and reported greater satisfaction with the course, the instructor, the reading assignments, and the grading system, and the treatment section of one course pair had fewer absences. In addition, the faculty members reported that, in the treatment sections, class discussions were more stimulating, the sections seemed more vital in terms of student involvement, the students seemed more challenged academically, more motivated to learn, and seemed to exert more effort in the course.

Online Education, Accrediting Standards, and Student Success: An Examination of the Relationship Between the Southern Association of Colleges and Schools Commission on Colleges Standards for Online Education and Student Success

Prior to 2000, there were less than 1.6 million students enrolled in at least one online course. By fall 2010, student enrollment in online distance education showed a phenomenal 283% increase to 6.1 million. Two years later, this number had grown to 7.1 million. In light of this significant growth and skepticism about quality, there have been calls for greater oversight of this format of educational delivery. Accrediting bodies tasked with this oversight have developed guidelines and standards for online education. There is a lack of empirical studies that examine the relationship between accrediting standards and student success. The purpose of this study was to examine the relationship between the presence of Southern Association of Colleges and Schools Commission on College (SACSCOC) standards for online education in online courses, (a) student support services and (b) curriculum and instruction, and student success. An original 24-item survey with an overall reliability coefficient of .94 was administered to students (N=464) at Florida International University, enrolled in 24 university-wide undergraduate online courses during fall 2014, who rated the presence of these standards in their online courses. The general linear model was utilized to analyze the data. The results of the study indicated that the two standards, student support services and curriculum and instruction were both significantly and positively correlated with student success but with small R2 and strengths of association less than .35 and .20 respectively. Mixed results were produced from Chi-square tests for differences in student success between higher and lower rated online courses when controlling for various covariates such as discipline, gender, race/ethnicity, GPA, age, and number of online courses previously taken. A multiple linear regression analysis revealed that the curriculum and instruction standard was the only variable that accounted for a significant amount of unique variance in student success. Another regression test revealed that no significant interaction effect exists between the two SACSCOC standards and GPA in predicting student success. The results of this study are useful for administrators, faculty, and researchers who are interested in accreditation standards for online education and how these standards relate to student success.

An evaluation of a curriculum response to the State of Florida mandate for computer literacy at a large comprehensive high school in Dade County, Florida

Minimum Student Performance Standards in Computer Literacy and Science were passed by the Florida Legislature through the Educational Reform Act of 1983. This act mandated that all Florida high school graduates receive training in computer literacy. Schools and school systems were charged with the task of determining the best methods to deliver this instruction to their students. The scope of this study is to evaluate one school's response to the state of Florida's computer literacy mandate. The study was conducted at Miami Palmetto Senior High School, located in Dade County, Florida. The administration of Miami Palmetto Senior High School chose to develop and implement a new program to comply with the state mandate - integrating computer literacy into the existing biology curriculum. The study evaluated the curriculum to determine if computer literacy could be integrated successfully and meet both the biology and computer literacy objectives. The findings in this study showed that there were no significant differences between biology scores of the students taking the integrated curriculum and those taking a traditional curriculum of biology. Student in the integrated curriculum not only met the biology objectives as well as those in the traditional curriculum, they also successfully completed the intended objectives for computer literacy. Two sets of objectives were successfully completed in the integrated classes in the same amount of time used to complete one set of objectives in the traditional biology classes. Therefore, integrated curriculum was the more efficient means of meeting the intended objectives of both biology and computer literacy.

Design and evaluation of a consulting system for database design

Database design is a difficult problem for non-expert designers. It is desirable to assist such designers during the problem solving process by means of a knowledge based (KB) system. Although a number of prototype KB systems have been proposed, there are many shortcomings. Firstly, few have incorporated sufficient expertise in modeling relationships, particularly higher order relationships. Secondly, there does not seem to be any published empirical study that experimentally tested the effectiveness of any of these KB tools. Thirdly, problem solving behavior of non-experts, whom the systems were intended to assist, has not been one of the bases for system design. In this project, a consulting system, called CODA, for conceptual database design that addresses the above short comings was developed and empirically validated. More specifically, the CODA system incorporates (a) findings on why non-experts commit errors and (b) heuristics for modeling relationships. Two approaches to knowledge base implementation were used and compared in this project, namely system restrictiveness and decisional guidance (Silver 1990). The Restrictive system uses a proscriptive approach and limits the designer's choices at various design phases by forcing him/her to follow a specific design path. The Guidance system approach, which is less restrictive, involves providing context specific, informative and suggestive guidance throughout the design process. Both the approaches would prevent erroneous design decisions. The main objectives of the study are to evaluate (1) whether the knowledge-based system is more effective than the system without a knowledge-base and (2) which approach to knowledge implementation - whether Restrictive or Guidance - is more effective. To evaluate the effectiveness of the knowledge base itself, the systems were compared with a system that does not incorporate the expertise (Control). An experimental procedure using student subjects was used to test the effectiveness of the systems. The subjects solved a task without using the system (pre-treatment task) and another task using one of the three systems, viz. Control, Guidance or Restrictive (experimental task). Analysis of experimental task scores of those subjects who performed satisfactorily in the pre-treatment task revealed that the knowledge based approach to database design support lead to more accurate solutions than the control system. Among the two KB approaches, Guidance approach was found to lead to better performance when compared to the Control system. It was found that the subjects perceived the Restrictive system easier to use than the Guidance system.

Evaluation of creative problem solving abilities in undergraduate structural engineers through interdisciplinary problem based learning

For a structural engineer, effective communication and interaction with architects cannot be underestimated as a key skill to success throughout their professional career. Structural engineers and architects have to share a common language and understanding of each other in order to achieve the most desirable architectural and structural designs. This interaction and engagement develops during their professional career but needs to be nurtured during their undergraduate studies. The objective of this paper is to present the strategies employed to engage higher order thinking in structural engineering students in order to help them solve complex problem-based learning (PBL) design scenarios presented by architecture students. The strategies employed were applied in the experimental setting of an undergraduate module in structural engineering at Queen’s University Belfast in the UK. The strategies employed were active learning to engage with content knowledge, the use of physical conceptual structural models to reinforce key concepts and finally, reinforcing the need for hand sketching of ideas to promote higher order problem-solving. The strategies employed were evaluated through student survey, student feedback and module facilitator (this author) reflection. The strategies were qualitatively perceived by the tutor and quantitatively evaluated by students in a cross-sectional study to help interaction with the architecture students, aid interdisciplinary learning and help students creatively solve problems (through higher order thinking). The students clearly enjoyed this module and in particular interacting with structural engineering tutors and students from another discipline

Web-based museum trails on PDAs for university-level design students: Design and evaluation

This paper describes the development and evaluation of web-based museum trails for university-level design students to access on handheld devices in the Victoria and Albert Museum (V&A) in London. The trails offered students a range of ways of exploring the museum environment and collections, some encouraging students to interpret objects and museum spaces in lateral and imaginative ways, others more straightforwardly providing context and extra information. In a three-stage qualitative evaluation programme, student feedback showed that overall the trails enhanced students’ knowledge of, interest in, and closeness to the objects. However, the trails were only partially successful from a technological standpoint due to device and network problems. Broader findings suggest that technology has a key role to play in helping to maintain the museum as a learning space which complements that of universities as well as schools. This research informed my other work in visitor-constructed learning trails in museums, specifically in the theoretical approach to data analysis used, in the research design, and in informing ways to structure visitor experiences in museums. It resulted in a conference presentation, and more broadly informed my subsequent teaching practice.

The Development and Evaluation of a School-Based Programme to Promote and the Enhance Well-Being of Primary School Children

The current study was carried out to evaluate the impact of a well-being curriculum based on existing knowledge of themes within PP, which contribute to well-being. The Positive Well-Being Curriculum consists of twelve ninety minute sessions delivered weekly during a school term. The twelve well-being sessions fit into four domains: positive experience, positive emotions, positive relationships, achievement and meaning (Seligman, 2007). The objectives of the study were to test the practical implications of running a well-being curriculum, to develop a range of activities within each domain and to evaluate the impact on student well-being with regard to life satisfaction, positive affectivity and subjective happiness. A pilot was carried out as preparation for the main mixed method intervention study, which was conducted in two London primary schools. Pre and post data was collected using standardised measures, focus groups and one to one interviews. Findings from the pilot demonstrated a significant increase in well-being as demonstrated by increases in: life satisfaction, positive affect and subjective happiness. Additional information was gathered which informed the content and implementation of the curriculum in the main study. The experience of taking part in the study as evidenced through qualitative and quantitative results, indicate that the Positive Well-being Curriculum was perceived by participating teachers and children to contribute positively to the well-being of the children. These findings would be of interest to educational psychologists as there is an increasing interest by schools to include creative and validated resources to support and enhance the well-being of all children. A number of useful insights were developed about the usefulness of the curriculum for children in a variety of educational settings.

European student mobility and intercultural learning at a portuguese university

The purpose of this study is to understand, impact and evaluate the development of intercultural communicative competencies among European credit-seeking exchange students and other sojourners through purposeful intercultural pedagogy. This pedagogy encompasses intentional intercultural- educational approaches which aim to support and enhance sojourners’ intercultural learning throughout the study abroad cycle (pre-departure, in-country and reentry phases). To test and validate these pedagogies a 20-hour intervention was designed and implemented among two cohorts of 31 sojourners during the in-country phase of their sojourn in Portugal. The process to develop and validate the intercultural intervention was driven by a mixed-methods methodology which combined quantitative and qualitative data to triangulate, complement and expand research results from a pragmatic stance. The mixed methods research design adopted is multi-phased and encompasses a multi-case study and an evaluative component. The multi-case component is embodied by sojourner cohorts: (1) the primary case study involves 19 incoming students at the University of Aveiro (Portugal) as participants in the European exchange program Campus Europae; (2) the second case study comprises three incoming Erasmus students and nine highly skilled immigrants at the same university. All 31 sojourners attended two intermediate Portuguese as Foreign Language classrooms where the intervention was employed. Data collection was extensive and involved collecting, analyzing and mixing quantitative and qualitative strands across four research phases. These phases refer to the: (1) development, (2) implementation and (3) evaluation of the intervention, as well as to (4) a stakeholder analysis of the external value of the intervention and of the Campus Europae program. Data collection instruments included pre and posttest questionnaires and semi-structured interviews. Results confirm the intercultural effectiveness of the intervention and the positive impact upon research participants’ intercultural gains. This impact was, however, greater in case study 2. Among explanatory variables, three stand out: (1) participant meaning-making abilities, (2) host language proficiency and related variables, and (3) type of sojourn or exchange programs. Implications for further research highlight the need to systematize purposeful intercultural pedagogy in sojourner populations in general, and in European credit student mobility in particular. In the latter case, these pedagogies should be part of the design and delivery of credit-bearing exchange programs in pre- departure, in-country and re-entry phases. Implications for practice point to the urge to improve intercultural practices in: macro (higher education institutions), mezzo (exchange programs) and micro (sojourner language classrooms) contexts where this research took place, and wider social scenarios they represent.

EXAMINING THE IMPACT OF TEACHER PRACTICE ON STUDENT LEARNING IN INTERACTIVE RADIO INSTRUCTION (IRI) CLASSROOMS: LESSONS FROM NEW DELHI AND RAJASTHAN (INDIA)

This quantitative study examines the impact of teacher practices on student achievement in classrooms where the English is Fun Interactive Radio Instruction (IRI) programs were being used. A contemporary IRI design using a dual-audience approach, the English is Fun IRI programs delivered daily English language instruction to students in grades 1 and 2 in Delhi and Rajasthan through 120 30-minute programs via broadcast radio (the first audience) while modeling pedagogical techniques and behaviors for their teachers (the second audience). Few studies have examined how the dual-audience approach influences student learning. Using existing data from 32 teachers and 696 students, this study utilizes a multivariate multilevel model to examine the role of the primary expectations for teachers (e.g., setting up the IRI classroom, following instructions from the radio characters and ensuring students are participating) and the role of secondary expectations for teachers (e.g., modeling pedagogies and facilitating learning beyond the instructions) in promoting students’ learning in English listening skills, knowledge of vocabulary and use of sentences. The study finds that teacher practice on both sets of expectations mattered, but that practice in the secondary expectations mattered more. As expected, students made the smallest gains in the most difficult linguistic task (sentence use). The extent to which teachers satisfied the primary and secondary expectations was associated with gains in all three skills – confirming the relationship between students’ English proficiency and teacher practice in a dual-audience program. When it came to gains in students’ scores in sentence use, a teacher whose focus was greater on primary expectations had a negative effect on student performance in both states. In all, teacher practice clearly mattered but not in the same way for all three skills. An optimal scenario for teacher practice is presented in which gains in all three skills are maximized. These findings have important implications for the way the classroom teacher is cast in IRI programs that utilize a dual-audience approach and in the way IRI programs are contracted insofar as the role of the teacher in instruction is minimized and access is limited to instructional support from the IRI lessons alone.