Assessing learning in Higher Education

Assessing Learning in Higher Education addresses what is probably the most time-consuming part of the work of staff in higher education, and something to the complexity of which many of the recent developments in higher education have added. Getting assessment ‘right’– that is, designing and implementing appropriate models and methods, can determine the future lives and careers of students. But, as Professor Phil Race comments in his excellent and thought-provoking foreword, students entering higher education often have little idea about how exactly assessment will work, and often find that the process is very different from anything they have previously encountered. Assessing Learning in Higher Education contains innovative approaches to assessment drawn from many different cultures and disciplines. The chapter authors argue the need for changing assessment and feedback processes so that they embrace online collaboration and discussion between students as well as between ‘students’ and ‘faculty’. The chapters demonstrate that at some points there is a need to be able to measure individual achievement, and to do this in ways that are valid, transparent, authentic – and above all fair. Assessment and feedback processes need to ensure that students are well prepared for this individual assessment, but also to take account of collaboration and interaction. The respective chapters of Assessing Learning in Higher Education all of which are complete in themselves, but with very useful links to ideas in other chapters, provide numerous illustrations of how this can be achieved.

A study of learners' perceived value and satisfaction for implied effectiveness of online learning systems

Online learning systems (OLS) have become center stage for corporations and educational institutions as a competitive tool in the knowledge economy. The satisfaction construct has received extensive coverage in information systems literature as an indicator of effectiveness but has been criticized for lack of validity; yet, the value construct has been largely ignored, although it has a long history in psychology, sociology, and behavioral science. The purpose of this dissertation is to investigate the value and satisfaction constructs in the context of OLS, and their perceived by learners relationship for implied effectiveness of OLS. ^ First, a qualitative phase is employed to gather OLS values from learners' focus groups, followed by a pilot phase to refine a proposed instrument, and a main phase to validate the survey. Responses were received from 75 students in four focus groups, 141 in the pilot, and 207 the main survey. Extensive data cleaning and exploratory factor analysis were done to identify factors of learners' perceived value and satisfaction of OLS. Then, Value-Satisfaction grids and the Learners' Value Index of Satisfaction (LeVIS) were developed as benchmarking tools of OLS. Moreover, Multicriteria Decision Analysis (MCDA) techniques were employed to impute value from satisfaction scores in order to reduce survey response time. ^ The results provided four satisfaction and four value factors with high reliability (Cronbach's α). Moreover, value and satisfaction were found to have low linear and nonlinear correlations, indicating that they are two distinct uncorrelated constructs. This is consistent with the literature. Value-Satisfaction grids and the LeVIS index indicated relatively high effectiveness for technology and support characteristics, relatively low effectiveness for professor's characteristics, while course and learner characteristics indicated average effectiveness. ^ The main contributions of this study include identifying, defining, and articulating the relationship between value and satisfaction constructs as assessment of users' implied IS effectiveness, as well as assessing the accuracy of MCDA procedures to predict value scores, thus reducing by half the survey questionnaire size. ^

Signal processing of physiological signals for automated assessment of stress in computer users

The need to provide computers with the ability to distinguish the affective state of their users is a major requirement for the practical implementation of affective computing concepts. This dissertation proposes the application of signal processing methods on physiological signals to extract from them features that can be processed by learning pattern recognition systems to provide cues about a person's affective state. In particular, combining physiological information sensed from a user's left hand in a non-invasive way with the pupil diameter information from an eye-tracking system may provide a computer with an awareness of its user's affective responses in the course of human-computer interactions. In this study an integrated hardware-software setup was developed to achieve automatic assessment of the affective status of a computer user. A computer-based "Paced Stroop Test" was designed as a stimulus to elicit emotional stress in the subject during the experiment. Four signals: the Galvanic Skin Response (GSR), the Blood Volume Pulse (BVP), the Skin Temperature (ST) and the Pupil Diameter (PD), were monitored and analyzed to differentiate affective states in the user. Several signal processing techniques were applied on the collected signals to extract their most relevant features. These features were analyzed with learning classification systems, to accomplish the affective state identification. Three learning algorithms: Naïve Bayes, Decision Tree and Support Vector Machine were applied to this identification process and their levels of classification accuracy were compared. The results achieved indicate that the physiological signals monitored do, in fact, have a strong correlation with the changes in the emotional states of the experimental subjects. These results also revealed that the inclusion of pupil diameter information significantly improved the performance of the emotion recognition system. ^

Assessment of Higher-Education Hospitality Programs

The function of assessment in higher-education hospitality programs is to improve student learning. Although the assessment process is common in higher-education institutions, examples of assessment practices in hospitality programs have not been made available to academic practitioners. This paper describes a method successful at formulating assessment in a hospitality college professional program.

Moving toward change: Institutionalizing reform through implementation of the Learning Assistant model and Open Source Tutorials

Florida International University has undergone a reform in the introductory physics classes by focusing on the laboratory component of these classes. We present results from the secondary implementation of two research-based instructional strategies: the implementation of the Learning Assistant model as developed by the University of Colorado at Boulder and the Open Source Tutorial curriculum developed at the University of Maryland, College Park. We examine the results of the Force Concept Inventory (FCI) for introductory students over five years (n=872) and find that the mean raw gain of students in transformed lab sections was 0.243, while the mean raw gain of the traditional labs was 0.159, with a Cohen’s d effect size of 0.59. Average raw gains on the FCI were 0.243 for Hispanic students and 0.213 for women in the transformed labs, indicating that these reforms are not widening the gaps between underrepresented student groups and majority groups. Our results illustrate how research-based instructional strategies can be successfully implemented in a physics department with minimal department engagement and in a sustainable manner.

A relationship between the Florida Comprehensive Assessment Test 2.0 mathematics scores and racial and ethnic concentration when considering socio-economic status, ESOL student population, and school climate

From the moment children are born, they begin a lifetime journey of learning about themselves and their surroundings. With the establishment of the No Child Left Behind Act of 2001, it mandates that all children receive a high-quality education in a positive school climate. Regardless of the school the child attends or the neighborhood in which the child lives, proper and quality education and resources must be provided and made available in order for the child to be academically successful. The purpose of this ex post facto study was to investigate the relationship between the FCAT 2.0 mathematics scores of public middle school students in Miami-Dade County, Florida and the concentrations of a school's racial and ethnic make-up (Whites, Blacks, and Hispanics), English for Speakers of other Languages (ESOL) population, socio-economic status (SES), and school climate. The research question of this study was: Is there a significant relationship between the FCAT 2.0 Mathematics scores and racial and ethnic concentration of public middle school students in Miami-Dade County when controlling SES, ESOL student population, and school climate for the 2010-2011 school year? The instruments used to collect the data were the FCAT 2.0 and Miami-Dade County Public Schools (M-DCPS) School Climate Survey. The study found that Economically Disadvantaged (SES) students socio-economic status had the strongest correlation with the FCAT 2.0 mathematics scores (r = -.830). The next strongest correlation was with the number of students who agreed that their school climate was positive and helped them learn (r = .741) and the third strongest correlation was a school percentage of White students (r = .668). The study concluded that the FCAT 2.0 mathematics scores of M-DCPS middle school students have a significant relationship with socio-economic status, school climate, and racial concentration.

Undergraduate Students’ Perceptions of Study Abroad and Their Level of Achievement of Global Learning Outcomes

This study expanded on current research on study abroad and global learning, using the Global Perspective Inventory (GPI), and conducted at Florida International University (FIU) in Miami, FL. The GPI assesses the holistic development of a global perspective in higher education within three domains and their respective FIU-determined equivalents: cognitive (global awareness), intrapersonal (global perspective), and interpersonal (global engagement). The main purpose of this study was to assess FIU’s undergraduate students’ perceptions of study abroad on their level of achievement of global awareness, global perspective, and global engagement. The secondary purpose was to determine how the students described their study abroad experience and achievement of global learning. The research design for this study consisted of parallel mixed methods. The quantitative component was an ex post facto with hypothesis design, using a pretest/posttest nonequivalent group methodology. FIU undergraduates (N=147) who studied abroad for one semester or more completed the GPI pre- and post-tests. Descriptive statistics and paired t-tests were conducted to compare the means. The interviews included 10 students, and were analyzed through Structural coding, Saldaña’s In Vivo coding, and Value coding. Quantitative analyses indicated positive changes in the students’ global awareness and global perspective. These analyses also showed that the FIU students achieved higher post-test means on all the domains of the GPI compared to other studies. Qualitative analyses showed that the students’ experiences incorporated all three global learning outcomes, most notably global awareness and perspective.