Developing a process to create and validate an instrument assessing student attainment of competencies at an intercultural university in Mexico

This study took place at one of the intercultural universities (IUs) of Mexico that serve primarily indigenous students. The IUs are pioneers in higher education despite their numerous challenges (Bertely, 1998; Dietz, 2008; Pineda & Landorf, 2010; Schmelkes, 2009). To overcome educational inequalities among their students (Ahuja, Berumen, Casillas, Crispín, Delgado et al., 2004; Schmelkes, 2009), the IUs have embraced performance-based assessment (PBA; Casillas & Santini, 2006). PBA allows a shared model of power and control related to learning and evaluation (Anderson, 1998). While conducting a review on PBA strategies of the IUs, the researcher did not find a PBA instrument with valid and reliable estimates. The purpose of this study was to develop a process to create a PBA instrument, an analytic general rubric, with acceptable validity and reliability estimates to assess students' attainment of competencies in one of the IU's majors, Intercultural Development Management. The Human Capabilities Approach (HCA) was the theoretical framework and a sequential mixed method (Creswell, 2003; Teddlie & Tashakkori, 2009) was the research design. IU participants created a rubric during two focus groups, and seven Spanish-speaking professors in Mexico and the US piloted using students' research projects. The evidence that demonstrates the attainment of competencies at the IU is a complex set of actual, potential and/or desired performances or achievements, also conceptualized as "functional capabilities" (FCs; Walker, 2008), that can be used to develop a rubric. Results indicate that the rubric's validity and reliability estimates reached acceptable estimates of 80% agreement, surpassing minimum requirements (Newman, Newman, & Newman, 2011). Implications for practice involve the use of PBA within a formative assessment framework, and dynamic inclusion of constituencies. Recommendations for further research include introducing this study's instrument-development process to other IUs, conducting parallel mixed design studies exploring the intersection between HCA and assessment, and conducting a case study exploring assessment in intercultural settings. Education articulated through the HCA empowers students (Unterhalter & Brighouse, 2007; Walker, 2008). This study aimed to contribute to the quality of student learning assessment at the IUs by providing a participatory process to develop a PBA instrument.

Developing a Process to Create and Validate an Instrument Assessing Student Attainment of Competencies at an Intercultural University in Mexico

This study took place at one of the intercultural universities (IUs) of Mexico that serve primarily indigenous students. The IUs are pioneers in higher education despite their numerous challenges (Bertely, 1998; Dietz, 2008; Pineda & Landorf, 2010; Schmelkes, 2009). To overcome educational inequalities among their students (Ahuja, Berumen, Casillas, Crispín, Delgado et al., 2004; Schmelkes, 2009), the IUs have embraced performance-based assessment (PBA; Casillas & Santini, 2006). PBA allows a shared model of power and control related to learning and evaluation (Anderson, 1998). While conducting a review on PBA strategies of the IUs, the researcher did not find a PBA instrument with valid and reliable estimates. The purpose of this study was to develop a process to create a PBA instrument, an analytic general rubric, with acceptable validity and reliability estimates to assess students’ attainment of competencies in one of the IU’s majors, Intercultural Development Management. The Human Capabilities Approach (HCA) was the theoretical framework and a sequential mixed method (Creswell, 2003; Teddlie & Tashakkori, 2009) was the research design. IU participants created a rubric during two focus groups, and seven Spanish-speaking professors in Mexico and the US piloted using students’ research projects. The evidence that demonstrates the attainment of competencies at the IU is a complex set of actual, potential and/or desired performances or achievements, also conceptualized as “functional capabilities” (FCs; Walker, 2008), that can be used to develop a rubric. Results indicate that the rubric’s validity and reliability estimates reached acceptable estimates of 80% agreement, surpassing minimum requirements (Newman, Newman, & Newman, 2011). Implications for practice involve the use of PBA within a formative assessment framework, and dynamic inclusion of constituencies. Recommendations for further research include introducing this study’s instrument-development process to other IUs, conducting parallel mixed design studies exploring the intersection between HCA and assessment, and conducting a case study exploring assessment in intercultural settings. Education articulated through the HCA empowers students (Unterhalter & Brighouse, 2007; Walker, 2008). This study aimed to contribute to the quality of student learning assessment at the IUs by providing a participatory process to develop a PBA instrument.

Development and validation of an instrument to predict online student success using faculty perceptions

Distance learning is growing and transforming educational institutions. The increasing use of distance learning by higher education institutions and particularly community colleges coupled with the higher level of student attrition in online courses than in traditional classrooms suggests that increased attention should be paid to factors that affect online student course completion. The purpose of the study was to develop and validate an instrument to predict community college online student course completion based on faculty perceptions, yielding a prediction model of online course completion rates. Social Presence and Media Richness theories were used to develop a theoretically-driven measure of online course completion. This research study involved surveying 311 community college faculty who taught at least one online course in the past 2 years. Email addresses of participating faculty were provided by two south Florida community colleges. Each participant was contacted through email, and a link to an Internet survey was given. The survey response rate was 63% (192 out of 303 available questionnaires). Data were analyzed through factor analysis, alpha reliability, and multiple regression. The exploratory factor analysis using principal component analysis with varimax rotation yielded a four-factor solution that accounted for 48.8% of the variance. Consistent with Social Presence theory, the factors with their percent of variance in parentheses were: immediacy (21.2%), technological immediacy (11.0%), online communication and interactivity (10.3%), and intimacy (6.3%). Internal consistency of the four factors was calculated using Cronbach's alpha (1951) with reliability coefficients ranging between .680 and .828. Multiple regression analysis yielded a model that significantly predicted 11% of the variance of the dependent variable, the percentage of student who completed the online course. As indicated in the literature (Johnson & Keil, 2002; Newberry, 2002), Media Richness theory appears to be closely related to Social Presence theory. However, elements from this theory did not emerge in the factor analysis.

Development of a new student evaluation instrument of instructor effectiveness in online courses

The purpose of this study was to (a) develop an evaluation instrument capable of rating students' perceptions of the instructional quality of an online course and the instructor’s performance, and (b) validate the proposed instrument with a study conducted at a major public university. The instrument was based upon the Seven Principles of Good Practice for Undergraduate Education (Chickering & Gamson, 1987). The study examined four specific questions. 1. Is the underlying factor structure of the new instrument consistent with Chickering and Gamson's Seven Principles? 2. Is the factor structure of the new instrument invariant for male and female students? 3. Are the scores on the new instrument related students’ expected grades? 4. Are the scores on the new instrument related to the students' perceived course workload? ^ The instrument was designed to measure students’ levels of satisfaction with their instruction, and also gathered information concerning the students’ sex, the expected grade in the course, and the students’ perceptions of the amount of work required by the course. A cluster sample consisting of an array of online courses across the disciplines yielded a total 297 students who responded to the online survey. The students for each course selected were asked to rate their instructors with the newly developed instrument. ^ Question 1 was answered using exploratory factor analysis, and yielded a factor structure similar to the Seven Principles.^ Question 2 was answered by separately factor-analyzing the responses of male and female students and comparing the factor structures. The resulting factor structures for men and women were different. However, 14 items could be realigned under five factors that paralleled some of the Seven Principles. When the scores of only those 14 items were entered in two principal components factor analyses using only men and only women, respectively and restricting the factor structure to five factors, the factor structures were the same for men and women.^ A weak positive relationship between students’ expected grades and their scores on the instrument was found (Question 3). There was no relationship between students’ perceived workloads for the course and their scores on the instrument (Question 4).^

Development of an instrument to measure high school students' global awareness and attitudes: Looking through the lens of social sciences

The purpose of this study was to develop an instrument to measure high school students’ perspectives on global awareness and attitudes toward social issues. The research questions that guided this study were: (a) Can acceptable validity and reliability estimates be established for an instrument developed to measure high schools students' global awareness? (b) Can acceptable validity and reliability estimates be established for an instrument developed to measure high schools students' attitudes towards global social issues? (c) What is the relationship between high school students’ GPA, race/ethnicity, gender, socio-economic status, parents’ education, getting the news, reading and listening habits, the number of classes taken in the social sciences, whether they speak a second language, and have experienced living in or visiting other countries, and their perception of global awareness and attitudes toward global social issues. ^ An ex post facto research design was used and the data were collected using a 4-part Likert-type survey. It was administered to 14 schools in the Miami-Dade County, Florida area to 704 students. A factor analysis with an orthogonal varimax rotation was vii used to select the factors that best represented the three constructs – global education, global citizenship, and global workforce. This was done to establish construct validity. Cronbach’s alpha was used to determine the reliability of the instrument. Descriptive statistics and a hierarchical multiple regression were used for the demographics to establish their relationship, if any, to the findings. ^ Key findings of the study were that reliable and valid estimates can be developed for the instrument. The multiple regression analysis for model 1 and 2 accounted for a variance of 3% and 5% for self-perceptions of global awareness (factor 1). The regression model also accounted for a 5% and 13% variance in the two models for attitudes toward global social issues (factor 2). The demographics that were statistically significant were: ethnicity, gender, SES, parents’ education, listening to music, getting the news, speaking a second language, GPA, classes taken in the social sciences, and visiting other countries. An important finding for the study was those attending public schools (as opposed to private schools) had more positive attitudes towards global social issues (factor 2) The statistics indicated that these students had taken history, economics, and social studies – a curriculum infused with global perspectives.^

Development of Knowledge Domains and an Instrument to Assess Probation Officers’ Knowledge of Offenders with Intellectual Disabilities

The purpose of this study was to develop knowledge domains and an instrument to assess probation officers’ knowledge levels of offenders with intellectual disabilities by utilizing a synthesis of subject matter analysis technique and a comprehensive review of literature. Results can be used to develop effective training for probation officers.

Development and implementation of a tool to assess the micro/macro elements in an organization

This study examines the congruency of planning between organizational structure and process, through an evaluation and planning model known as the Micro/Macro Dynamic Planning Grid. The model compares day-to-day planning within an organization to planning imposed by organizational administration and accrediting agencies. A survey instrument was developed to assess the micro and macro sociological analysis elements utilized by an organization.^ The Micro/Macro Dynamic Planning Grid consists of four quadrants. Each quadrant contains characteristics that reflect the interaction between the micro and macro elements of planning, objectives and goals within an organization. The Over Macro/Over Micro, Quadrant 1, contains attributes that reflect a tremendous amount of action and ongoing adjustments, typical of an organization undergoing significant changes in either leadership, program and/or structure. Over Macro/Under Micro, Quadrant 2, reflects planning characteristics found in large, bureaucratic systems with little regard given to the workings of their component parts. Under Macro/Under Micro, Quadrant 3, reflects the uncooperative, uncoordinated organization, one that contains a multiplicity of viewpoints, language, objectives and goals. Under Macro/Under Micro, Quadrant 4 represents the worst case scenario for any organization. The attributes of this quadrant are very reactive, chaotic, non-productive and redundant.^ There were three phases to the study: development of the initial instrument, pilot testing the initial instrument and item revision, and administration and assessment of the refined instrument. The survey instrument was found to be valid and reliable for the purposes and audiences herein described.^ In order to expand the applicability of the instrument to other organizational settings, the survey was administered to three professional colleges within a university.^ The first three specific research questions collectively answered, in the affirmative, the basic research question: Can the Micro/Macro Dynamic Planning Grid be applied to an organization through an organizational development tool? The first specific question: Can an instrument be constructed that applies the Micro/Macro Dynamic Planning Grid? The second specific research question: Is the constructed instrument valid and reliable? The third specific research question: Does an instrument that applies the Micro/Macro Dynamic Planning Grid assess congruency of micro and macro planning, goals and objectives within an organization? The fourth specific research question: What are the differences in the responses based on roles and responsibilities within an organization? involved statistical analysis of the response data and comparisons obtained with the demographic data. (Abstract shortened by UMI.) ^

Development of ``HealthServe'': A measure of customer service orientation in healthcare

An assessment tool designed to measure a customer service orientation among RN's and LPN's was developed using a content-oriented approach. Critical incidents were first developed by asking two samples of healthcare managers (n = 52 and 25) to identify various customer-contact situations. The critical incidents were then used to formulate a 121-item instrument. Patient-contact workers from 3 hospitals (n = 102) completed the instrument along with the NEO-FFI, a measure of the Big Five personality factors. Concurrently, managers completed a performance evaluation scale on the employees participating in the study in order to determine the predictive validity of the instrument.^ Through a criterion-keying approach, the instrument was scaled down to 38 items. The correlation between HealthServe and the supervisory ratings of performance evaluation data supported the instrument's criterion-related validity (r =.66, p $<$.0001). Incremental validity of HealthServe over the Big Five was found with HealthServe accounting for 46% of the variance.^ The NEO-FFI was used to assess the correlation between personality traits and HealthServe. A factor analysis of HealthServe suggested 4 factors which were correlated with the NEO-FFI scores. Results indicated that HealthServe was related to Extraversion, Openness to Experience, Agreeableness, Conscientiousness and negatively related to Neuroticism.^ The benefits of the test construction procedure used here over the use of broad-based measures of personality were discussed as well as the limitations of using a concurrent validation strategy. Recommendations for future studies were provided. ^

An assessment of computer usage within the Bahamian public school system and recommended guidelines towards the development of a strategic national computer plan

Computers have dramatically changed the way we live, conduct business, and deliver education. They have infiltrated the Bahamian public school system to the extent that many educators now feel the need for a national plan. The development of such a plan is a challenging undertaking, especially in developing countries where physical, financial, and human resources are scarce. This study assessed the situation with regard to computers within the Bahamian public school system, and provided recommended guidelines to the Bahamian government based on the results of a survey, the body of knowledge about trends in computer usage in schools, and the country's needs. ^ This was a descriptive study for which an extensive review of literature in areas of computer hardware, software, teacher training, research, curriculum, support services and local context variables was undertaken. One objective of the study was to establish what should or could be relative to the state-of-the-art in educational computing. A survey was conducted involving 201 teachers and 51 school administrators from 60 randomly selected Bahamian public schools. A random stratified cluster sampling technique was used. ^ This study used both quantitative and qualitative research methodologies. Quantitative methods were used to summarize the data about numbers and types of computers, categories of software available, peripheral equipment, and related topics through the use of forced-choice questions in a survey instrument. Results of these were displayed in tables and charts. Qualitative methods, data synthesis and content analysis, were used to analyze the non-numeric data obtained from open-ended questions on teachers' and school administrators' questionnaires, such as those regarding teachers' perceptions and attitudes about computers and their use in classrooms. Also, interpretative methodologies were used to analyze the qualitative results of several interviews conducted with senior public school system's officials. Content analysis was used to gather data from the literature on topics pertaining to the study. ^ Based on the literature review and the data gathered for this study a number of recommendations are presented. These recommendations may be used by the government of the Commonwealth of The Bahamas to establish policies with regard to the use of computers within the public school system. ^

Development of a personality profile of a firefighter

This study investigated the differences in personality, consistent with the vocational theory of personality as proposed by Holland (1997), for a modern day firefighter. This study also investigates the relationships between personality characteristics and job duties performed by firefighters and firefighter-paramedics. Archival data from employees (N = 98) of a Southeastern Florida fire department who completed the Hogan Personality Inventory (HPI), Hogan Development Survey (HDS) and Motives, Values, Preferences Inventory (MVPI), as well as a self-report questionnaire on variety proneness (boredom), job satisfaction, and affective well-being data were analyzed. The scores of the firefighters on the HPI, HDS, and MVPI were used as predictive data, and criterion data used in this study were self-report satisfaction data on job involvement, variety proneness (boredom), and affective well-being. In addition, criterion data on performance were obtained from the employment histories of the participants, and were correlated with the personality scale scores to determine if personality is predictive of aspects of performance. ^ Participants in this study varied with respect to the type of firefighter duties required from them on their jobs. The participants were categorized into three duty classifications: Group 1 (G1) are the firefighters hired before 1990 and are only certified as firefighters; Group 2 (G2) are the firefighters hired before 1990 who became paramedics at some point after employment and after fire college training; and Group 3 (G3) are the firefighters hired after 1990 who were trained as paramedics in the fire college and who were aware of the paramedic requirement at time of application or were already trained as paramedics at the time of application. From the research reviewed and presented in this paper, hypotheses were generated about differences between the personality types of firefighter groups G1 and G2 versus G3, in accordance with Holland's theories. In addition, it was hypothesized that personality will predict outcomes of satisfaction and performance. ^ Results found that job satisfaction was not found to be statistically different among the groups. However, the groups differed significantly on 5 of the predictive instrument scales, and personality was found to be a predictor of limited performance data. ^

Human resource development professionals' perceptions about the Academy of Human Resource Development Standards on Ethics and Integrity

Professional standards of ethics proclaim the core values of a profession, describe expected professional duties and responsibilities, and provide a framework for ethical practice and ethical decision-making. The purpose of this mixed, quantitative and qualitative, survey study was to examine HRD professionals' perceptions about the AHRD Standards on Ethics and Integrity, how HRD professionals used the Standards for research and decision-making, and the extent to which the Standards provided guidance for ethical decision-making. Through an on-line survey instrument, 182 members of AHRD were surveyed. The open-ended questions were analyzed using thematic analysis to expand on, inform, and support the quantitative findings. The close-ended questions were analyzed with frequency distributions, descriptive statistics, cross tabulations, and Spearman rank correlations. The results showed a significant relationship between (a) years of AHRD membership and level of familiarity with the Standards, (b) years of AHRD membership and use of the Standards for research, and (c) level of familiarity with the Standards and use of the Standards for research. There were no significant differences among scholars, scholar practitioners, practitioners, and students regarding their perceptions about the Standards. The results showed that the Standards were not well known or widely used. Nevertheless, the results indicated overall positive perceptions about the Standards. Seventy percent agreed that the Standards provided an appropriate set of ethical principles and reflected respondents' own standards of conduct. Seventy-eight percent believed that the Standards were important for defining HRD as a profession and 54% believed they were important for developing a sense of belonging to the HRD profession. Fifty-one percent believed the Standards should be enforceable and 61% agreed members should sign the membership application form showing willingness to adhere to the Standards. Seventy-seven percent based work-related ethical decisions on personal beliefs of right and wrong and 56% on established professional values and rules of right and wrong. The findings imply that if the professional standards of ethics are to influence the profession, they should be widely publicized and discussed among members, they should have some binding power, and their use should be encouraged.

Development and testing of the Elderly Social Vulnerability Index (ESVI): A composite indicator to measure social vulnerability in the Jamaican elderly population

Over the last two decades social vulnerability has emerged as a major area of study, with increasing attention to the study of vulnerable populations. Generally, the elderly are among the most vulnerable members of any society, and widespread population aging has led to greater focus on elderly vulnerability. However, the absence of a valid and practical measure constrains the ability of policy-makers to address this issue in a comprehensive way. This study developed a composite indicator, The Elderly Social Vulnerability Index (ESVI), and used it to undertake a comparative analysis of the availability of support for elderly Jamaicans based on their access to human, material and social resources. The results of the ESVI indicated that while the elderly are more vulnerable overall, certain segments of the population appear to be at greater risk. Females had consistently lower scores than males, and the oldest-old had the highest scores of all groups of older persons. Vulnerability scores also varied according to place of residence, with more rural parishes having higher scores than their urban counterparts. These findings support the political economy framework which locates disadvantage in old age within political and ideological structures. The findings also point to the pervasiveness and persistence of gender inequality as argued by feminist theories of aging. Based on the results of the study it is clear that there is a need for policies that target specific population segments, in addition to universal policies that could make the experience of old age less challenging for the majority of older persons. Overall, the ESVI has displayed usefulness as a tool for theoretical analysis and demonstrated its potential as a policy instrument to assist decision-makers in determining where to target their efforts as they seek to address the issue of social vulnerability in old age. Data for this study came from the 2001 population and housing census of Jamaica, with multiple imputation for missing data. The index was derived from the linear aggregation of three equally weighted domains, comprised of eleven unweighted indicators which were normalized using z-scores. Indicators were selected based on theoretical relevance and data availability.

The development of optimized high performance liquid chromatography systems for the analysis of improvised explosives

Existing instrumental techniques must be adaptable to the analysis of novel explosives if science is to keep up with the practices of terrorists and criminals. The focus of this work has been the development of analytical techniques for the analysis of two types of novel explosives: ascorbic acid-based propellants, and improvised mixtures of concentrated hydrogen peroxide/fuel. In recent years, the use of these explosives in improvised explosive devices (IEDs) has increased. It is therefore important to develop methods which permit the identification of the nature of the original explosive from post-blast residues. Ascorbic acid-based propellants are low explosives which employ an ascorbic acid fuel source with a nitrate/perchlorate oxidizer. A method which utilized ion chromatography with indirect photometric detection was optimized for the analysis of intact propellants. Post-burn and post-blast residues if these propellants were analyzed. It was determined that the ascorbic acid fuel and nitrate oxidizer could be detected in intact propellants, as well as in the post-burn and post-blast residues. Degradation products of the nitrate and perchlorate oxidizers were also detected. With a quadrupole time-of-flight mass spectrometer (QToFMS), exact mass measurements are possible. When an HPLC instrument is coupled to a QToFMS, the combination of retention time with accurate mass measurements, mass spectral fragmentation information, and isotopic abundance patterns allows for the unequivocal identification of a target analyte. An optimized HPLC-ESI-QToFMS method was applied to the analysis of ascorbic acid-based propellants. Exact mass measurements were collected for the fuel and oxidizer anions, and their degradation products. Ascorbic acid was detected in the intact samples and half of the propellants subjected to open burning; the intact fuel molecule was not detected in any of the post-blast residue. Two methods were optimized for the analysis of trace levels of hydrogen peroxide: HPLC with fluorescence detection (HPLC-FD), and HPLC with electrochemical detection (HPLC-ED). Both techniques were extremely selective for hydrogen peroxide. Both methods were applied to the analysis of post-blast debris from improvised mixtures of concentrated hydrogen peroxide/fuel; hydrogen peroxide was detected on variety of substrates. Hydrogen peroxide was detected in the post-blast residues of the improvised explosives TATP and HMTD.

Development of a dynamic headspace concentration technique for the non-contact sampling of human odor samples and the creation of canine training aids

Human scent and human remains detection canines are used to locate living or deceased humans under many circumstances. Human scent canines locate individual humans on the basis of their unique scent profile, while human remains detection canines locate the general scent of decomposing human remains. Scent evidence is often collected by law enforcement agencies using a Scent Transfer Unit, a dynamic headspace concentration device. The goals of this research were to evaluate the STU-100 for the collection of human scent samples, and to apply this method to the collection of living and deceased human samples, and to the creation of canine training aids. The airflow rate and collection material used with the STU-100 were evaluated using a novel scent delivery method. Controlled Odor Mimic Permeation Systems were created containing representative standard compounds delivered at known rates, improving the reproducibility of optimization experiments. Flow rates and collection materials were compared. Higher air flow rates usually yielded significantly less total volatile compounds due to compound breakthrough through the collection material. Collection from polymer and cellulose-based materials demonstrated that the molecular backbone of the material is a factor in the trapping and releasing of compounds. The weave of the material also affects compound collection, as those materials with a tighter weave demonstrated enhanced collection efficiencies. Using the optimized method, volatiles were efficiently collected from living and deceased humans. Replicates of the living human samples showed good reproducibility; however, the odor profiles from individuals were not always distinguishable from one another. Analysis of the human remains samples revealed similarity in the type and ratio of compounds. Two types of prototype training aids were developed utilizing combinations of pure compounds as well as volatiles from actual human samples concentrated onto sorbents, which were subsequently used in field tests. The pseudo scent aids had moderate success in field tests, and the Odor pad aids had significant success. This research demonstrates that the STU-100 is a valuable tool for dog handlers and as a field instrument; however, modifications are warranted in order to improve its performance as a method for instrumental detection.

Bondage and Discipline, Dominance and Submission, and Sadism and Masochism (BDSM) Identity Development

Identity shapes how people make sense of the world. Sexual minorities’ sexual orientations and gender identities fall outside of heteronormative categorizations. Adults engage in diverse relationships: many of them fall outside of heteronormative boundaries. As an instrument of social justice, Adult Education can be a site for BDSM identity development.