The impact of bandwidth in structured interviewing and criterion stability in predicting and assessing manager performance across transition and maintenance stages

The current study examined the role of three important components in the use of structured employment interviewing in performance prediction: construct bandwidth, observed communication skill, and the stability/dynamicity of performance criteria over time. A matched sample of 242 hospitality managers was derived from a field data set provided by a large hospitality management organization. Interview data and two years of performance appraisal data were provided. Bandwidth analysis demonstrated only minimal differences in prediction between matched predictor-criterion pairs compared with predictor to overall aggregate ratings (unmatched). The communication skill analysis revealed that this interviewer rated observation significantly predicted a number of the individual performance dimensions as well as overall performance over time. Of the five interview items, the strongest overall predictor of performance was interviewer rated communication skill. The stability/dynamicity analyses demonstrated the performance criteria to be generally stable over the two year period examined, which provides support for the long held notion that performance criteria is stabile over time. However, there were two exceptions. The interview dimension customer service orientation had shifting relationships over time with four of the criteria over the two year period. The performance criteria employee development also demonstrated some instability in its relationships with predictors. Thus, some evidence of dynamicity in performance criteria was revealed. Interestingly, both of the most noteworthy findings in the study involved items that were rated differently than the others in the study. The rated interview item communication skill and the rated performance criteria client satisfaction were ratings that involved a more direct level of observation. Additional analyses also revealed evidence of a general factor of performance. These two themes are more fully covered in the discussion.

The relationship of aptitude, course grades, and program length, to performance on a standards based test (NCLEX -RN)

The purpose of this study was threefold: first, to investigate variables associated with learning, and performance as measured by the National Council Licensure Examination for Registered Nurses (NCLEX-RN). The second purpose was to validate the predictive value of the Assessment Technologies Institute (ATI) achievement exit exam, and lastly, to provide a model that could be used to predict performance on the NCLEX-RN, with implications for admission and curriculum development. The study was based on school learning theory, which implies that acquisition in school learning is a function of aptitude (pre-admission measures), opportunity to learn, and quality of instruction (program measures). Data utilized were from 298 graduates of an associate degree nursing program in the Southeastern United States. Of the 298 graduates, 142 were Hispanic, 87 were Black, non-Hispanic, 54 White, non-Hispanic, and 15 reported as Others. The graduates took the NCLEX-RN for the first time during the years 2003–2005. This study was a predictive, correlational design that relied upon retrospective data. Point biserial correlations, and chi-square analyses were used to investigate relationships between 19 selected predictor variables and the dichotomous criterion variable, NCLEX-RN. The correlation and chi square findings indicated that men did better on the NCLEX-RN than women; Blacks had the highest failure rates, followed by Hispanics; older students were more likely to pass the exam than younger students; and students who passed the exam started and completed the nursing program with a higher grade point average, than those who failed the exam. Using logistic regression, five statistical models that used variables associated with learning and student performance on the NCLEX-RN were tested with a model adapted from Bloom's (1976) and Carroll's (1963) school learning theories. The derived model included: NCLEX-RNsuccess = f (Nurse Entrance Test and advanced medical-surgical nursing course grade achieved). The model demonstrates that student performance on the NCLEX-RN can be predicted by one pre-admission measure, and a program measure. The Assessment Technologies Institute achievement exit exam (an outcome measure) had no predictive value for student performance on the NCLEX-RN. The model developed accurately predicted 94% of the student's successful performance on the NCLEX-RN.

Development of ``HealthServe'': A measure of customer service orientation in healthcare

An assessment tool designed to measure a customer service orientation among RN's and LPN's was developed using a content-oriented approach. Critical incidents were first developed by asking two samples of healthcare managers (n = 52 and 25) to identify various customer-contact situations. The critical incidents were then used to formulate a 121-item instrument. Patient-contact workers from 3 hospitals (n = 102) completed the instrument along with the NEO-FFI, a measure of the Big Five personality factors. Concurrently, managers completed a performance evaluation scale on the employees participating in the study in order to determine the predictive validity of the instrument.^ Through a criterion-keying approach, the instrument was scaled down to 38 items. The correlation between HealthServe and the supervisory ratings of performance evaluation data supported the instrument's criterion-related validity (r =.66, p $<$.0001). Incremental validity of HealthServe over the Big Five was found with HealthServe accounting for 46% of the variance.^ The NEO-FFI was used to assess the correlation between personality traits and HealthServe. A factor analysis of HealthServe suggested 4 factors which were correlated with the NEO-FFI scores. Results indicated that HealthServe was related to Extraversion, Openness to Experience, Agreeableness, Conscientiousness and negatively related to Neuroticism.^ The benefits of the test construction procedure used here over the use of broad-based measures of personality were discussed as well as the limitations of using a concurrent validation strategy. Recommendations for future studies were provided. ^

The predictive power of homework assignments on student achievement in grade three

Homework has been a controversial issue in education for the past century. Research has been scarce and has yielded results at both ends of the spectrum. This study examined the relationship between homework performance (percent of homework completed and percent of homework correct), student characteristics (SAT-9 score, gender, ethnicity, and socio-economic status), perceptions, and challenges and academic achievement determined by the students' average score on weekly tests and their score on the FCAT NRT mathematics assessment. ^ The subjects for this study consisted of 143 students enrolled in Grade 3 at a suburban elementary school in Miami, Florida. Pearson's correlations were used to examine the associations of the predictor variables with average test scores and FCAT NRT scores. Additionally, simultaneous regression analyses were carried out to examine the influence of the predictor variables on each of the criterion variables. Hierarchical regression analyses were performed on the criterion variables from the predictor variables. ^ Homework performance was significantly correlated with average test score. Controlling for the other variables homework performance was highly related to average test score and FCAT NRT score. ^ This study lends support to the view that homework completion is highly related to student academic achievement at the lower elementary level. It is suggested that at the elementary level more consideration be given to the amount of homework completed by students and to utilize the information in formulating intervention strategies for student who may not be achieving at the appropriate levels. ^

Algebra I and other predictors of high school dropout

This study investigated the relation of several predictors to high school dropout. The data, composed of records from a cohort of students ( N = 10,100) who entered ninth grade in 2001, were analyzed via logistic regression. The predictor variables were: (a) Algebra I grade, (b) Florida Comprehensive Assessment Test (FCAT) level, (c) language proficiency, (d) gender, (e) race/ethnicity, (f) Exceptional Student Education program membership, and (g) socio-economic status. The criterion was graduation status: graduated or dropped out. Algebra I grades were an important predictor of whether students drop out or graduate; students who failed this course were 4.1 times more likely to drop out than those who passed the course. Other significant predictors of high school dropout were language proficiency, Florida Comprehensive Assessment Test (FCAT) level, gender, and socio-economic status. The main focus of the study was on Algebra I as a predictor, but the study was not designed to discover the specific factors related to or underlying success in this course. Nevertheless, because Algebra I may be considered an important prerequisite for other major facets of the curriculum and because of its high relationship to high school dropout, a recommendation emerging from these findings is that districts address the issue of preventing failure in this course. Adequate support mechanisms for improving retention include addressing the students' readiness for enrolling in mathematics courses as well as curriculum improvements that enhance student readiness through such processes as remediation. Assuring that mathematics instruction is monitored and improved and that remedial programs are in place to facilitate content learning in all subjects for all students, but especially for those having limited English proficiency, are critical educational responsibilities.

Development of safety performance functions for empirical Bayes estimation of crash reduction factors

Crash reduction factors (CRFs) are used to estimate the potential number of traffic crashes expected to be prevented from investment in safety improvement projects. The method used to develop CRFs in Florida has been based on the commonly used before-and-after approach. This approach suffers from a widely recognized problem known as regression-to-the-mean (RTM). The Empirical Bayes (EB) method has been introduced as a means to addressing the RTM problem. This method requires the information from both the treatment and reference sites in order to predict the expected number of crashes had the safety improvement projects at the treatment sites not been implemented. The information from the reference sites is estimated from a safety performance function (SPF), which is a mathematical relationship that links crashes to traffic exposure. The objective of this dissertation was to develop the SPFs for different functional classes of the Florida State Highway System. Crash data from years 2001 through 2003 along with traffic and geometric data were used in the SPF model development. SPFs for both rural and urban roadway categories were developed. The modeling data used were based on one-mile segments that contain homogeneous traffic and geometric conditions within each segment. Segments involving intersections were excluded. The scatter plots of data show that the relationships between crashes and traffic exposure are nonlinear, that crashes increase with traffic exposure in an increasing rate. Four regression models, namely, Poisson (PRM), Negative Binomial (NBRM), zero-inflated Poisson (ZIP), and zero-inflated Negative Binomial (ZINB), were fitted to the one-mile segment records for individual roadway categories. The best model was selected for each category based on a combination of the Likelihood Ratio test, the Vuong statistical test, and the Akaike's Information Criterion (AIC). The NBRM model was found to be appropriate for only one category and the ZINB model was found to be more appropriate for six other categories. The overall results show that the Negative Binomial distribution model generally provides a better fit for the data than the Poisson distribution model. In addition, the ZINB model was found to give the best fit when the count data exhibit excess zeros and over-dispersion for most of the roadway categories. While model validation shows that most data points fall within the 95% prediction intervals of the models developed, the Pearson goodness-of-fit measure does not show statistical significance. This is expected as traffic volume is only one of the many factors contributing to the overall crash experience, and that the SPFs are to be applied in conjunction with Accident Modification Factors (AMFs) to further account for the safety impacts of major geometric features before arriving at the final crash prediction. However, with improved traffic and crash data quality, the crash prediction power of SPF models may be further improved.

Examining a proposed job retention model for adult workers with mental retardation in supported employment

This quantitative study investigated the predictive relationships and interaction between factors such as work-related social behaviors (WRSB), self-determination (SD), person-job congruency (PJC), job performance (JP), job satisfaction (JS), and job retention (JR). A convenience sample of 100 working adults with MR were selected from supported employment agencies. Data were collected using a survey test battery of standardized instruments. The hypotheses were analyzed using three multiple regression analyses to identify significant relationships. Beta weights and hierarchical regression analysis determined the percentage of the predictor variables contribution to the total variance of the criterion variables, JR, JP, and JS. ^ The findings highlight the importance of self-determination skills in predicting job retention, satisfaction, and performance for employees with MR. Consistent with the literature and hypothesized model, there was a predictive relationship between SD, JS and JR. Furthermore, SD and PJC were predictors of JP. SD and JR were predictors of JS. Interestingly, the results indicated no significant relationship between JR and JP, or between JP and JS, or between PJC and JS. This suggests that there is a limited fit between the hypothesized model and the study's findings. However, the theoretical contribution made by this study is that self-determination is a particularly relevant predictor of important work outcomes including JR, JP, and JS. This finding is consistent with Deci's (1992) Self-Determination Theory and Wehmeyer's (1996) argument that SD skills in individuals with disabilities have important consequences for the success in transitioning from school to adult and work life. This study provides job retention strategies that offer rehabilitation and HR professionals a useful structure for understanding and implementing job retention interventions for people with MR. ^ The study concluded that workers with mental retardation who had more self-determination skills were employed longer, more satisfied, and better performers on the job. Also, individuals whose jobs were matched to their interests and abilities (person-job congruency) were better at self-determination skills. ^

The relationship between prekindergarten social and emotional development and academic success among Hispanic children from low-income families

Social and emotional development has been considered an important factor in child development which has been placed at the end of the learning spectrum due to high stakes testing. Social and emotional development consists of the relationships an individual has with others, the level of self-control, and the motivation and perseverance a person has during an activity (Bandura, 1989). This study examined the relationship between Hispanic children’s prekindergarten social and emotional development and their academic success. Hispanic children from a large southeastern city whose parents were receiving subsidized child-care were followed from their prekindergarten year through third grade (N=1,978). Several hierarchical regressions were run to determine the relationship between children’s social and emotional development, during their prekindergarten year using the DECA (Devereaux Early Childhood Assessment), and the their academic success, as measured by kindergarten through third grade end of the year reading and mathematics academic grades, second grade SAT (Stanford Achievement Test) scores, and third grade FCAT (Florida Comprehensive Assessment Test) and NRT (Norm Referenced Test) scores. Hierarchical regressions were conducted for each grade and subject in order to control for demographics and prior achievement. The results of this study revealed that for Hispanic children from low-income families, the best predictor for academic success was the children’s prior academic achievement. Social and emotional development showed no significant predictive value for the third grade criterion variables as well as end of the year academic grades in second grade and kindergarten reading. Evidence did suggest that for first grade end of the year academic grades and kindergarten math, social and emotional development had a small predictive value. Further research must be conducted as to why social and emotional development, after controlling for demographics and previous academic achievement, bears such a small predictive value when it is clear that many professionals feel it is the most important factor for school readiness.

The effects of school mobility in pre -assessment grade levels on student performance and school accountability in urban Title I schools

This study examined the effects of student mobility and educational enrollment experiences on academic achievement. The educational progress, school enrollments and transfers of inner-city elementary students were tracked over a four-year period. Student achievement was measured by criterion-referenced reading tests administered in the second semester of the third grade. It further analyzed the degree to which the switch to different basal reading textbooks interrupted the continuity of education thereby contributing to the detrimental effects of intra-district mobility. ^ Mobility histories of 2,913 third grade students were collected to evaluate the number of times each student entered or withdrew from a Miami-Dade County Public School beginning in August 2000 through March 2004, and distinguished between transfers that occurred during the academic school year and those that occurred during summer months. Data were analyzed using Pearson correlations and multiple regressions to determine if school mobility contributed to performance on the Florida Comprehensive Assessment Third Grade Reading Test (FCAT). Transferring from one school to another was found to have a significant negative impact on student test scores. Transfers within the academic school year were more detrimental than transfers that occurred during the summer months. Third grade students who transferred into schools that used the same reading textbook series were found to have significantly higher FCAT reading scores than third graders who transferred into schools that used different reading textbooks. ^ The effects of mobility rates on overall school performance were also examined. Data was collected on 124 Title I elementary schools to determine the degree to which mobility affected school accountability scores. Title I schools with high student mobility rates had significantly lower accountability scores than schools with lower student mobility rates. ^ The results of this study highlight the impact of education and housing policy and imply a need for programs and practices that promote stability in the early elementary years. ^

The relationship of managers' self -reported leadership practices and their spirituality

The role of spirituality in leadership in business and other organizations has gained growing recognition. The purpose of this study was to explore the relationship between spirituality and nine selected transformational leadership practices. Community leaders (N = 138) in business, education, and other professions who were graduates of a 10-week leadership program, Leadership Fort Lauderdale, from 1994 to 2004 completed the Spirituality Assessment Scale (SAS), the Leadership Practices Inventory (LPI), and four transformational leadership items of the Multifactor Leadership Questionnaire (MLQ). ^ The predictor variables were participants' scores on the LPI and MLQ. The criterion variable was their score on the SAS. Stepwise multiple regression analysis was used to test the hypothesis: Is there a combination of nine selected transformational leadership practices that would account for a significant portion of the variance of each of two spirituality measures? The Definitive and Correlated dimensions and Total spirituality score of the SAS were used in the analysis. ^ Results showed that two of the LPI leadership practices were significantly related to spirituality. The variable Inspiring a Shared Vision accounted for 10% of the variance of the SAS Definitive dimension. The variable Encouraging the Heart accounted for 30% of the variance of the Correlated dimension. For the Total spirituality score, two models were revealed. In the first model, Encouraging the Heart accounted for 28% of the variance of the total spirituality score. In the second model, Encouraging the Heart and Inspiring a Shared Vision together accounted for 31% of the total spirituality score. None of the transformational leadership practices from the MLQ were significantly related to spirituality. ^ The data partially support the hypothesis: two of the nine leadership variables did in combination correlate with leaders' spirituality. The results also support at least a partial relationship between spirituality and certain transformational leadership practices among leaders in various spheres, such as education, business, and other professions. ^

Effects of sex, third grade reading achievement and motivation as predictors of fourth grade reading achievement of Hispanic students: A path analysis

This study explored the topic of motivation for intermediate students combining both an objective criterion measure (i.e., standardized test scores) and the self-report of students on self-concept and value of reading. The purpose of this study was to examine how third grade reading achievement correlated with the motivation of fourth grade boys and girls, and, in turn, how motivation related to fourth grade reading achievement. The participants were fourth grade students (n=207) attending two public, elementary schools in Miami-Dade County who were of primarily Hispanic origin or descent. Data were collected using the Reading Survey portion of the Motivation to Read Profile (1996) which measures self-concept and value of reading in order to measure motivation and the Third and Fourth Grade Reading Florida Comprehensive Assessment Tests 2.0 (FCAT 2.0) to assess achievement. First, a one way Analysis of Variance (ANOVA) was conducted to determine whether motivation differed significantly between fourth grade boys and girls. Second, a path analysis was used to determine whether motivation mediated or moderated the association between FCAT 2.0 third and fourth grade scores. Results of the ANOVA indicated that motivation, as measured by the Motivation to Read Profile did not differ significantly by sex. Results from the path analysis indicated that the model was significant and that third grade FCAT 2.0 scores accounted for a significant amount of the variance in fourth grade FCAT 2.0 scores once motivation was entered. Results of the study demonstrated that motivation partially mediates, but does not moderate the relationship between FCAT 2.0 third and fourth grade scores. In conclusion, it can be determined that past student achievement for fourth grade students plays a role in current student achievement when motivation is also considered. It is therefore important in order to improve the quality of fourth grade student's current performance to take into account a student's motivation and past achievement. An effort must be made to address students' motivational needs whether through school wide programs or at the classroom level in addition or in conjunction with cognition. Future research on the effect of self-concept in reading achievement is recommended.

Work Creativity as a Dimension of Job Performance

To stay competitive, many employers are looking for creative and innovative employees to add value to their organization. However, current models of job performance overlook creative performance as an important criterion to measure in the workplace. The purpose of this dissertation is to conduct two separate but related studies on creative performance that aim to provide support that creative performance should be included in models of job performance, and ultimately included in performance evaluations in organizations. Study 1 is a meta-analysis on the relationship between creative performance and task performance, and the relationship between creative performance and organizational citizenship behavior (OCB). Overall, I found support for a medium to large corrected correlation for both the creative performance-task performance (ρ = .51) and creative performance-OCB (ρ = .49) relationships. Further, I also found that both rating-source and study location were significant moderators. Study 2 is a process model that includes creative performance alongside task performance and OCB as the outcome variables. I test a model in which both individual differences (specifically: conscientiousness, extraversion, proactive personality, and self-efficacy) and job characteristics (autonomy, feedback, and supervisor support) predict creative performance, task performance, and OCB through engagement as a mediator. In a sample of 299 employed individuals, I found that all the individual differences and job characteristics were positively correlated with all three performance criteria. I also looked at these relationships in a multiple regression framework and most of the individual differences and job characteristics still predicted the performance criteria. In the mediation analyses, I found support for engagement as a significant mediator of the individual differences-performance and job characteristics-performance relationships. Taken together, Study 1 and Study 2 support the notion that creative performance should be included in models of job performance. Implications for both researchers and practitioners alike are discussed.

Work creativity as a dimension of job performance

To stay competitive, many employers are looking for creative and innovative employees to add value to their organization. However, current models of job performance overlook creative performance as an important criterion to measure in the workplace. The purpose of this dissertation is to conduct two separate but related studies on creative performance that aim to provide support that creative performance should be included in models of job performance, and ultimately included in performance evaluations in organizations. Study 1 is a meta-analysis on the relationship between creative performance and task performance, and the relationship between creative performance and organizational citizenship behavior (OCB). Overall, I found support for a medium to large corrected correlation for both the creative performance-task performance (ρ = .51) and creative performance-OCB (ρ = .49) relationships. Further, I also found that both rating-source and study location were significant moderators. Study 2 is a process model that includes creative performance alongside task performance and OCB as the outcome variables. I test a model in which both individual differences (specifically: conscientiousness, extraversion, proactive personality, and self-efficacy) and job characteristics (autonomy, feedback, and supervisor support) predict creative performance, task performance, and OCB through engagement as a mediator. In a sample of 299 employed individuals, I found that all the individual differences and job characteristics were positively correlated with all three performance criteria. I also looked at these relationships in a multiple regression framework and most of the individual differences and job characteristics still predicted the performance criteria. In the mediation analyses, I found support for engagement as a significant mediator of the individual differences-performance and job characteristics-performance relationships. Taken together, Study 1 and Study 2 support the notion that creative performance should be included in models of job performance. Implications for both researchers and practitioners alike are discussed.^