Assessing the construct validity of a measure of learning agility

This study examined the construct validity of the Choices questionnaire that purported to support the theory of Learning Agility. Specifically, Learning Agility attempts to predict an individual's potential performance in new tasks. The construct validity will be measured by examining the convergent/discriminant validity of the Choices Questionnaire against a cognitive ability measure and two personality measures. The Choices Questionnaire did tap a construct that is unique to the cognitive ability and the personality measures, thus suggesting that this measure may have considerable value in personnel selection. This study also examined the relationship of this pew measure to job performance and job promotability. Results of this study found that the Choices Questionnaire predicted job performance and job promotability above and beyond cognitive ability and personality. Data from 107 law enforcement officers, along with two of their co-workers and a supervisor resulted in a correlation of .08 between Learning Agility and cognitive ability. Learning Agility correlated .07 with Learning Goal Orientation and. 17 with Performance Goal Orientation. Correlations with the Big Five Personality factors ranged from −.06 to. 13 with Conscientiousness and Openness to Experience, respectively. Learning Agility correlated .40 with supervisory ratings of job promotability and correlated .3 7 with supervisory ratings of overall job performance. Hierarchical regression analysis found incremental validity for Learning Agility over cognitive ability and the Big Five factors of personality for supervisory ratings of both promotability and overall job performance. A literature review was completed to integrate the Learning Agility construct into a nomological net of personnel selection research. Additionally, practical applications and future research directions are discussed. ^

Evaluating the external validity of comorbid disorders in youth using rating scales and behavior observations

To evaluate the theoretical underpinnings of current categorical approaches to classify childhood psychopathological conditions, this dissertation examined whether children with a single diagnosis of an anxiety disorder (ANX only) and children with an anxiety diagnosis comorbid with other diagnoses (i.e., anxiety + anxiety disorder [ANX + ANX], anxiety + depressive disorder [ANX + DEP], and anxiety + disruptive disorder [ANX + EXT]) could be differentiated using external validation criteria of clinical phenomenology (i.e., levels of anxiety, depression, and internalizing, externalizing and total behavior problems). This study further examined whether the four groups could be differentiated in terms of their interaction patterns with their parents and peers, respectively. The sample consisted of 129 youth and their parents who presented to the Child Anxiety and Phobia Program (CAPP) housed within the Child and Family Psychosocial Research Center at Florida International University, Miami. Youth were between the ages of 8 and 14 years old. A battery of questionnaires was used to assess participants' clinical presentation in terms of levels of anxiety, depression, and internalizing and externalizing symptoms. Family and peer interaction were evaluated through rating scales and through behavior observation tasks. Statistics based on the parameter estimates of the structured equation models indicated that all the comorbid groups were significantly different from the pure anxiety disorder group when it came to depression indices of clinical phenomenology. Further, significant differences appeared mainly in terms of the ANX + DEP comorbid group relative to the other comorbid groups. In terms of Parent-child interaction the ANX + EXT and the ANX + DEP comorbid groups were differentiated from the pure anxiety disorder and ANX + ANX comorbid group when it came to the appraisal of the parent/child relationship by the parent, and the acceptance subscale according to the mother report. In terms of peer-child interaction the ANX + EXT and the ANX + DEP comorbid groups were statistically significantly different from the pure anxiety disorder only when it came to the positive interactions and the social skills as rated by mother. Limitations and future research recommendations are discussed.

Evaluation of the internal structural validity of the Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition) anxiety disorders in children and adolescents

The purpose of the present dissertation was to evaluate the internal validity of symptoms of four common anxiety disorders included in the Diagnostic and Statistical Manual of Mental Disorders fourth edition (text revision) (DSM-IV-TR; American Psychiatric Association, 2000), namely, separation anxiety disorder (SAD), social phobia (SOP), specific phobia (SP), and generalized anxiety disorder (GAD), in a sample of 625 youth (ages 6 to 17 years) referred to an anxiety disorders clinic and 479 parents. Confirmatory factor analyses (CFAs) were conducted on the dichotomous items of the SAD, SOP, SP, and GAD sections of the youth and parent versions of the Anxiety Disorders Interview Schedule for DSM-IV (ADIS-IV: C/P; Silverman & Albano, 1996) to test and compare a number of factor models including a factor model based on the DSM. Contrary to predictions, findings from CFAs showed that a correlated model with five factors of SAD, SOP, SP, GAD worry, and GAD somatic distress, provided the best fit of the youth data as well as the parent data. Multiple group CFAs supported the metric invariance of the correlated five factor model across boys and girls. Thus, the present study’s finding supports the internal validity of DSM-IV SAD, SOP, and SP, but raises doubt regarding the internal validity of GAD.^

An Examination of the Validity of English Language Achievement Test Scores in a LEP Student Population

Approximately 1,700 limited English proficient (LEP) and recently exited LEP students in grades 4 and 10 were tested using both an English and a comparable Spanish language achievement test. Many LEP and former LEP students performed better in math taking the test in Spanish compared to taking it in English.

Variables affecting the success of older applicants for employment in a public organization: An age -cued cognitive model of administrative decision-making

The dissertation takes a multivariate approach to answer the question of how applicant age, after controlling for other variables, affects employment success in a public organization. In addition to applicant age, there are five other categories of variables examined: organization/applicant variables describing the relationship of the applicant to the organization; organization/position variables describing the target position as it relates to the organization; episodic variables such as applicant age relative to the ages of competing applicants; economic variables relating to the salary needs of older applicants; and cognitive variables that may affect the decision maker's evaluation of the applicant. ^ An exploratory phase of research employs archival data from approximately 500 decisions made in the past three years to hire or promote applicants for positions in one public health administration organization. A logit regression model is employed to examine the probability that the variables modify the effect of applicant age on employment success. A confirmatory phase of the dissertation is a controlled experiment in which hiring decision makers from the same public organization perform a simulated hiring decision exercise to evaluate hypothetical applicants of similar qualifications but of different ages. The responses of the decision makers to a series of bipolar adjective scales add support to the cognitive component of the theoretical model of the hiring decision. A final section contains information gathered from interviews with key informants. ^ Applicant age has tended to have a curvilinear relationship with employment success. For some positions, the mean age of the applicants most likely to succeed varies with the values of the five groups of moderating variables. The research contributes not only to the practice of public personnel administration, but is useful in examining larger public policy issues associated with an aging workforce. ^

Participation in a positive youth development program and multi -problem alternative high school students' transformative goals: An investigation of the theoretical and methodological utility and validity of the Transformative Goal Attainment (TGAS)

There is a growing body of literature that provides evidence for the efficacy of positive youth development programs in general and preliminary empirical support for the efficacy of the Changing Lives Program (CLP) in particular. This dissertation sought to extend previous efforts to develop and preliminarily examine the Transformative Goal Attainment Scale (TGAS) as a measure of participant empowerment in the promotion of positive development. Consistent with recent advances in the use of qualitative research methods, this dissertation sought to further investigate the utility of Relational Data Analysis (RDA) for providing categorizations of qualitative open-ended response data. In particular, a qualitative index of Transformative Goals, TG, was developed to complement the previously developed quantitative index of Transformative Goal Attainment (TGA), and RDA procedures for calculating reliability and content validity were refined. Second, as a Stage I pilot/feasibility study this study preliminarily examined the potentially mediating role of empowerment, as indexed by the TGAS, in the promotion of positive development. ^ Fifty-seven participants took part in this study, forty CLP intervention participants and seventeen control condition participants. All 57 participants were administered the study's measures just prior to and just following the fall 2003 semester. This study thus used a short-term longitudinal quasi-experimental research design with a comparison control group. ^ RDA procedures were refined and applied to the categorization of open-ended response data regarding participants' transformative goals (TG) and future possible selves (PSQ-QE). These analyses revealed relatively strong, indirect evidence for the construct validity of the categories as well as their theoretically meaningful structural organization, thereby providing sufficient support for the utility of RDA procedures in the categorization of qualitative open-ended response data. ^ In addition, transformative goals (TG) and future possible selves (PSQ-QE), and the quantitative index of perceived goal attainment (TGA) were evaluated as potential mediators of positive development by testing their relationships to other indices of positive intervention outcome within a four-step method involving both analysis of variance (ANOVA and RMANOVAs) and regression analysis. Though more limited in scope than the efforts at the development and refinement of the measures of these mediators, the results were also promising. ^

Examining the validity of the GRE General Test scores and undergraduate GPA for predicting success in graduate school at a large racially and ethnically diverse public university in Southeast Florida

The purpose of the study was to determine the degree of relationships among GRE scores, undergraduate GPA (UGPA), and success in graduate school, as measured by first year graduate GPA (FGPA), cumulative graduate GPA, and degree attainment status. A second aim of the study was to determine whether the relationships between the composite predictor (GRE scores and UGPA) and the three success measures differed by race/ethnicity and sex. A total of 7,367 graduate student records (masters, 5,990; doctoral: 1,377) from 2000 to 2010 were used to evaluate the relationships among GRE scores, UGPA and the three success measures. Pearson's correlation, multiple linear and logistic regression, and hierarchical multiple linear and logistic regression analyses were performed to answer the research questions. The results of the correlational analyses differed by degree level. For master's students, the ETS proposed prediction that GRE scores are valid predictors of first year graduate GPA was supported by the findings from the present study; however, for doctoral students, the proposed prediction was only partially supported. Regression and correlational analyses indicated that UGPA was the variable that consistently predicted all three success measures for both degree levels. The hierarchical multiple linear and logistic regression analyses indicated that at master's degree level, White students with higher GRE Quantitative Reasoning Test scores were more likely to attain a degree than Asian Americans, while International students with higher UGPA were more likely to attain a degree than White students. The relationships between the three predictors and the three success measures were not significantly different between men and women for either degree level. Findings have implications both for practice and research. They will provide graduate school administrators with institution-specific validity data for UGPA and the GRE scores, which can be referenced in making admission decisions, while they will provide empirical and professionally defensible evidence to support the current practice of using UGPA and GRE scores for admission considerations. In addition, new evidence relating to differential predictions will be useful as a resource reference for future GRE validation researchers.

The Validity of Broad and Narrow Personality Traits For Predicting Job Performance: The Differential Effects of Time

Research into the dynamicity of job performance criteria has found evidence suggesting the presence of rank-order changes to job performance scores across time as well as intraindividual trajectories in job performance scores across time. These findings have influenced a large body of research into (a) the dynamicity of validities of individual differences predictors of job performance and (b) the relationship between individual differences predictors of job performance and intraindividual trajectories of job performance. In the present dissertation, I addressed these issues within the context of the Five Factor Model of personality. The Five Factor Model is arranged hierarchically, with five broad higher-order factors subsuming a number of more narrowly tailored personality facets. Research has debated the relative merits of broad versus narrow traits for predicting job performance, but the entire body of research has addressed the issue from a static perspective -- by examining the relative magnitude of validities of global factors versus their facets. While research along these lines has been enlightening, theoretical perspectives suggest that the validities of global factors versus their facets may differ in their stability across time. Thus, research is needed to not only compare the relative magnitude of validities of global factors versus their facets at a single point in time, but also to compare the relative stability of validities of global factors versus their facets across time. Also necessary to advance cumulative knowledge concerning intraindividual performance trajectories is research into broad vs. narrow traits for predicting such trajectories. In the present dissertation, I addressed these issues using a four-year longitudinal design. The results indicated that the validities of global conscientiousness were stable across time, while the validities of conscientiousness facets were more likely to fluctuate. However, the validities of emotional stability and extraversion facets were no more likely to fluctuate across time than those of the factors. Finally, while some personality factors and facets predicted performance intercepts (i.e., performance at the first measurement occasion), my results failed to indicate a significant effect of any personality variable on performance growth. Implications for research and practice are discussed.

Examining the Validity of the GRE General Test Scores and Undergraduate GPA for Predicting Success in Graduate School at a Large Racially and Ethnically Diverse Public University in Southeast Florida

The purpose of the study was to determine the degree of relationships among GRE scores, undergraduate GPA (UGPA), and success in graduate school, as measured by first year graduate GPA (FGPA), cumulative graduate GPA, and degree attainment status. A second aim of the study was to determine whether the relationships between the composite predictor (GRE scores and UGPA) and the three success measures differed by race/ethnicity and sex. A total of 7,367 graduate student records (masters, 5,990; doctoral: 1,377) from 2000 to 2010 were used to evaluate the relationships among GRE scores, UGPA and the three success measures. Pearson’s correlation, multiple linear and logistic regression, and hierarchical multiple linear and logistic regression analyses were performed to answer the research questions. The results of the correlational analyses differed by degree level. For master’s students, the ETS proposed prediction that GRE scores are valid predictors of first year graduate GPA was supported by the findings from the present study; however, for doctoral students, the proposed prediction was only partially supported. Regression and correlational analyses indicated that UGPA was the variable that consistently predicted all three success measures for both degree levels. The hierarchical multiple linear and logistic regression analyses indicated that at master’s degree level, White students with higher GRE Quantitative Reasoning Test scores were more likely to attain a degree than Asian Americans, while International students with higher UGPA were more likely to attain a degree than White students. The relationships between the three predictors and the three success measures were not significantly different between men and women for either degree level. Findings have implications both for practice and research. They will provide graduate school administrators with institution-specific validity data for UGPA and the GRE scores, which can be referenced in making admission decisions, while they will provide empirical and professionally defensible evidence to support the current practice of using UGPA and GRE scores for admission considerations. In addition, new evidence relating to differential predictions will be useful as a resource reference for future GRE validation researchers.