Assessment of interrater agreement using a binomial-based statistic: K(bin)

This project develops K(bin), a relatively simple, binomial based statistic for assessing interrater agreement in which expected agreement is calculated a priori from the number of raters involved in the study and number of categories on the rating tool. The statistic is logical in interpretation, easily calculated, stable for small sample sizes, and has application over a wide range of possible combinations from the simplest case of two raters using a binomial scale to multiple raters using a multiple level scale.^ Tables of expected agreement values and tables of critical values for K(bin) which include power to detect three levels of the population parameter K for n from 2 to 30 and observed agreement $\ge$.70 calculated at alpha =.05,.025, and.01 are included.^ An example is also included which describes the use of the tables for planning and evaluating an interrater reliability study using the statistic, K(bin). ^

Patient classification system: An integrated method for measuring nursing intensity and optimizing resource allocation

A patient classification system was developed integrating a patient acuity instrument with a computerized nursing distribution method based on a linear programming model. The system was designed for real-time measurement of patient acuity (workload) and allocation of nursing personnel to optimize the utilization of resources.^ The acuity instrument was a prototype tool with eight categories of patients defined by patient severity and nursing intensity parameters. From this tool, the demand for nursing care was defined in patient points with one point equal to one hour of RN time. Validity and reliability of the instrument was determined as follows: (1) Content validity by a panel of expert nurses; (2) predictive validity through a paired t-test analysis of preshift and postshift categorization of patients; (3) initial reliability by a one month pilot of the instrument in a practice setting; and (4) interrater reliability by the Kappa statistic.^ The nursing distribution system was a linear programming model using a branch and bound technique for obtaining integer solutions. The objective function was to minimize the total number of nursing personnel used by optimally assigning the staff to meet the acuity needs of the units. A penalty weight was used as a coefficient of the objective function variables to define priorities for allocation of staff.^ The demand constraints were requirements to meet the total acuity points needed for each unit and to have a minimum number of RNs on each unit. Supply constraints were: (1) total availability of each type of staff and the value of that staff member (value was determined relative to that type of staff's ability to perform the job function of an RN (i.e., value for eight hours RN = 8 points, LVN = 6 points); (2) number of personnel available for floating between units.^ The capability of the model to assign staff quantitatively and qualitatively equal to the manual method was established by a thirty day comparison. Sensitivity testing demonstrated appropriate adjustment of the optimal solution to changes in penalty coefficients in the objective function and to acuity totals in the demand constraints.^ Further investigation of the model documented: correct adjustment of assignments in response to staff value changes; and cost minimization by an addition of a dollar coefficient to the objective function. ^

Self -management behavior in patients with cystic fibrosis

Self-management is being promoted in cystic fibrosis (CF). However, it has not been well studied. Principal aims of this research were (1) to evaluate psychometric properties of a CF disease status measure, the NIH Clinical Score; (2) to develop and validate a measure of self-management behavior, the SMQ-CF scale, and (3) to examine the relation between self-management and disease status in CF patients over two years.^ In study 1, NIH Clinical Scores for 200 patients were used. The scale was examined for internal consistency, interrater reliability, and content validity using factor analysis. The Cronbach's alpha (.81) and interrater reliability (.90) for the total scale were high. General scale items were less reliable. Factor analysis indicated that most of the variance in disease status is accounted for by Factor 1 which consists of pulmonary disease items.^ The SMQ-CF measures the performance of CF self-management. Pilot testing was done with 98 CF primary caregivers. Internal consistency reliability, social desirability bias, and content validity using factor analysis were examined. Internal consistency was good (alpha =.95). Social desirability correlation was low (r =.095). Twelve factors identified were consistent with conceptual groupings of behaviors. Around two hundred caregivers from two CF centers were surveyed and multivariate analysis of variance was used to assess construct validity. Results confirmed expected relations between self-management, patient age, and disease status. Patient age accounted for 50% and disease status 18% of the variance in the SMQ-CF scale.^ It was hypothesized that self-management would positively affect future disease status. Data from 199 CF patients (control and education intervention groups) were examined. Models of hypothesized relations were tested using LISREL structural equation modeling. Results indicated that the relations between baseline self-management and Time 1 disease status were not significant. Significant relations were observed in self-management behaviors from time 1 to time 2 and patterns of significant relations differed between the two groups.^ This research has contributed to refinements in the ability to measure self-management behavior and disease status outcomes in cystic fibrosis. In addition, it provides the first steps in exploratory behavioral analysis with regard to self-management in this disease. ^

Validity and reliability of a measure of Latina mother's nutritional self-efficacy

Background. Not only has obesity played a role in Texas adults but it is also becoming a large issue among low-income Latino children. In Latino children between 2-5 years of age, the Pediatric Nutrition Surveillance data in 1997 found the prevalence of obesity was 12 percent, highest among all ethnic groups. Children learn what and how to eat from their environment. Despite many mothers being working mothers they are still the principal caregivers and source of influence on their toddler's diet. Self-efficacy, a concept created by Albert Bandura, one's belief that one is capable of performing a behavior needed to reach an intended goal, is increasingly becoming important in nutrition and health education. This study is important to understand the degree of impact that a mother's self-efficacy will have on a child's diet. This is useful knowing if influencing a mother's self-efficacy could improve a child's diet to prevent certain public health issues such as obesity and diabetes. The purpose of this study was to examine nutrition self-efficacy of Latina mothers, focusing on sweets and beverage and if their self-efficacy impacted their child's diet. Methods. The data was collected during July-September 2008. Mothers were recruited from two federally qualified San Antonio health centers. In order to qualify, participants had to be Hispanic with children of toddler age. Mothers were informed of incentives available upon completion. The interview consisted of demographic info, a set of five self-efficacy questions repeated at completion, testing reliability and a 24-hour food recall diary asked of the participant's child's diet. Results. There were 225 mothers who participated between both clinics. The Crohnbach alpha scores for the two different times the self-efficacy questions were asked were .44 corresponding to the first time and .49 for the second time. The three most common beverages reported were milk, juice, and water. The mothers who met or gave their child more milk than recommended by the scientific community, 800mg of calcium/3 cups (24oz) set, had a higher self-efficacy score than those who did not meet the standard at all. Mothers who gave their children more juice than the standard recommends, 4-6oz for children 1-6 years of age, had slightly higher self-efficacy scores than mother's who simply met the standard. In general, the lower the mother's self-efficacy, the more sweets they gave their child and vice versa. Conclusion. This study's Kappa values were adequate and this research showed that Latina mothers did in fact have high self-efficacy. In general some of the children's diets did not reflect the current scientific nutrition recommendations. In order to improve self-efficacy and have an impact on children's diets, the scientific community has a responsibility to make recommendations that are easily understood and can be put into practice. The public health community needs to ensure that we encourage those we serve to be more active in their health and educate them about what constitutes good health and nutrition for both themselves and their children.^

Validity and reliability of using MVP 4 Function Walk4Life digital pedometers to assess physical activity levels among preschool-aged Head Start children

The Surgeon General recommends preschoolers 3-5 years old accumulate 60 minutes of moderate-to-vigorous physical activity (MVPA) per day. However, there is limited data measuring physical activity (PA) and MVPA amongst this population. The purpose of this cross-sectional study is to determine the validity, reliability, and feasibility of using MVP 4 Function Walk4Life digital pedometers (MVP-4) in measuring MVPA among preschoolers using the newly modified direct observational technique, System for Observing Fitness Instruction Time-Preschool Version (SOFIT-P) as the gold standard. An ethnically diverse population of 3-5 year old underserved children were recruited from two Harris County Department of Education (HCDE) Head Start centers. For 2 days at baseline and 2 days at post-test, 75 children enrolled wore MVP-4 pedometers for approximately 6-hours per observation day and were observed using SOFIT-P during predominantly active times. Statistical analyses used Pearson "r" correlation coefficients to determine mean minutes of PA and MVPA, convergent and criterion validity, and reliability. Significance was set at p = <0.05. Feasibility was determined through process evaluation information collected during this study via observations from data collectors and teacher input. Results show mean minutes of PA and MVPA ranged between 30-42 and 11-14 minutes, respectively. Convergent validity comparing BMI percentiles with MVP-4 PA outcomes show no significance at pre-test; however, each measurement at post-test showed significance for MVPA (p = 0.0247, p = 0.0056), respectively. Criterion validity comparing percent MVPA time between SOFIT-P and MVP-4 pedometers was determined; however, results deemed insufficient due to inconsistency in observation times while using the newly developed SOFIT-P. Reliability measures show no significance at pre-test, yet show significant results for all PA outcomes at post-test (p = 0.001, p = 0.001, p = 0.0010, p = 0.003), respectively. Finally, MVP-4 pedometers lacked feasibility due to logistical barriers in design. Researchers feel the significant results at post-test are secondary to increased familiarity and more accurate placement of pedometers across time. Researchers suggest manufacturers of MVP-4 pedometers further modify the instrument for ease of use with this population, following which future studies ought to determine validity using objective measures or all-day direct observation techniques.^

Evidence of reliability and validity of IPAQ for Mexican-American adults

Physical activity is a key component of life-style modification process which helps to reduce the risk of developing chronic diseases. It is important to have accurate estimates of physical activity to identify sedentary populations where interventions might be helpful. The International Physical Activity Questionnaire (IPAQ) short version has been used to estimate physical activity in diverse populations. However, there is little literature depicting the use of the IPAQ short version in Mexican America population. This study addressed the predictive validity and test-retest reliability of the IPAQ short version in Mexican American adults. The analysis was performed on 97 participants enrolled in the Cameron County Hispanic Cohort. Individuals selected in this study were 18 years of age or older. The predictive validity was evaluated by studying the relationship between physical activity and biomarkers known to be correlated with physical activity, namely, TNF-α, Adiponectin, and HDL. Multiple linear regression analysis was performed to delineate predictive validity. To assess test-retest reliability, two IPAQ-short last seven days questionnaires were interviewer administered to the participants on the same day, approximately two hours apart. Test-Retest reliability of IPAQ was estimated by performing intraclass correlations between the readings at two different time periods. The study showed that the IPAQ – short version used in the above study had acceptable test-retest reliability in the Mexican American population. This study showed that the IPAQ – short version did not have acceptable predictive validity when looking at physical activity and TNF-α, Adiponectin, and HDL in this sample.^

An examination of the cross-cultural reliability and construct validity of a measure of loneliness

Loneliness is a pervasive, rather common experience in American culture, particularly notable among adolescents. However, the phenomenon is not well documented in the cross-cultural psychiatric literature. For psychiatric epidemiology to encompass a wide array of psychopathologic phenomena, it is important to develop useful measures to characterize and classify both non-clinical and clinical dysfunction in diverse subgroups and cultures.^ The goal of this research was to examine the cross-cultural reliability and construct validity of a scale designed to measure loneliness. The Roberts Loneliness Scale (RLS-8) was administered to 4,060 adolescents ages 10-19 years enrolled in high schools along either side of the Texas-Tamaulipas border region between the U.S. and Mexico. Data collected in 1988 from a study focusing on substance use and psychological distress among adolescents in these regions were used to examine the operating characteristics of the RLS-8. A sample stratified by nationality and language, age, gender, and grade was used for analysis.^ Results indicated that in general the RLS-8 has moderate reliability in the U.S. sample, but not in the Mexican sample. Validity analyses demonstrated that there was evidence for convergent validity of the RLS-8 in the U.S. sample, but none in the Mexican sample. Discriminant validity of the measures in neither sample could be established. Based on the factor structure of the RLS-8, two subscales were created and analyzed for construct validity. Evidence for convergent validity was established for both subscales in both national samples. However, the discriminant validity of the measure remains unsubstantiated in both national samples. Also, the dimensionality of the scale is unresolved.^ One primary goal for future cross-cultural research would be to develop and test better defined culture-specific models of loneliness within the two cultures. From such scientific endeavor, measures of loneliness can be developed or reconstructed to classify the phenomenon in the same manner across cultures. Since estimates of prevalence and incidence are contingent upon reliable and valid screening or diagnostic measures, this objective would serve as an important foundation for future psychiatric epidemiologic inquiry into loneliness. ^

AN INVESTIGATION OF THE RELIABILITY AND VALIDITY OF THE CENTER FOR EPIDEMIOLOGIC STUDIES DEPRESSION SCALE IN THREE ETHNIC GROUPS

Epidemiologic studies of mental disorder have called attention to the need for identifying untreated cases and to the inadequacies of the instruments available for this purpose. Accurate case ascertainment devices are the basis of sound epidemiology. Without these, neither case classification nor analytic studies of risk factors is possible.^ The purpose of this research was to examine the reliability and validity of an instrument designed to measure depressive symptoms in community populations--the Center for Epidemiologic Studies Depression Scale (CES-D Scale). Two particular foci of the study were whether or not the scale had the same statistical structure across three ethnic groups and whether or not the magnitude and pattern of rates of symptoms for these groups were affected by one source of response error, that due to response tendencies. The effects of age and education on the pattern and magnitude of rates also were examined. In addition, the reliability and validity of the measures of response tendencies were assessed.^ The study population consisted of residents of Alameda County, California. A stratified sample of approximately 700 whites, blacks and Mexican-Americans was interviewed in the summer and fall of 1978.^ The results of the analysis indicated that the scale was reliable and measured a similar content domain across the three ethnic groups. The unadjusted sex- and ethnic-specific rates of depressive symptoms showed an ethnic pattern for both sexes: rates for whites were lowest, those for Mexican-Americans were highest, and those for blacks were intermediate. Measures of response tendencies--need for social approval, trait desirability, and acquiescence--affected the magnitude of the rates for most comparisons. Likewise, the pattern of rates changed somewhat from that originally observed. The one fairly consistent observation was that rates for Mexican-American women were higher than those for the other two female subgroups in most of the comparisons. These results must be considered in the context of the reliability and validity assessment of the measures of response tendencies which indicated the tenuousness of these measures.^ Age affected the ethnic pattern of rates for men in an inconsistent way; for women, Mexican-Americans continued to have higher rates than whites or blacks in all age categories. Education affected the magnitude of rates for women but not for men. For both men and women, Mexican-Americans had higher rates in all educational strata. Rates for women showed an inverse association with education while those for men did not. ^