Measuring Consumer Satisfaction in Family Preservation Services: Identifying Instrument Domains

Measuring consumer satisfaction in the social services has become an important source of feedback for the improvement of service delivery. Consumer satisfaction has recently been incorporated into family preservation evaluation. This article reviews instruments used to measure consumer satisfaction in family preservation services and other related areas. Trends in current practice are examined and instrument dimensions are identified. Finally, some recommendations are made about the application of consumer satisfaction measurement in family preservation services.

THE DESIGN AND DEVELOPMENT OF A SURVEY INSTRUMENT FOR THE ASSESSMENT OF OCCUPATIONAL STRESS

Using stress and coping as a unifying theoretical concept, a series of five models was developed in order to synthesize the survey questions and to classify information. These models identified the question, listed the research study, described measurements, listed workplace data, and listed industry and national reference data.^ A set of 38 instrument questions was developed within the five coping correlate categories. In addition, a set of 22 stress symptoms was also developed. The study was conducted within two groups, police and professors, on a large university campus. The groups were selected because their occupations were diverse, but they were a part of the same macroenvironment. The premise was that police officers would be more highly stressed than professors.^ Of a total study group of 80, there were 37 respondents. The difference in the mean stress responses was observable between the two groups. Not only were the responses similar within each group, but the stress level of response was also similar within each group. While the response to the survey instrument was good, only 3 respondents answered the stress symptom survey properly. It was determined that none of the 37 respondents believed that they were ill. This perception of being well was also evidenced by the grand mean of the stress scores of 2.76 (3.0 = moderate stress). This also caused fewer independent variables to be entered in the multiple regression model.^ The survey instrument was carefully designed to be universal. Universality is the ability to transcend occupational or regional definitions as applied to stress. It is the ability to measure responses within broad categories such as physiological, emotional, behavioral, social, and cognitive functions without losing the ability to measure the detail within the individual questions, or the relationships between questions and categories.^ Replication is much easier to achieve with standardized categories, questions, and measurement procedures such as those developed for the universal survey instrument. Because the survey instrument is universal it can be used as an analytical device, an assessment device, a basic tool for planning and a follow-up instrument to measure individual response to planned reductions in occupational stress. (Abstract shortened with permission of author.) ^

Measuring health state preferences for hemophilia: Development of a disease-specific utility instrument

Research has shown that disease-specific health related quality of life (HRQoL) instruments are more responsive than generic instruments to particular disease conditions. However, only a few studies have used disease-specific instruments to measure HRQoL in hemophilia. The goal of this project was to develop a disease-specific utility instrument that measures patient preferences for various hemophilia health states. The visual analog scale (VAS), a ranking method, and the standard gamble (SG), a choice-based method incorporating risk, were used to measure patient preferences. Study participants (n = 128) were recruited from the UT/Gulf States Hemophilia and Thrombophilia Center and stratified by age: 0–18 years and 19+. ^ Test retest reliability was demonstrated for both VAS and SG instruments: overall within-subject correlation coefficients were 0.91 and 0.79, respectively. Results showed statistically significant differences in responses between pediatric and adult participants when using the SG (p = .045). However, no significant differences were shown between these groups when using the VAS (p = .636). When responses to VAS and SG instruments were compared, statistically significant differences in both pediatric (p < .0001) and adult (p < .0001) groups were observed. Data from this study also demonstrated that persons with hemophilia with varying severity of disease, as well as those who were HIV infected, were able to evaluate a range of health states for hemophilia. This has important implications for the study of quality of life in hemophilia and the development of disease-specific HRQoL instruments. ^ The utility measures obtained from this study can be applied in economic evaluations that analyze the cost/utility of alternative hemophilia treatments. Results derived from the SG indicate that age can influence patients' preferences regarding their state of health. This may have implications for considering treatment options based on the mean age of the population under consideration. Although both instruments independently demonstrated reliability and validity, results indicate that the two measures may not be interchangeable. ^

Use of cognitive interviewing in the development of a Spanish language genetic knowledge instrument

Study purpose. Genetic advances are significantly impacting healthcare, yet recent studies of ethnic group participation in genetic services demonstrate low utilization rates by Latinos. Limited genetic knowledge is a major barrier. The purpose of this study was to field test items in a Spanish-language instrument that will be used to measure genetic knowledge relevant to type 2 diabetes among members of the ethnically heterogeneous U.S. Latino community. Accurate genetic knowledge measurement can provide the foundation for interventions to enhance genetic service utilization. ^ Design. Three waves of cognitive interviews were conducted in Spanish to field test 44 instrument items Thirty-six Latinos, with 12 persons representative of Mexican, Central and South American, and Cuban heritage participated, including 7 males and 29 females between 22 and 60 years of age; 17 participants had 12 years or less of education. ^ Methods. Text narratives from transcriptions of audiotaped interviews were qualitatively analyzed using a coding strategy to indicate potential sources of response error. Through an iterative process of instrument refinement, codes that emerged from the data were used to guide item revisions at the conclusion of each phase; revised items were examined in subsequent interview waves. ^ Results. Inter-cultural and cross-cultural themes associated with difficulties in interpretation and grammatical structuring of items were identified; difficulties associated with comprehension reflected variations in educational level. Of the original 44 items, 32 were retained, 89% of which were revised. Six additional items reflective of cultural knowledge were constructed, resulting in a 38-item instrument. ^ Conclusions. Use of cognitive interviewing provided a valuable tool for detecting both potential sources of response error and cultural variations in these sources. Analysis of interview data guided successive instrument revisions leading to improved item interpretability and comprehension. Although testing in a larger sample will be essential to test validity and reliability, the outcome of field testing suggests initial content validity of a Spanish-language instrument to measure genetic knowledge relative to type 2 diabetes. ^ Keywords. Latinos, genetic knowledge, instrument development, cognitive interviewing ^

Validation of an instrument for assessing the medicolegal death investigation system: A public health approach

Background. This study validated the content of an instrument designed to assess the performance of the medicolegal death investigation system. The instrument was modified from Version 2.0 of the Local Public Health System Performance Assessment Instrument (CDC) and is based on the 10 Essential Public Health Services. ^ Aims. The aims were to employ a cognitive testing process to interview a randomized sample of medicolegal death investigation office leaders, qualitatively describe the results, and revise the instrument accordingly. ^ Methods. A cognitive testing process was used to validate the survey instrument's content in terms of the how well participants could respond to and interpret the questions. Twelve randomly selected medicolegal death investigation chiefs (or equivalent) that represented the seven types of medicolegal death investigation systems and six different state mandates were interviewed by telephone. The respondents also were representative of the educational diversity within medicolegal death investigation leadership. Based on respondent comments, themes were identified that permitted improvement of the instrument toward collecting valid and reliable information when ultimately used in a field survey format. ^ Results. Responses were coded and classified, which permitted the identification of themes related to Comprehension/Interpretation, Retrieval, Estimate/Judgment, and Response. The majority of respondent comments related to Comprehension/Interpretation of the questions. Respondents identified 67 questions and 6 section explanations that merited rephrasing, adding, or deleting examples or words. In addition, five questions were added based on respondent comments. ^ Conclusion. The content of the instrument was validated by cognitive testing method design. The respondents agreed that the instrument would be a useful and relevant tool for assessing system performance. ^

A STUDY TO INVESTIGATE DIET DIVERSITY AS A NUTRITION ASSESSMENT INSTRUMENT

Nutrient intake and specific food item data from 24-hour dietary recalls were utilized to study the relationship between measures of diet diversity and dietary adequacy in a population of white females of child-bearing age and socioeconomic subgroups of that population. As the basis of the diet diversity measures, twelve food groups were constructed from the 24-hour recall data and the number of unique foods per food group counted and weighted according to specified weighting schemes. Utilizing these food groups, nine diet diversity indices were developed.^ Sensitivity/specificity analysis was used to determine the ability of varying levels of selected diet diversity indices to identify individuals above and below preselected intakes of different nutrients. The true prevalence proportions, sensitivity and specificity, false positive and false negative rates, and positive predictive values observed at the selected levels of diet diversity indices were investigated in relation to the objectives and resources of a variety of nutrition improvement programs. Diet diversity indices constructed from the total population data were evaluated as screening tools for respondent nutrient intakes in each of the socioeconomic subgroups as well.^ The results of the sensitivity/specificity analysis demonstrated that the false positive rate, the false negative rate, or both were too high at each diversity cut-off level to validate the widespread use of any of the diversity indices in the dietary assessment of the study population. Although diet diversity has been shown to be highly correlated with the intakes of a number of nutrients, the diet diversity indices constructed in this study did not adequately represent nutrient intakes in the diet as reported, in this study, intakes as reported in the 24-hour dietary recall. Specific cut-off levels of selected diversity indices might have limited application in some nutrition programs. The results were applicable to the sensitivity/specificity analyses in the socioeconomic subgroups as well as in the total population. ^