Challenges and perspectives of academic evaluation

Academic evaluation has been an essential component of modern science since its inception, as science has moved away from personalized patronage toward its contemporary role as an essential enterprise of contemporary, democratic societies. In recent years, Brazil has experienced sustained growth in its scientific output, which is nowadays fully compatible with its status as a high middle-income country striving to become a fully developed, more equitable country in the years to come. Growth usually takes place amidst challenges and dilemmas and, in Brazil as elsewhere, academic evaluation is not exempt from such difficulties. In a large, profoundly heterogeneous country with a national evaluation system and nationwide on-line platforms disseminating information on the most disparate fields of knowledge, the main challenges refer to how to pay attention to detail without losing sight of comprehensiveness and how to handle social and regional diversity while preserving academic excellence as the fundamental benchmark.

Avaliação acadêmica, ideologia e poder

O artigo discute a avaliação acadêmica no contexto caracterizado pelo produtivismo. Como introdução, analisa o episódio da lista dos improdutivos, ocorrido em 1988 na USP, procurando apontar algumas tendências do debate sobre avaliação à época. Em seguida, apresenta, em linhas gerais, ideias que constituem a ideologia produtivista e examina seus desdobramentos na construção de um sistema nacional de avaliação no Brasil. Por último, considera os efeitos desse sistema no trabalho acadêmico, tomando como referência o lugar das exigências de publicação no processo de avaliação.

Pitfalls of artificial grouping and stratification of scientific journals based on their Impact Factor: a case study in Brazilian Zoology

The present contribution explores the impact of the QUALIS metric system for academic evaluation implemented by CAPES (Coordination for the Development of Personnel in Higher Education) upon Brazilian Zoological research. The QUALIS system is based on the grouping and ranking of scientific journals according to their Impact Factor (IF). We examined two main points implied by this system, namely: 1) its reliability as a guideline for authors; 2) if Zoology possesses the same publication profile as Botany and Oceanography, three fields of knowledge grouped by CAPES under the subarea "BOZ" for purposes of evaluation. Additionally, we tested CAPES' recent suggestion that the area of Ecology would represent a fourth field of research compatible with the former three. Our results indicate that this system of classification is inappropriate as a guideline for publication improvement, with approximately one third of the journals changing their strata between years. We also demonstrate that the citation profile of Zoology is distinct from those of Botany and Oceanography. Finally, we show that Ecology shows an IF that is significantly different from those of Botany, Oceanography, and Zoology, and that grouping these fields together would be particularly detrimental to Zoology. We conclude that the use of only one parameter of analysis for the stratification of journals, i.e., the Impact Factor calculated for a comparatively small number of journals, fails to evaluate with accuracy the pattern of publication present in Zoology, Botany, and Oceanography. While such simplified procedure might appeals to our sense of objectivity, it dismisses any real attempt to evaluate with clarity the merit embedded in at least three very distinct aspects of scientific practice, namely: productivity, quality, and specificity.

School Anxiety Inventory: Reliability and Validity Evidence in a Sample of Slovenian Adolescents

The purpose of this study was to examine the reliability and validity of the School Anxiety Inventory (SAI) using a sample of 646 Slovenian adolescents (48% boys), ranging in age from 12 to 19 years. Single confirmatory factor analyses replicated the correlated four-factor structure of scores on the SAI for anxiety-provoking school situations (Anxiety about School Failure and Punishment, Anxiety about Aggression, Anxiety about Social Evaluation, and Anxiety about Academic Evaluation), and the three-factor structure of the anxiety response systems (Physiological Anxiety, Cognitive Anxiety, and Behavioral Anxiety). Equality of factor structures was compared using multigroup confirmatory factor analyses. Measurement invariance for the four- and three-factor models was obtained across gender and school-level samples. The scores of the instrument showed high internal reliability and adequate test–retest reliability. The concurrent validity of the SAI scores was also examined through its relationship with the Social Anxiety Scale for Adolescents (SASA) scores and the Questionnaire about Interpersonal Difficulties for Adolescents (QIDA) scores. Correlations of the SAI scores with scores on the SASA and the QIDA were of low to moderate effect sizes.

Long-Term Evaluation in Aesthetic Rhinoplasty in an Academic Referral Center

Plastic surgery is based on improving esthetic for the patient. In most services, the surgery outcome is evaluated in a subjective manner. Aim: to objectively assess the degree of patient satisfaction one year after rhinoplasty using the Rhinoplasty Outcome Evaluation questionnaire at a referral academic center. Materials and Methods: 69 patients operated in the otorhinolaryngology service were selected. The patients were operated upon by third year residents during the period from January to December 2007 and answered the questionnaire translated by the authors of this study. Results: we obtained a mean value of 73.25% of satisfaction for primary rhinoplasty and a mean value of 72.02% of satisfaction for secondary rhinoplasty. Conclusion: the level of satisfaction presented by the patients was considered to be very good.

Educator Evaluation of Academic and Social Competence in Students with Acquired Brain Injury (ABI) Relative to Assessed Performance and Sense of Belonging

Acquired brain injury (ABI) is the leading cause of death and disability amongst children and adolescents andpresents itself with challenges associated in cognitive, social, emotional, and behavioural domains. These changes may interfere with academic performance and social inclusion, influencing self-esteem and personal success. The current study examined a subset of data to capture the sense of academic and social belonging for students with ABI as a function of the classroom teachers’ subjective perception of ability, their ABI knowledge, and student identification. Overall, a discrepancy was found between educators’ subjective ratings of student performance and students’ neurocognitive capacity. Educator knowledge and identification of ABI influenced student success in academic and social domains independent of teaching approach. This research has implications for the identification of ABI in the classroom and related challenges students experience. Educators are underprepared for the reintegration of students returning to school and lack appropriate knowledge and strategies to accommodate individual needs.

Optimizing the Performance Evaluation Process for Non-Academic Staff

If conducted poorly and without support from all employee levels, performance management programs at institutions may devolve into annual evaluations that represent a staff burden rather than an ongoing career development opportunity. This brief analyzes the key components of successful performance evaluation systems for non-academic staff, incorporating insights from employers outside of the higher education sector as well. It examines the importance of midyear check-in meetings; employee goal-setting; simplified rating scales on evaluation forms; and core staff competencies that reflect institutional priorities. It also describes how institutions communicate modifications to the evaluation system and offers recommendations for the implementation of process change.

The effect of dual -language instruction on academic achievement as measured by the SAT: An evaluation of the "International Studies Program" of Miami -Dade County Public Schools

This study investigated the influence that receiving instruction in two languages, English and Spanish, had on the performance of students enrolled in the International Studies Program (delayed partial immersion model) of Miami Dade County Public Schools on a standardized test in English, the Stanford Achievement Test, eighth edition, for three of its sections, Reading Comprehension, Mathematics Computations, and Mathematics Applications.^ The performance of the selected IS program/Spanish section cohort of students (N = 55) on the SAT Reading Comprehension, Mathematics Computation, and Mathematics Application along four consecutive years was contrasted with that of a control group of comparable students selected within the same feeder pattern where the IS program is implemented (N = 21). The performance of the group was also compared to the cross-sectional achievement patterns of the school's corresponding feeder pattern, region, and district.^ The research model for the study was a variation of the "causal-comparative" or "ex post facto design" sometimes referred to as "prospective". After data were collected from MDCPS, t-tests were performed to compare IS-Spanish students SAT performance for grades 3 to 6 for years 1994 to 1997 to control group, feeder pattern, region and district norms for each year for Reading Comprehension, Mathematics Computation, and Mathematics Applications. Repeated measures ANOVA and Tukey's tests were calculated to compare the mean percentiles of the groups under study and the possible interactions of the different variables. All tests were performed at the 5% significance level.^ From the analyses of the tests it was deduced that the IS group performed significantly better than the control group for all the three measures along the four years. The IS group mean percentiles on the three measures were also significantly higher than those of the feeder pattern, region, and district. The null hypotheses were rejected and it was concluded that receiving instruction in two languages did not negatively affect the performance of IS program students on tests taken in English. It was also concluded that the particular design the IS program enhances the general performance of participant students on Standardized tests.^ The quantitative analyses were coupled with interviews from teachers and administrators of the IS program to gain additional insight about different aspects of the implementation of the program at each particular school. ^

Evaluation of the spouted bed dried leaf extract of Bauhinia forficata for the treatment of experimental diabetes in rats

Previously, we have demonstrated that treatment of experimental diabetes with a decoction of Bauhinia forficata leaves is beneficial. In this study, we prepared a two-fold concentrate of this extract and tested its effects on physiological, biochemical and toxicity markers in streptozotocin-diabetic rats. Dried and ground leaves were extracted with warm 70% hydroethanol and the filtrate concentrated by evaporation at 50 degrees C. This solution was mixed with colloidal silicon dioxide (Tixosil-333 (R)) and dried in a spouted bed (BfT). Rats were treated with water, insulin and Tixosil particles at low or high doses, alone or coated with dried BfT. Animals were periodically weighed and monitored for water and food intake; urinary volume, glucose, urea and protein; blood glucose, serum lipids, liver toxicity markers transaminase and phosphatase and masses of adipose tissue and skeletal muscle. Insulin treatment gave best rat growth and lowest values for all other markers. No other treatment affected any diabetic marker, but the enzyme activities were changed by diabetes and BfT. Thus, BfT toxicity could arise from secondary products of plant constituents or Tixosil interaction. Therefore, BfT prepared in the spouted bed as described, is unsuitable for treatment of diabetes, which implies that the method of preparation of any medicine is critical for its efficacy and toxicity.

Evaluation of the Incidence of Bladder Perforation After Transurethral Bladder Tumor Resection in a Residency Setting

Purpose: To evaluate prospectively the actual bladder perforation incidence during transurethral resection of bladder tumor (TURB) performed by residents and to identify possible predisposing factors to such condition. Patients and Methods: Thirty-four patients with bladder tumor were submitted to TURB in our academic institution in April 2006, and were prospectively studied. Procedures were all done by senior residents under an attending direct supervision. All patients had a cystograms performed after the procedure by the injection of 400 mL of saline-diluted contrast solution with low-pressure infusion through the Foley catheter. The cystograms were evaluated blindly by a single radiologist. All patients were examined by cystoscopy and/or CT every 3 months for the first 2 years postoperatively. Results: The cystogram showed contrast leaking compatible with bladder perforation in 17 (50%) cases. None of the perforations were recognized intraoperatively by the surgeon. All perforations were extraperitoneal and managed conservatively. There was no significant correlation between the incidence of bladder perforation and the patient age (p = 0.508), the tumor stage (p = 0.998), the tumor grade (p = 0.833), the number of lesions (p = 0.394), and the tumor size (p = 0.651). The only factor that had impact on the development of bladder perforation was tumor localization at the bottom of the bladder (p = 0.035; OR, 6750; 95% CI, 1.14, 39.8). Conclusion: Asymptomatic perforations of the bladder wall occur very frequently after a TURB procedure performed by residents in training and, most of the time, are not noticed by the surgeon. Localization of the tumor at bladder dome was the only factor that negatively influenced perforation rates.

Brazilian flavonoid database: Application of quality evaluation system

Much information on flavonoid content of Brazilian foods has already been obtained; however, this information is spread in scientific publications and non-published data. The objectives of this work were to compile and evaluate the quality of national flavonoid data according to the United States Department of Agriculture`s Data Quality Evaluation System (USDA-DQES) with few modifications, for future dissemination in the TBCA-USP (Brazilian Food Composition Database). For the compilation, the most abundant compounds in the flavonoid subclasses were considered (flavonols, flavones, isoflavones, flavanones, flavan-3-ols, and anthocyanidins) and the analysis of the compounds by HPLC was adopted as criteria for data inclusion. The evaluation system considers five categories, and the maximum score assigned to each category is 20. For each data, a confidence code (CC) was attributed (A, B, C and D), indicating the quality and reliability of the information. Flavonoid data (773) present in 197 Brazilian foods were evaluated. The CC ""C"" (as average) was attributed to 99% of the data and ""B"" (above average) to 1%. The main categories assigned low average scores were: number of samples; sampling plan and analytical quality control (average scores 2, 5 and 4, respectively). The analytical method category received an average score of 9. The category assigned the highest score was the sample handling (20 average). These results show that researchers need to be conscious about the importance of the number and plan of evaluated samples and the complete description and documentation of all the processes of methodology execution and analytical quality control. (C) 2010 Elsevier Inc. All rights reserved.

Evaluation of protective effects of fish oil against oxidative damage in rats exposed to methylmercury

The present study evaluates a possible protective effect of fish oil against oxidative damage promoted by methylmercury (MeHg) in sub-chronically exposed rats. Reduced glutathione peroxidase and catalase enzyme activity and reduced glutathione levels were observed in MeHg-exposed animals compared to controls. Methylmercury exposure was also associated with DNA damage. Administration of fish oil to the methylmercury-exposed animals did not ameliorate enzyme activity or glutathione levels. On the other hand, a significant DNA protective effect (about 30%) was observed with fish oil treatment. There were no differences in the total mercury concentration in rat liver, kidney, heart or brain after MeHg administration with or without fish oil co-administration. Histopathological analyses showed a significant leukocyte infiltration in rat tissues after MeHg exposure, but this effect was significantly reduced after co-administration of fish oil. Taken together, our findings demonstrate oxidative damage even after low-level MeHg exposure and the protective effect of fish oil. This protection seems not to be related to antioxidant defenses or mercury re-distribution in rat tissues. It is probably due to the anti-inflammatory effects of fish oil. (C) 2010 Elsevier Inc. All rights reserved.

Corticosterone evaluation in Wistar rats infected with the Y strain of Trypanosoma cruzi during the chronic phase

Understanding the mechanisms responsible for mediating the effects of stress on Trypanosoma cruzi infection is crucial for determining the full impact of stress on Chagas` disease and for devising effective interventions. Dehydroepiandrosterone (DHEA), a steroid hormone synthesized from pregnenolone, is secreted by the adrenal cortex in response to stress. Although its physiologic role has not been fully defined, DHEA has been shown to modulate immune function. In the present study, we evaluated the levels of corticosterone and the ability of T. cruzi infection to modulate the expression of Th2 cytokines in Wistar rats with chronic Chagas` disease submitted to repetitive stress. The animals submitted to stress displayed enhanced levels of corticosterone as compared to control counterparts. Stress and infection triggered the most elevated concentrations of corticosterone. DHEA significantly reduced corticosterone levels for infected and stressed animals with DHEA. The infected animals displayed enhanced levels of IL-10 and IL-4 as compared to control ones. Stress combined with infection triggered the higher levels of IL-10 and IL-4. DHEA alone and combined with infection and stress significantly increased IL-10 and IL-4 levels. Then, this study might provide additional clues about factors that regulate some of the immunoregulatory aspects of T. cruzi infection and might offer new opportunities for therapeutic interventions. (C) 2010 Elsevier Inc. All rights reserved.