Making sense of research quality assessment

Purpose – For many academics in UK universities the nature and orientation of their research is overwhelmingly determined by considerations of how that work will be graded in research assessment exercises (RAEs). The grades awarded to work in a particular subject area can have a considerable impact on the individual and their university. There is a need to better understand those factors which may influence these grades. The paper seeks to address this issue. Design/methodology/approach – The paper considers relationships between the grades awarded and the quantitative information provided to the assessment panels for the 1996 and 2001 RAEs for two subject areas, built environment and town and country planning, and for three other subject areas, civil engineering, geography and archaeology, in the 2001 RAE. Findings – A simple model demonstrating strong and consistent relationships is established. RAE performance relates to numbers of research active staff, the production of books and journal papers, numbers of research studentships and graduations, and research income. Important differences between subject areas are identified. Research limitations/implications – Important issues are raised about the extent to which the new assessment methodology to be adopted for the 2008 RAE will capture the essence of good quality research in architecture and built environment. Originality/value – The findings provide a developmental perspective of RAEs and show how, despite a changed methodology, various research activities might be valued in the 2008 RAE. The basis for a methodology for reviewing the credibility of the judgements of panels is proposed.

Truth - the essential dimension of ethics in scholarship, teaching, learning and research

The introduction of metrics, league tables, performance targets, research assessment exercises and a range of other pressures placed by society, funding bodies and employers on scholars, teachers and students have resulted in diminished value being placed on the essential ethical criterion of truth. The impact of reduced valuation for truth has a huge impact on the standing of science and not least horticultural science in the eyes of the general public at a time when this should be a primary concern. This contribution discusses examples of the impact of diminished valuation of truth, the causes of this phenomenon, the results that come from this situation and remedies that are needed.

Aberrant brain responses to emotionally valent words is normalised after cognitive behavioural therapy in female depressed adolescents

AbstractBackground Depression in adolescence is debilitating with high recurrence in adulthood, yet its pathophysiological mechanism remains enigmatic. To examine the interaction between emotion, cognition and treatment, functional brain responses to sad and happy distractors in an affective go/no-go task were explored before and after Cognitive Behavioural Therapy (CBT) in depressed female adolescents, and healthy participants. Methods Eighty-two Depressed and 24 healthy female adolescents, aged 12 to 17 years, performed a functional magnetic resonance imaging (fMRI) affective go/no-go task at baseline. Participants were instructed to withhold their responses upon seeing happy or sad words. Among these participants, 13 patients had CBT over approximately 30 weeks. These participants and 20 matched controls then repeated the task. Results At baseline, increased activation in response to happy relative to neutral distractors was observed in the orbitofrontal cortex in depressed patients which was normalized after CBT. No significant group differences were found behaviourally or in brain activation in response to sad distractors. Improvements in symptoms (mean: 9.31, 95% CI: 5.35-13.27) were related at trend-level to activation changes in orbitofrontal cortex. Limitations In the follow-up section, a limited number of post-CBT patients were recruited. Conclusions To our knowledge, this is the first fMRI study addressing the effect of CBT in adolescent depression. Although a bias toward negative information is widely accepted as a hallmark of depression, aberrant brain hyperactivity to positive distractors was found and normalised after CBT. Research, assessment and treatment focused on positive stimuli could be a future consideration. Moreover, a pathophysiological mechanism distinct from adult depression may be suggested and awaits further exploration.

Human exposure modelling for chemical risk assessment: a review of current approaches and research and policy implications

A wide variety of exposure models are currently employed for health risk assessments. Individual models have been developed to meet the chemical exposure assessment needs of Government, industry and academia. These existing exposure models can be broadly categorised according to the following types of exposure source: environmental, dietary, consumer product, occupational, and aggregate and cumulative. Aggregate exposure models consider multiple exposure pathways, while cumulative models consider multiple chemicals. In this paper each of these basic types of exposure model are briefly described, along with any inherent strengths or weaknesses, with the UK as a case study. Examples are given of specific exposure models that are currently used, or that have the potential for future use, and key differences in modelling approaches adopted are discussed. The use of exposure models is currently fragmentary in nature. Specific organisations with exposure assessment responsibilities tend to use a limited range of models. The modelling techniques adopted in current exposure models have evolved along distinct lines for the various types of source. In fact different organisations may be using different models for very similar exposure assessment situations. This lack of consistency between exposure modelling practices can make understanding the exposure assessment process more complex, can lead to inconsistency between organisations in how critical modelling issues are addressed (e.g. variability and uncertainty), and has the potential to communicate mixed messages to the general public. Further work should be conducted to integrate the various approaches and models, where possible and regulatory remits allow, to get a coherent and consistent exposure modelling process. We recommend the development of an overall framework for exposure and risk assessment with common approaches and methodology, a screening tool for exposure assessment, collection of better input data, probabilistic modelling, validation of model input and output and a closer working relationship between scientists and policy makers and staff from different Government departments. A much increased effort is required is required in the UK to address these issues. The result will be a more robust, transparent, valid and more comparable exposure and risk assessment process. (C) 2006 Elsevier Ltd. All rights reserved.

A proposal for a new scenario framework to support research and assessment in different climate research communities

In this paper, we propose a scenario framework that could provide a scenario “thread” through the different climate research communities (climate change – vulnerability, impact, and adaptation (VIA) and mitigation) in order to provide assessment of mitigation and adaptation strategies and other VIA challenges. The scenario framework is organised around a matrix with two main axes: radiative forcing levels and socio-economic conditions. The radiative forcing levels (and the associated climate signal) are described by the new Representative Concentration Pathways. The second axis, socio-economic developments, comprises elements that affect the capacity for mitigation and adaptation, as well as the exposure to climate impacts. The proposed scenarios derived from this framework are limited in number, allow for comparison across various mitigation and adaptation levels, address a range of vulnerability characteristics, provide information across climate forcing and vulnerability states and span a full century time scale. Assessments based on the proposed scenario framework would strengthen cooperation between integrated-assessment modelers, climate modelers and vulnerability, impact and adaptation researchers, and most importantly, facilitate the development of more consistent and comparable research within and across communities.

The next generation of scenarios for climate change research and assessment

Advances in the science and observation of climate change are providing a clearer understanding of the inherent variability of Earth’s climate system and its likely response to human and natural influences. The implications of climate change for the environment and society will depend not only on the response of the Earth system to changes in radiative forcings, but also on how humankind responds through changes in technology, economies, lifestyle and policy. Extensive uncertainties exist in future forcings of and responses to climate change, necessitating the use of scenarios of the future to explore the potential consequences of different response options. To date, such scenarios have not adequately examined crucial possibilities, such as climate change mitigation and adaptation, and have relied on research processes that slowed the exchange of information among physical, biological and social scientists. Here we describe a new process for creating plausible scenarios to investigate some of the most challenging and important questions about climate change confronting the global community

A Tropospheric Assessment of the ERA-40, NCEP, and JRA-25 Global Reanalyses in the Polar Regions

The reliability of the global reanalyses in the polar regions is investigated. The overview stems from an April 2006 Scientific Committee on Antarctic Research (SCAR) workshop on the performance of global reanalyses in high latitudes held at the British Antarctic Survey. Overall, the skill is much higher in the Arctic than the Antarctic, where the reanalyses are only reliable in the summer months prior to the modern satellite era. In the Antarctic, large circulation differences between the reanalyses are found primarily before 1979, when vast quantities of satellite sounding data started to be assimilated. Specifically for ERA-40, this data discontinuity creates a marked jump in Antarctic snow accumulation, especially at high elevations. In the Arctic, the largest differences are related to the reanalyses depiction of clouds and their associated radiation impacts; ERA-40 captures the cloud variability much better than NCEP1 and JRA-25, but the ERA-40 and JRA-25 clouds are too optically thin for shortwave radiation. To further contrast the reanalyses skill, cyclone tracking results are presented. In the Southern Hemisphere, cyclonic activity is markedly different between the reanalyses, where there are few matched cyclones prior to 1979. In comparison, only some of the weaker cyclones are not matched in the Northern Hemisphere from 1958-2001, again indicating the superior skill in this hemisphere. Although this manuscript focuses on deficiencies in the reanalyses, it is important to note that they are a powerful tool for climate studies in both polar regions when used with a recognition of their limitations.

Dynamical downscaling: Assessment of model system dependent retained and added variability for two different regional climate models

Uncertainty assessment of a process-based integrated catchment model of phosphorus

Despite the many models developed for phosphorus concentration prediction at differing spatial and temporal scales, there has been little effort to quantify uncertainty in their predictions. Model prediction uncertainty quantification is desirable, for informed decision-making in river-systems management. An uncertainty analysis of the process-based model, integrated catchment model of phosphorus (INCA-P), within the generalised likelihood uncertainty estimation (GLUE) framework is presented. The framework is applied to the Lugg catchment (1,077 km2), a River Wye tributary, on the England–Wales border. Daily discharge and monthly phosphorus (total reactive and total), for a limited number of reaches, are used to initially assess uncertainty and sensitivity of 44 model parameters, identified as being most important for discharge and phosphorus predictions. This study demonstrates that parameter homogeneity assumptions (spatial heterogeneity is treated as land use type fractional areas) can achieve higher model fits, than a previous expertly calibrated parameter set. The model is capable of reproducing the hydrology, but a threshold Nash-Sutcliffe co-efficient of determination (E or R 2) of 0.3 is not achieved when simulating observed total phosphorus (TP) data in the upland reaches or total reactive phosphorus (TRP) in any reach. Despite this, the model reproduces the general dynamics of TP and TRP, in point source dominated lower reaches. This paper discusses why this application of INCA-P fails to find any parameter sets, which simultaneously describe all observed data acceptably. The discussion focuses on uncertainty of readily available input data, and whether such process-based models should be used when there isn’t sufficient data to support the many parameters.

Glacier shrinkage and climatic change in the Russian Altai from the mid-20th century: An assessment using remote sensing and PRECIS regional climate model

This paper examines changes in the surface area of glaciers in the North and South Chuya Ridges, Altai Mountains in 1952-2004 and their links with regional climatic variations. The glacier surface areas for 2004 were derived from the Advanced Spaceborne Thermal Emission and Reflection Radiometer (ASTER) imagery. Data from the World Glacier Inventory (WGI)dating to 1952 and aerial photographs from 1952 were used to estimate the changes. 256 glaciers with a combined area of 253±5.1 km2 have been identified in the region in 2004. Estimation of changes in extent of 126 glaciers with the individual areas not less than 0.5 km2 in 1952 revealed a 19.7±5.8% reduction. The observed glacier retreat is primarily driven by an increase in summer temperatures since the 1980s when air temperatures were increasing at a rate of 0.10 - 0.13oC a-1 at the glacier tongue elevation. The regional climate projections for A2 and B2 CO2 emission scenarios developed using PRECIS regional climate model indicate that summer temperatures will increase in the Altai in 2071-2100 by 6-7oC and 3-5oC respectively in comparison with 1961-1990 while annual precipitation will increase by 15% and 5%. The length of the ablation season will extend from June-August to the late April – early October. The projected increases in precipitation will not compensate for the projected warming and glaciers will continue to retreat in the 21st century under both B2 and A2 scenarios.

Assessment of the consistency of near-infrared water vapor line intensities using high-spectral-resolution ground-based Fourier transform measurements of solar radiation

Calculations of the absorption of solar radiation by atmospheric gases, and water vapor in particular, are dependent on the quality of databases of spectral line parameters. There has been increasing scrutiny of databases such as HITRAN in recent years, but this has mostly been performed on a band-by-band basis. We report nine high-spectral-resolution (0.03 cm(-1)) measurements of the solar radiation reaching the surface in southern England over the wave number range 2000 to 12,500 cm(-1) (0.8 to 5 mm) that allow a unique assessment of the consistency of the spectral line databases over this entire spectral region. The data are assessed in terms of the modeled water vapor column that is required to bring calculations and observations into agreement; for an entirely consistent database, this water vapor column should be constant with frequency. For the HITRAN01 database, the spread in water vapor column is about 11%, with distinct shifts between different spectral regions. The HITRAN04 database is in significantly better agreement (about 5% spread) in the completely updated 3000 to 8000 cm(-1) spectral region, but inconsistencies between individual spectral regions remain: for example, in the 8000 to 9500 cm(-1) spectral region, the results indicate an 18% (+/- 1%) underestimate in line intensities with respect to the 3000 to 8000 cm(-1) region. These measurements also indicate the impact of isotopic fractionation of water vapor in the 2500 to 2900 cm(-1) range, where HDO lines dominate over the lines of the most abundant isotope of H2O.

Uptake of a free coronary heart disease risk screening programme in community pharmacy: the Healthy Heart Assessment service

Introduction Health promotion (HP) aims to enhance good health while preventing ill-health at three levels of activity; primary (preventative), secondary (diagnostic) and tertiary (management).1 It can range from simple provision of health education to ongoing support, but the effectiveness of HP is ultimately dependent on its ability to influence change. HP as part of the Community Pharmacy Contract (CPC) aims to increase public knowledge and target ‘hard-to-reach’ individuals by focusing mainly on primary and tertiary HP. The CPC does not include screening programmes (secondary HP) as a service. Coronary heart disease (CHD) is a significant cause of morbidity and mortality in the UK. While there is evidence to support the effectiveness of some community pharmacy HP strategies in CHD, there is paucity of research in relation to screening services.2 Against this background, Alliance Pharmacy introduced a free CHD risk screening programme to provide tailored HP advice as part of a participant–pharmacist consultation. The aim of this study is to report on the CHD risk levels of participants and to provide a qualitative indication of consultation outcomes. Methods Case records for 12 733 people who accessed a free CHD risk screening service between August 2004 and April 2006 offered at 217 community pharmacies were obtained. The service involved initial self-completion of the Healthy Heart Assessment (HHA) form and measurement of height, weight, body mass index, blood pressure, total cholesterol and highdensity lipoprotein levels by pharmacists to calculate CHD risk.3 Action taken by pharmacists (lifestyle advice, statin recommendation or general practitioner (GP) referral) and qualitative statements of advice were recorded, and a copy provided to the participants. The service did not include follow-up of participants. All participants consented to taking part in evaluations of the service. Ethical committee scrutiny was not required for this service development evaluation. Results Case records for 10 035 participants (3658 male) were evaluable; 5730 (57%) were at low CHD risk (<15%); 3636 (36%) at moderate-to-high CHD risk (≥15%); and 669 (7%) had existing heart disease. A significantly higher proportion of male (48% versus 30% female) participants were at moderate- to-high risk of CHD (chi-square test; P < 0.005). A range of outcomes resulted from consultations. Lifestyle advice was provided irrespective of participants’ CHD risk or existing disease. In the moderate-to-high-risk group, of which 52% received prescribed medication, lifestyle advice was recorded for 62%, 16% were referred and 34% were advised to have a re-assessment. Statin recommendations were made in 1% of all cases. There was evidence of supportive and motivational statements in the advice recorded. Discussion Pharmacists were able to identify individuals’ level of CHD risk and provide them with bespoke advice. Identification of at-risk participants did not automatically result in referrals or statin recommendation. One-third of those accessing the screening service had moderate-to-high risk of CHD, a significantly higher proportion of whom were men. It is not known whether these individuals had been previously exposed to HP but presumably by accessing this service they may have contemplated change. As effectiveness of HP advice will depend among other factors on ability to influence change, future consultations may need to explore patients’ attitude towards change in relation to the Trans Theoretical Model4 to better tailor HP advice. The high uptake of the service by those at moderate-to-high CHD risk indicates a need for this type of screening programme in community pharmacy, perhaps specifically to reach men who access medical services less.