Making sense of research quality assessment

Purpose – For many academics in UK universities the nature and orientation of their research is overwhelmingly determined by considerations of how that work will be graded in research assessment exercises (RAEs). The grades awarded to work in a particular subject area can have a considerable impact on the individual and their university. There is a need to better understand those factors which may influence these grades. The paper seeks to address this issue. Design/methodology/approach – The paper considers relationships between the grades awarded and the quantitative information provided to the assessment panels for the 1996 and 2001 RAEs for two subject areas, built environment and town and country planning, and for three other subject areas, civil engineering, geography and archaeology, in the 2001 RAE. Findings – A simple model demonstrating strong and consistent relationships is established. RAE performance relates to numbers of research active staff, the production of books and journal papers, numbers of research studentships and graduations, and research income. Important differences between subject areas are identified. Research limitations/implications – Important issues are raised about the extent to which the new assessment methodology to be adopted for the 2008 RAE will capture the essence of good quality research in architecture and built environment. Originality/value – The findings provide a developmental perspective of RAEs and show how, despite a changed methodology, various research activities might be valued in the 2008 RAE. The basis for a methodology for reviewing the credibility of the judgements of panels is proposed.

Biomarkers of the intake of dietary polyphenols: strengths, limitations and application in nutrition research

In order to establish firm evidence for the health effects of dietary polyphenol consumption, it is essential to have quantitative information regarding their dietary intake. The usefulness of the current methods, which rely mainly on the assessment of polyphenol intake using food records and food composition tables, is limited as they fail to assess total intake accurately. This review highlights the problems associated with such methods with regard to polyphenol-intake predictions. We suggest that the development of biological biomarkers, measured in both blood and urine, are essential for making accurate estimates of polyphenol intake. However, the relationship between dietary intakes and nutritional biomarkers are often highly complex. This review identifies the criteria that must be considered in the development of such biomarkers. In addition, we provide an assessment of the limited number of potential biomarkers of polyphenol intake currently available.

Modelling bi-directional research: a fresh approach to stakeholder theory

Purpose – While Freeman's stakeholder management approach has attracted much attention from both scholars and practitioners, little empirical work has considered the interconnectedness of organisational perspectives and stakeholder perspectives. The purpose of this paper is to respond to this gap by developing and empirically testing a bi-directional model of organisation/stakeholder relationships. Design/methodology/approach – A conceptual framework is developed that integrates how stakeholders are affected by organisations with how they affect organisations. Quantitative data relating to both sides of the relationship are obtained from 700 customers of a European service organisation and analysed using partial least squares structural equation modelling technique. Findings – The findings provide empirical support for the notion of mutual dependency between organisations and stakeholders as advocated by stakeholder theorists. The results suggest that the way stakeholders relate to organisations is dependent on how organisations relate to stakeholders. Originality/value – The study is original on two fronts: first, it provides a framework and process that can be used by researchers to model bi-directional research with other stakeholder groups and in different contexts. Second, the study presents an example application of bi-directional research by empirically linking organisational and stakeholder expectations in the case of customers of a UK service organisation.

Appraiser behaviour and appraisal smoothing: some qualitative and quantitative evidence

There is a substantial literature which suggests that appraisals are smoothed and lag the true level of prices. This study combines a qualitative interview survey of the leading fund manager/owners in the UK and their appraisers with a empirical study of the number of appraisals which change each month within the IPD Monthly Index. The paper concentrates on how the appraisal process operates for commercial property performance measurement purposes. The survey interviews suggest that periodic appraisal services are consolidating in fewer firms and, within these major firms, appraisers adopt different approaches to changing appraisals on a period by period basis, with some wanting hard transaction evidence while others act on "softer' signals. The survey also indicates a seasonal effect with greater effort and information being applied to annual and quarterly appraisals than monthly. The analysis of the appraisals within the Investment Property Databank Monthly Index confirms this effect with around 5% more appraisals being moved at each quarter day than the other months. January and August have significantly less appraisal changes than other months.

Quantifying the impact and relevance of scientific research

Qualitative and quantitative methods are being developed to measure the impacts of research on society, but they suffer from serious drawbacks associated with linking a piece of research to its subsequent impacts. We have developed a method to derive impact scores for individual research publications according to their contribution to answering questions of quantified importance to end users of research. To demonstrate the approach, here we evaluate the impacts of research into means of conserving wild bee populations in the UK. For published papers, there is a weak positive correlation between our impact score and the impact factor of the journal. The process identifies publications that provide high quality evidence relating to issues of strong concern. It can also be used to set future research agendas.

Memo to marketers: quantitative evidence for change: how user-generated content really affects brands

Developed in response to the new challenges of the social Web, this study investigates how involvement with brand-related user-generated content (UGC) affects consumers’ perceptions of brands. The authors develop a model that provides new insights into the links between drivers of UGC creation, involvement, and consumer-based brand equity. Expert opinions were sought on a hypothesized model, which further was tested through data from an online survey of 202 consumers. The results provide guidance for managerial initiatives involving UGC campaigns for brand building. The findings indicate that consumer perceptions of co-creation, community, and self-concept have a positive impact on UGC involvement that, in turn, positively affects consumer based brand equity. These empirical results have significant implications for avoiding problems and building deeper relationships between consumers and brands in the age of social media.

Extracting quantitative structural parameters for disordered polymers from neutron scattering data

The organization of non-crystalline polymeric materials at a local level, namely on a spatial scale between a few and 100 a, is still unclear in many respects. The determination of the local structure in terms of the configuration and conformation of the polymer chain and of the packing characteristics of the chain in the bulk material represents a challenging problem. Data from wide-angle diffraction experiments are very difficult to interpret due to the very large amount of information that they carry, that is the large number of correlations present in the diffraction patterns.We describe new approaches that permit a detailed analysis of the complex neutron diffraction patterns characterizing polymer melts and glasses. The coupling of different computer modelling strategies with neutron scattering data over a wide Q range allows the extraction of detailed quantitative information on the structural arrangements of the materials of interest. Proceeding from modelling routes as diverse as force field calculations, single-chain modelling and reverse Monte Carlo, we show the successes and pitfalls of each approach in describing model systems, which illustrate the need to attack the data analysis problem simultaneously from several fronts.

A flow system for the on-line quantitative measurement of the retention of dosage forms on biological surfaces using spectroscopy and image analysis

Measuring the retention, or residence time, of dosage forms to biological tissue is commonly a qualitative measurement, where no real values to describe the retention can be recorded. The result of this is an assessment that is dependent upon a user's interpretation of visual observation. This research paper outlines the development of a methodology to quantitatively measure, both by image analysis and by spectrophotometric techniques, the retention of material to biological tissues, using the retention of polymer solutions to ocular tissue as an example. Both methods have been shown to be repeatable, with the spectrophotometric measurement generating data reliably and quickly for further analysis.

Evaluation and comparison of SYBR Green I Real-Time PCR and TaqMan Real-Time PCR methods for quantitative assay of Listeria monocytogenes in nutrient broth and milk

Specific traditional plate count method and real-time PCR systems based on SYBR Green I and TaqMan technologies using a specific primer pair and probe for amplification of iap-gene were used for quantitative assay of Listeria monocytogenes in seven decimal serial dilution series of nutrient broth and milk samples containing 1.58 to 1.58×107 cfu /ml and the real-time PCR methods were compared with the plate count method with respect to accuracy and sensitivity. In this study, the plate count method was performed using surface-plating of 0.1 ml of each sample on Palcam Agar. The lowest detectable level for this method was 1.58×10 cfu/ml for both nutrient broth and milk samples. Using purified DNA as a template for generation of standard curves, as few as four copies of the iap-gene could be detected per reaction with both real-time PCR assays, indicating that they were highly sensitive. When these real-time PCR assays were applied to quantification of L. monocytogenes in decimal serial dilution series of nutrient broth and milk samples, 3.16×10 to 3.16×105 copies per reaction (equals to 1.58×103 to 1.58×107 cfu/ml L. monocytogenes) were detectable. As logarithmic cycles, for Plate Count and both molecular assays, the quantitative results of the detectable steps were similar to the inoculation levels.

Teaching English to speakers of other languages teachers’ views on the relationship between research and practice

This paper reports on a study investigating teachers’ views and beliefs about the relationship between second language (L2) research and practice. Although a gap has been frequently reported between the two, there is little empirical data to show what teachers’ views on this relationship are or how these views and beliefs influence their use of research. A total of 60 TESOL1 teachers in England responded to a questionnaire which sought both qualitative and quantitative data. Results of the data analysis suggest that although their views on research and its usefulness are positive, teachers are mainly sceptical about the practicality and relevance of L2 research. More importantly, they expect research to originate from rather than end in classrooms and maintain that the prime responsibility of bringing research and practice together is to be shared by teacher training programmes and educational policies of the institutions they work in. Our analysis of the data further implies that there are differences between teachers’ epistemological assumptions and the more established notions of research.

Handbook of research methods and applications in empirical finance

The objective of this book is to present the quantitative techniques that are commonly employed in empirical finance research together with real world, state of the art research examples. Each chapter is written by international experts in their fields. The unique approach is to describe a question or issue in finance and then to demonstrate the methodologies that may be used to solve it. All of the techniques described are used to address real problems rather than being presented for their own sake and the areas of application have been carefully selected so that a broad range of methodological approaches can be covered. This book is aimed primarily at doctoral researchers and academics who are engaged in conducting original empirical research in finance. In addition, the book will be useful to researchers in the financial markets and also advanced Masters-level students who are writing dissertations.

Participatory research approaches in the development of improved management practices in indigenous chickens production systems with smallholder farmers in Kenya

This thesis is concerned with development of improved management practices in indigenous chicken production systems in a research process that includes participatory approaches with smallholder farmers and other stakeholders in Kenya. The research process involved a wide range of activities that included on-station experiments, field surveys, stakeholder consultations in workshops, seminars and visits, and on-farm farmer participatory research to evaluate the effect of some improved management interventions on production performance of indigenous chickens. The participatory research was greatly informed from collective experiences and lessons of the previous activities. The on-station studies focused on hatching, growth and nutritional characteristics of the indigenous chickens. Four research publications from these studies are included in this thesis. Quantitative statistical analyses were applied and they involved use of growth models estimated with non-linear regressions for the growth characteristics, chi-square determinations to investigate differences among different reciprocal crosses of indigenous chickens and general linear models and covariance determination for the nutrition study. The on-station studies brought greater understanding of performance and production characteristics of indigenous chickens and the influence of management practices on these characteristics. The field surveys and stakeholder consultations helped in understanding the overarching issues affecting the productivity of the indigenous chickens systems and their place in the livelihoods of smallholder farmers. These activities created strong networking opportunities with stakeholders from a wide spectrum. The on-farm farmer participatory research involved selection of 200 farmers in five regions followed by training and introduction of interventions on improved management practices which included housing, vaccination, deworming and feed supplementation. Implementation and monitoring was mainly done by individual farmers continuously for close to one and half years. Six quarterly visits to the farms were made by the research team to monitor and provide support for on-going project activities. The data collected has been analysed for 5 consecutive 3-monthly periods. Descriptive and inferential statistics were applied to analyse the data collected involving treatment applications, production characteristics and flock demography characteristics. Out of the 200 farmers initially selected, 173 had records on treatment applications and flock demography characteristics while 127 farmers had records on production characteristics. The demographic analysis with a dissimilarity index of flock size produced 7 distinct farm groups from among the 173 farms. Two of these farm groups were represented in similar numbers in each of the five regions. The research process also involved a number of dissemination and communication strategies that have brought the process and project outcomes into the domain of accessibility by wider readership locally and globally. These include workshops, seminars, field visits and consultations, local and international conferences, electronic conferencing, publications and personal communication via emailing and conventional posting. A number of research and development proposals were also developed based on the knowledge and experiences gained from the research process. The thesis captures the research process activities and outcomes in 8 chapters which include in ascending order – introduction, theoretical concepts underpinning FPR, research methodology and process, on-station research output, FPR descriptive statistical analysis, FPR inferential statistical analysis on production characteristics, FPR demographic analysis and conclusions. Various research approaches both quantitative and qualitative have been applied in the research process indicating the possibilities and importance of combining both systems for greater understanding of issues being studied. In our case, participatory studies of the improved management of indigenous chickens indicates their potential importance as livelihood assets for poor people.

A method to assess individual research outputs

We present here a straightforward method which can be used to obtain a quantitative indication of an individual research output for an academic. Different versions, selections and options are presented to enable a user to easily calculate values both for stand-alone papers and overall for the collection of outputs for a person. The procedure is particularly useful as a metric to give a quantitative indication of the research output of a person over a time window. Examples are included to show how the method works in practice and how it compares to alternative techniques.