Evaluation of research in efficiency and productivity:a survey and analysis of the first 30 years of scholarly literature in DEA

Since the original Data Envelopment Analysis (DEA) study by Charnes et al. [Measuring the efficiency of decision-making units. European Journal of Operational Research 1978;2(6):429–44], there has been rapid and continuous growth in the field. As a result, a considerable amount of published research has appeared, with a significant portion focused on DEA applications of efficiency and productivity in both public and private sector activities. While several bibliographic collections have been reported, a comprehensive listing and analysis of DEA research covering its first 30 years of history is not available. This paper thus presents an extensive, if not nearly complete, listing of DEA research covering theoretical developments as well as “real-world” applications from inception to the year 2007. A listing of the most utilized/relevant journals, a keyword analysis, and selected statistics are presented.

The contextual evaluation of research on sexually explicit materials

This thesis criticises many psychological experiments on 'pornography' which attempt to demonstrate how 'pornography' causes and/or equals rape. It challenges simplistic definitions of 'pornography', arguing that sexually explicit materials (SEM) are constructed and interpreted in a number of different ways; and demonstrates that how, when and where materials are depicted or viewed will influence perceptions and reactions. In addition, it opposes the overreliance on male undergraduates as participants in 'porn' research. Theories of feminist psychology and reflexivity are used throughout the thesis, and provide a detailed contextual framework in a complex area. Results from a number of interlinking studies which use a variety of methodological approaches (focus groups, questionnaires and content analysis), indicate how contextual issues are omitted in much existing research on SEM. These include the views and experiences participants' hold prior to completing SEM studies; their opinions about those who 'use' 'pornography'; their understanding of key terms linked with SEM (eg: pornography and erotica); and discussions of sexual magazines aimed at male and female audiences. Participants' reactions to images and texts associated with SEM presented in different contexts are discussed. Three main conclusions are drawn from this thesis. Firstly, images deemed 'pornographic' differ through historical and cultural periods' and political, economic and social climates, so 'experimental' approaches may not always be the most appropriate research tool. Secondly, there is not one definition, source, or factor which may be named 'pornography'; and thirdly the context and presentation of materials influence how images are perceived and reacted to. The thesis argues a number of factors influence view of 'pornography', suggesting SEM may be 'in the eye of the beholder'.

Handbook of research on user interface design and evaluation for mobile technology

In recent years, mobile technology has been one of the major growth areas in computing. Designing the user interface for mobile applications, however, is a very complex undertaking which is made even more challenging by the rapid technological developments in mobile hardware. Mobile human-computer interaction, unlike desktop-based interaction, must be cognizant of a variety of complex contextual factors affecting both users and technology. The Handbook of Research on User Interface Design and Evaluation provides students, researchers, educators, and practitioners with a compendium of research on the key issues surrounding the design and evaluation of mobile user interfaces, such as the physical environment and social context in which a mobile device is being used and the impact of multitasking behavior typically exhibited by mobile-device users. Compiling the expertise of over 150 leading experts from 26 countries, this exemplary reference tool will make an indispensable addition to every library collection.

Sensemaking tools for understanding research literatures:design, implementation and user evaluation

This paper describes the work undertaken in the Scholarly Ontologies Project. The aim of the project has been to develop a computational approach to support scholarly sensemaking, through interpretation and argumentation, enabling researchers to make claims: to describe and debate their view of a document's key contributions and relationships to the literature. The project has investigated the technicalities and practicalities of capturing conceptual relations, within and between conventional documents in terms of abstract ontological structures. In this way, we have developed a new kind of index to distributed digital library systems. This paper reports a case study undertaken to test the sensemaking tools developed by the Scholarly Ontologies project. The tools used were ClaiMapper, which allows the user to sketch argument maps of individual papers and their connections, ClaiMaker, a server on which such models can be stored and saved, which provides interpretative services to assist the querying of argument maps across multiple papers and ClaimFinder, a novice interface to the search services in ClaiMaker.

Finding qualitative research: an evaluation of search strategies

Background Qualitative research makes an important contribution to our understanding of health and healthcare. However, qualitative evidence can be difficult to search for and identify, and the effectiveness of different types of search strategies is unknown. Methods Three search strategies for qualitative research in the example area of support for breast-feeding were evaluated using six electronic bibliographic databases. The strategies were based on using thesaurus terms, free-text terms and broad-based terms. These strategies were combined with recognised search terms for support for breast-feeding previously used in a Cochrane review. For each strategy, we evaluated the recall (potentially relevant records found) and precision (actually relevant records found). Results A total yield of 7420 potentially relevant records was retrieved by the three strategies combined. Of these, 262 were judged relevant. Using one strategy alone would miss relevant records. The broad-based strategy had the highest recall and the thesaurus strategy the highest precision. Precision was generally poor: 96% of records initially identified as potentially relevant were deemed irrelevant. Searching for qualitative research involves trade-offs between recall and precision. Conclusions These findings confirm that strategies that attempt to maximise the number of potentially relevant records found are likely to result in a large number of false positives. The findings also suggest that a range of search terms is required to optimise searching for qualitative evidence. This underlines the problems of current methods for indexing qualitative research in bibliographic databases and indicates where improvements need to be made.

Strategic supplier performance evaluation:a case-based action research of a UK manufacturing organisation

The main aim of this research is to demonstrate strategic supplier performance evaluation of a UK-based manufacturing organisation using an integrated analytical framework. Developing long term relationship with strategic suppliers is common in today's industry. However, monitoring suppliers' performance all through the contractual period is important in order to ensure overall supply chain performance. Therefore, client organisations need to measure suppliers' performance dynamically and inform them on improvement measures. Although there are many studies introducing innovative supplier performance evaluation frameworks and empirical researches on identifying criteria for supplier evaluation, little has been reported on detailed application of strategic supplier performance evaluation and its implication on overall performance of organisation. Additionally, majority of the prior studies emphasise on lagging factors (quality, delivery schedule and value/cost) for supplier selection and evaluation. This research proposes both leading (organisational practices, risk management, environmental and social practices) and lagging factors for supplier evaluation and demonstrates a systematic method for identifying those factors with the involvement of relevant stakeholders and process mapping. The contribution of this article is a real-life case-based action research utilising an integrated analytical model that combines quality function deployment and the analytic hierarchy process method for suppliers' performance evaluation. The effectiveness of the method has been demonstrated through number of validations (e.g. focus group, business results, and statistical analysis). Additionally, the study reveals that enhanced supplier performance results positive impact on operational and business performance of client organisation.

Research on reliability evaluation of reconfigurable manufacturing system

It is a crucial task to evaluate the reliability of manufacturing process in product development process. Process reliability is a measurement of production ability of reconfigurable manufacturing system (RMS), which serves as an integrated performance indicator of the production process under specified technical constraints, including time, cost and quality. An integration framework of manufacturing process reliability evaluation is presented together with product development process. A mathematical model and algorithm based on universal generating function (UGF) is developed for calculating the reliability of manufacturing process with respect to task intensity and process capacity, which are both independent random variables. The rework strategies of RMS are analyzed under different task intensity based on process reliability is presented, and the optimization of rework strategies based on process reliability is discussed afterwards.

Evaluation of Tottenham Thinking Space Pilot: Final Report. Research Report 11.

This reports summarises research that began in March 2014 and was completed in October 2015 by an experienced inter-disciplinary research team from the Centre for Social Justice and Change and Psycho-Social Research Group, School of Social Sciences, the University of East London (UEL) and included Dr Yang Li from the Centre for Geo-Information Studies, UEL, for the first phase of the study. Tottenham ‘Thinking Space’ (TTS) was a pilot therapeutic initiative based in local communities and delivered by the Tavistock & Portman NHS Foundation Trust and funded by the London Borough of Haringey Directorate of Public Health. TTS aimed to improve mental health and enable and empower local communities. TTS was situated within a mental health agenda that was integral to Haringey’s Health and Wellbeing Strategy 2012-2015 and aimed to encourage people to help themselves and each other and develop confident communities. On the one hand TTS was well-suited to this agenda, but, on the other, participants were resistant to, and were trying to free themselves from labelling that implied ‘mental health difficulties’. A total of 243 meetings were held and 351 people attended 1,716 times. The majority of participants attended four times or less, and 33 people attended between 5 and 10 times and 39 people attended over 10 times. Attending a small number of times does not necessarily mean that the attendee was not helped. Attendees reflected the ethnic diversity of Tottenham; 29 different ethnic groups attended. The opportunity to meet with people from different cultural backgrounds in a safe space was highly valued by attendees. Similarly, participants valued the wide age range represented and felt that they benefited from listening to inter-generational experiences. The majority of participants were women (72%) and they were instrumental in initiating further Thinking Spaces, topic specific meetings, the summer programme of activities for mothers and young children and training to meet their needs. The community development worker had a key role in implementing the initiative and sustaining its growth throughout the pilot period. We observed that TTS attracted those whose life experiences were marked by personal struggle and trauma. Many participants felt safe enough to disclose mental health difficulties (85% of those who completed a questionnaire). Participants also came seeking a stronger sense of community in their local area. Key features of the meetings are that they are democratic, non-judgemental, respectful, and focussed on encouraging everyone to listen and to try to understand. We found that the therapeutic method was put in place by high quality facilitators and health and personal outcomes for participants were consistent with those predicted by the underpinning psychoanalytical and systemic theories. Outcomes included a reduction in anxieties and improved personal and social functioning; approximately two thirds of those who completed a questionnaire felt better understood, felt more motivated and more hopeful for the future. The overwhelming majority of survey respondents also felt good about contributing to their community, said that they were more able to cooperate with others and accepting of other cultures, and had made new friends. Participants typically had a better understanding of their current situation and how to take positive action; of those who completed a questionnaire, over half felt more confident to seek support for a personal issue and to contact services. Members of TTS supported each other and instilled hope and build community-mindedness that reduced social isolation.

Applying the Analytic Hierarchy Process in healthcare research: A systematic literature review and evaluation of reporting

Background: The Analytic Hierarchy Process (AHP), developed by Saaty in the late 1970s, is one of the methods for multi-criteria decision making. The AHP disaggregates a complex decision problem into different hierarchical levels. The weight for each criterion and alternative are judged in pairwise comparisons and priorities are calculated by the Eigenvector method. The slowly increasing application of the AHP was the motivation for this study to explore the current state of its methodology in the healthcare context. Methods: A systematic literature review was conducted by searching the Pubmed and Web of Science databases for articles with the following keywords in their titles or abstracts: "Analytic Hierarchy Process," "Analytical Hierarchy Process," "multi-criteria decision analysis," "multiple criteria decision," "stated preference," and "pairwise comparison." In addition, we developed reporting criteria to indicate whether the authors reported important aspects and evaluated the resulting studies' reporting. Results: The systematic review resulted in 121 articles. The number of studies applying AHP has increased since 2005. Most studies were from Asia (almost 30 %), followed by the US (25.6 %). On average, the studies used 19.64 criteria throughout their hierarchical levels. Furthermore, we restricted a detailed analysis to those articles published within the last 5 years (n = 69). The mean of participants in these studies were 109, whereas we identified major differences in how the surveys were conducted. The evaluation of reporting showed that the mean of reported elements was about 6.75 out of 10. Thus, 12 out of 69 studies reported less than half of the criteria. Conclusion: The AHP has been applied inconsistently in healthcare research. A minority of studies described all the relevant aspects. Thus, the statements in this review may be biased, as they are restricted to the information available in the papers. Hence, further research is required to discover who should be interviewed and how, how inconsistent answers should be dealt with, and how the outcome and stability of the results should be presented. In addition, we need new insights to determine which target group can best handle the challenges of the AHP. © 2015 Schmidt et al.