Lean supply chain performance evaluation method

Increasing global competitiveness worldwide has forced manufacturing organizations to produce high-quality products more quickly and at a competitive cost which demand of continuous improvements techniques. In this paper, we propose a fuzzy based performance evaluation method for lean supply chain. To understand the overall performance of cost competitive supply chain, we investigate the alignment of market strategy and position of the supply chain. Competitive strategies can be achieved by using a different weight calculation for different supply chain situations. By identifying optimal performance metrics and applying performance evaluation methods, managers can predict the overall supply chain performance under lean strategy.

Meta-evaluation to improve learning, evaluation capacity development and sustainability : Findings from a participatory evaluation project in Nepal

Recent studies suggest that meta-evaluation can be valuable in developing new approaches to evaluation, building evaluation capacities, and enhancing organizational learning. These new extensions of the concept of meta-evaluation are significant, given the growing emphasis on improving the quality and effectiveness of evaluation practices in the South Asian region. Following a review of the literature, this paper presents a case study of the use of concurrent meta-evaluation in the four-year project Assessing Communication for Social Change which developed and trialled a participatory impact assessment methodology in collaboration with a development communication Non-government organization (NGO) in Nepal. Key objectives of the meta-evaluation included to: continuously develop, adapt and improve the impact assessment methodology, Monitoring and Evaluation (M&E) systems and process and other project activities; identify impacts of the project; and build capacities in critical reflection and review. Our analysis indicates that this meta-evaluation was essential to understanding various constraints related to the organizational context that affected the success of the project and the development of improved M&E systems and capacities within the NGO. We identified several limitations of our meta-evaluation methods, which were balanced by the strengths of other methods. Our case study suggests that as well as assessing the quality, credibility and value of evaluation practices, meta-evaluations need to focus on important contextual issues that can have significant impacts on the outcomes of participatory evaluation projects. They include hierarchical organizational cultures, communication barriers, power/knowledge relations, and the time and resources available. Meta-evaluations also need to consider wider issues such as the sustainability of evaluation systems and approaches.

Cost effectiveness of patient education for the prevention of falls in hospital : economic evaluation from a randomized controlled trial

Background Falls are one of the most frequently occurring adverse events that impact upon the recovery of older hospital inpatients. Falls can threaten both immediate and longer-term health and independence. There is need to identify cost-effective means for preventing falls in hospitals. Hospital-based falls prevention interventions tested in randomized trials have not yet been subjected to economic evaluation. Methods Incremental cost-effectiveness analysis was undertaken from the health service provider perspective, over the period of hospitalization (time horizon) using the Australian Dollar (A$) at 2008 values. Analyses were based on data from a randomized trial among n = 1,206 acute and rehabilitation inpatients. Decision tree modeling with three-way sensitivity analyses were conducted using burden of disease estimates developed from trial data and previous research. The intervention was a multimedia patient education program provided with trained health professional follow-up shown to reduce falls among cognitively intact hospital patients. Results The short-term cost to a health service of one cognitively intact patient being a faller could be as high as A$14,591 (2008). The education program cost A$526 (2008) to prevent one cognitively intact patient becoming a faller and A$294 (2008) to prevent one fall based on primary trial data. These estimates were unstable due to high variability in the hospital costs accrued by individual patients involved in the trial. There was a 52% probability the complete program was both more effective and less costly (from the health service perspective) than providing usual care alone. Decision tree modeling sensitivity analyses identified that when provided in real life contexts, the program would be both more effective in preventing falls among cognitively intact inpatients and cost saving where the proportion of these patients who would otherwise fall under usual care conditions is at least 4.0%. Conclusions This economic evaluation was designed to assist health care providers decide in what circumstances this intervention should be provided. If the proportion of cognitively intact patients falling on a ward under usual care conditions is 4% or greater, then provision of the complete program in addition to usual care will likely both prevent falls and reduce costs for a health service.

Course evaluation : reconfigurations for learning with learning management systems

The introduction of online delivery platforms such as learning management systems (LMS) in tertiary education has changed the methods and modes of curriculum delivery and communication. While course evaluation methods have also changed from paper-based in-class-administered methods to largely online-administered methods, the data collection instruments have remained unchanged. This paper reports on a small exploratory study of two tertiary-level courses. The study investigated why design of the instruments and methods to administer surveys in the courses are ineffective measures against the intrinsic characteristics of online learning. It reviewed the students' response rates of the conventional evaluations for the courses over an eight-year period. It then compared a newly developed online evaluation and the conventional methods over a two-year period. The results showed the response rates with the new evaluation method increased by more than 80% from the average of the conventional evaluations (below 30%), and the students' written feedback was more detailed and comprehensive than in the conventional evaluations. The study demonstrated the possibility that the LMS-based learning evaluation can be effective and efficient in terms of the quality of students' participation and engagement in their learning, and for an integrated pedagogical approach in an online learning environment.

Evaluation of driving ability of the disabled persons in the context of the psychological activity theory

In the future the number of the disabled drivers requiring a special evaluation of their driving ability will increase due to the ageing population, as well as the progress of adaptive technology. This places pressure on the development of the driving evaluation system. Despite quite intensive research there is still no consensus concerning what is the factual situation in a driver evaluation (methodology), which measures should be included in an evaluation (methods), and how an evaluation has to be carried out (practise). In order to find answers to these questions we carried out empirical studies, and simultaneously elaborated upon a conceptual model for driving and a driving evaluation. The findings of empirical studies can be condensed into the following points: 1) A driving ability defined by the on-road driving test is associated with different laboratory measures depending on the study groups. Faults in the laboratory tests predicted faults in the on-road driving test in the novice group, whereas slowness in the laboratory predicted driving faults in the experienced drivers group. 2) The Parkinson study clearly showed that even an experienced clinician cannot reliably accomplish an evaluation of a disabled person’s driving ability without collaboration with other specialists. 3) The main finding of the stroke study was that the use of a multidisciplinary team as a source of information harmonises the specialists’ evaluations. 4) The patient studies demonstrated that the disabled persons themselves, as well as their spouses, are as a rule not reliable evaluators. 5) From the safety point of view, perceptible operations with the control devices are not crucial, but correct mental actions which the driver carries out with the help of the control devices are of greatest importance. 6) Personality factors including higher-order needs and motives, attitudes and a degree of self-awareness, particularly a sense of illness, are decisive when evaluating a disabled person’s driving ability. Personality is also the main source of resources concerning compensations for lower-order physical deficiencies and restrictions. From work with the conceptual model we drew the following methodological conclusions: First, the driver has to be considered as a holistic subject of the activity, as a multilevel hierarchically organised system of an organism, a temperament, an individuality, and a personality where the personality is the leading subsystem from the standpoint of safety. Second, driving as a human form of a sociopractical activity, is also a hierarchically organised dynamic system. Third, in an evaluation of driving ability it is a question of matching these two hierarchically organised structures: a subject of an activity and a proper activity. Fourth, an evaluation has to be person centred but not disease-, function- or method centred. On the basis of our study a multidisciplinary team (practitioner, driving school teacher, psychologist, occupational therapist) is recommended for use in demanding driver evaluations. Primary in a driver’s evaluations is a coherent conceptual model while concrete methods of evaluations may vary. However, the on-road test must always be performed if possible.

A Method for Evaluation of Product Lifecycle Alternatives under Uncertainty

In each stage of product development, we need to take decisions, by evaluating multiple product alternatives based on multiple criteria. Classical evaluation methods like weighted objectives method assumes certainty about information available during product development. However, designers often must evaluate under uncertainty. Often the likely performance, cost or environmental impacts of a product proposal could be estimated only with certain confidence, which may vary from one proposal to another. In such situations, the classical approaches to evaluation can give misleading results. There is a need for a method that can aid in decision making by supporting quantitative comparison of alternatives to identify the most promising alternative, under uncertain information about the alternatives. A method called confidence weighted objectives method is developed to compare the whole life cycle of product proposals using multiple evaluation criteria under various levels of uncertainty with non crisp values. It estimates the overall worth of proposal and confidence on the estimate, enabling deferment of decision making when decisions cannot be made using current information available.

Instrumental and interpretive methods in evaluating urban programmes

The turn within urban policy to address increasingly complex social, economic and environmental problems has exposed some of the fragility of traditional measurement models and their reliance on the rational paradigm. This article looks at the experiences of the European Union (EU) Programme for Peace and Reconciliation in Northern Ireland and its particular attempt to construct new District Partnerships to deliver area-based regeneration programmes. It highlights the need to combine instrumental and interpretative evaluation methods in an attempt to explain the wider contribution of governance to conflict resolution and participatory practice in local development. It concludes by highlighting the value of conceptual approaches that deal with the politics of evaluation and the distributional effects of policy interventions designed to create new relationships within and between multiple stakeholders.

Efficient Evaluation of Multidimensional Time-Varying Density Forecasts

We propose two simple evaluation methods for time varying density forecasts of continuous higher dimensional random variables. Both methods are based on the probability integral transformation for unidimensional forecasts. The first method tests multinormal densities and relies on the rotation of the coordinate system. The advantage of the second method is not only its applicability to any continuous distribution but also the evaluation of the forecast accuracy in specific regions of its domain as defined by the user’s interest. We show that the latter property is particularly useful for evaluating a multidimensional generalization of the Value at Risk. In simulations and in an empirical study, we examine the performance of both tests.

Towards an automated evaluation process for software architectures

Optimizing and editing enterprise software systems, after the implementation process has started, is widely recognized to be an expensive process. This has led to increasing emphasis on locating mistakes within software systems at the design stage, to help minimize development costs. There is increasing interest in the field of architecture evaluation techniques that can identify problems at the design stage, either within complete, or partially complete architectures. Most current techniques rely on manual review-based evaluation methods that require advanced skills from architects and evaluators. We are currently considering what a formal Architecture Description Language (ADL) can contribute to the process of architecture evaluation and validation. Our investigation is considering the inter-relationships between the activities performed during the architecture evaluation process, the characteristics an ADL should possess to support these activities, and the tools needed to provide convenient access to, and presentation of architectural information.

Using Baseline Methods to Identify Non-technical Losses in the Context of Smart Grids

Demand response has gained increasing importance in the context of competitive electricity markets and smart grid environments. In addition to the importance that has been given to the development of business models for integrating demand response, several methods have been developed to evaluate the consumers’ performance after the participation in a demand response event. The present paper uses those performance evaluation methods, namely customer baseline load calculation methods, to determine the expected consumption in each period of the consumer historic data. In the cases in which there is a certain difference between the actual consumption and the estimated consumption, the consumer is identified as a potential cause of non-technical losses. A case study demonstrates the application of the proposed method to real consumption data.

Model Selection in Summary Evaluation

A difficulty in the design of automated text summarization algorithms is in the objective evaluation. Viewing summarization as a tradeoff between length and information content, we introduce a technique based on a hierarchy of classifiers to rank, through model selection, different summarization methods. This summary evaluation technique allows for broader comparison of summarization methods than the traditional techniques of summary evaluation. We present an empirical study of two simple, albeit widely used, summarization methods that shows the different usages of this automated task-based evaluation system and confirms the results obtained with human-based evaluation methods over smaller corpora.

Methods for exploring implementation variation and local context within a cluster tandomised community intervention trial

Insignificant or modest findings in intervention trials may be attributable to poorly designed or theorised interventions, poorly implemented interventions, or inadequate evaluation methods. The pre-existing context may also account for the effects observed. A combination of qualitative and quantitative methods is outlined that will permit the determination of how context level factors might modify intervention effectiveness, within a cluster randomised community intervention trial to promote the health of mothers with new babies. The methods include written and oral narratives, key informant interviews, impact logs, and inter-organisational network analyses. Context level factors, which may affect intervention uptake, success, and sustainability are the density of inter-organisational ties within communities at the start of the intervention, the centrality of the primary care agencies expected to take a lead with the intervention, the extent of context-level adaptation of the intervention, and the amount of local resources contributed by the participating agencies. Investigation of how intervention effects are modified by context is a new methodological frontier in community intervention trial research.

A proposed hybrid evaluation methodology for intelligent transport system deployment

The rapid deployment of intelligent transport systems in practice requires serious evaluation of new construction projects. Traditional transportation evaluation methods are insufficient to capture the new features of intelligent transport systems, particularly the applications of recently developed information technologies. This research aims to develop a new conceptual framework to evaluate the social, economic and environmental impacts of intelligent transport systems deployment, with explicit consideration of impacts of information technologies on variant travel behaviours. The research would provide a systematic evaluation methodology to enable decision makers and publics better evaluating and understanding the social, economic and environmental benefits of implementation of intelligent transport systems.

Assessing cost-effectiveness in obesity (ACE-Obesity) : an overview of the ACE approach, economic methods and cost results

Background : The aim of the ACE-Obesity study was to determine the economic credentials of interventions which aim to prevent unhealthy weight gain in children and adolescents. We have reported elsewhere on the modelled effectiveness of 13 obesity prevention interventions in children. In this paper, we report on the cost results and associated methods together with the innovative approach to priority setting that underpins the ACE-Obesity study.

Methods : The Assessing Cost-Effectiveness (ACE) approach combines technical rigour with 'due process' to facilitate evidence-based policy analysis. Technical rigour was achieved through use of standardised evaluation methods, a research team that assembles best available evidence and extensive uncertainty analysis. Cost estimates were based on pathway analysis, with resource usage estimated for the interventions and their 'current practice' comparator, as well as associated cost offsets. Due process was achieved through involvement of stakeholders, consensus decisions informed by briefing papers and 2nd stage filter analysis that captures broader factors that influence policy judgements in addition to cost-effectiveness results. The 2nd stage filters agreed by stakeholders were 'equity', 'strength of the evidence', 'feasibility of implementation', 'acceptability to stakeholders', 'sustainability' and 'potential for side-effects'.

Results : The intervention costs varied considerably, both in absolute terms (from cost saving [6 interventions] to in excess of AUD50m per annum) and when expressed as a 'cost per child' estimate (from <AUD1.0 [reduction of TV advertising of high fat foods/high sugar drinks] to >AUD31,000 [laparoscopic adjustable gastric banding for morbidly obese adolescents]). High costs per child reflected cost structure, target population and/or under-utilisation.

Conclusions : The use of consistent methods enables valid comparison of potential intervention costs and cost-offsets for each of the interventions. ACE-Obesity informs policy-makers about cost-effectiveness, health impact, affordability and 2nd stage filters for important options for preventing unhealthy weight gain in children. In related articles cost-effectiveness results and second stage filter considerations for each intervention assessed will be presented and analysed.

Toward the standardization of venture capital investment evaluation : decision criteria for rating investee business plans

This study examined the criteria used by venture capitalists to evaluate business plans in order to make investment decisions. A literature survey revealed two competing theories: “espoused criteria” where evaluation decisions are based on what venture capitalists say are the decisive factors; versus the use of “known attributes” that successful ventures actually possess. Brunswik’s Lens Model from Social Judgment Theory guided an empirical investigation of several different evaluation methods based on information contained in 129 business plans submitted for venture capital over a 3 year period. Data evaluation culminated in the comparison of the percentage of correct decisions (“hit-rate”) for each method. We found that decisions based on the known attributes of successful ventures have significantly better hit-rates than decisions made using espoused criteria. Discussion centred on the goal of achieving consistency in the conduct of venture analysis. Process standardization can aid in the achievement of consistency. Future research will both deepen and broaden insights.