Conducting statistical tests of hypotheses : five common misconceptions found in transportation research

Most statistical methods use hypothesis testing. Analysis of variance, regression, discrete choice models, contingency tables, and other analysis methods commonly used in transportation research share hypothesis testing as the means of making inferences about the population of interest. Despite the fact that hypothesis testing has been a cornerstone of empirical research for many years, various aspects of hypothesis tests commonly are incorrectly applied, misinterpreted, and ignored—by novices and expert researchers alike. On initial glance, hypothesis testing appears straightforward: develop the null and alternative hypotheses, compute the test statistic to compare to a standard distribution, estimate the probability of rejecting the null hypothesis, and then make claims about the importance of the finding. This is an oversimplification of the process of hypothesis testing. Hypothesis testing as applied in empirical research is examined here. The reader is assumed to have a basic knowledge of the role of hypothesis testing in various statistical methods. Through the use of an example, the mechanics of hypothesis testing is first reviewed. Then, five precautions surrounding the use and interpretation of hypothesis tests are developed; examples of each are provided to demonstrate how errors are made, and solutions are identified so similar errors can be avoided. Remedies are provided for common errors, and conclusions are drawn on how to use the results of this paper to improve the conduct of empirical research in transportation.

Evaluating the effect of a patient education brochure on patients’ expectations and satisfaction with emergency department service

Introduction: The purpose of this study was to assess the capacity of a written intervention, in this case a patient information brochure, to improve patient satisfaction during an Emergency Department (ED) visit. For the purpose of measuring the effect of the intervention the ED journey was conceptualised as a series of distinct areas of service comprising waiting time, service by the triage nurse, care from doctors and nurses and information giving Background of study: Research into patient satisfaction has become a widespread activity endorsed by both governments and hospital administrations. The literature on ED patient satisfaction has consistently indicated three primary areas of patient dissatisfaction: waiting time, nursing care and communication. Recent developments in the literature on patient satisfaction studies however have highlighted the relationship between patients. expectations of a service encounter and their consequent assessment of the experience as dissatisfying or satisfying. Disconfirmation theory posits that the degree to which expectations are confirmed will affect subsequent levels of satisfaction. The conceptual framework utilised in this study is Coye.s (2004) model of disconfirmation. Coye while reiterating satisfaction is a consequence of the degree expectations are either confirmed or disconfirmed also posits that expectations can be modified by interventions. Coye.s work conceptualises these interventions as intra encounter experiences (cues) which function to adjust expectations. Coye suggests some cues are unintended and may have a negative impact which also reinforces the value of planned cues intended to meet or exceed consumer expectations. Consequently the brochure can be characterized as a potentially positive cue, encouraging the patient to understand processes and to orient them in what can be a confronting environment. Only a limited number of studies have examined the effect of written interventions within an ED. No studies could be located which have tested the effect of ED interventions using a conceptual framework which relates the effect of the degree to which expectations are confirmed or disconfirmed in terms of satisfaction with services. Method: Two studies were conducted. Study One used qualitative methods to explore patients. expectations of the ED from the perspective of both patients and health care professionals. Study One was used in part to direct the development of the intervention (brochure) in Study Two. The brochure was an intervention designed to modify patients. expectations thus increasing their satisfaction with the provision of ED service. As there was no existing tools to measure ED patients. expectations and satisfaction a new tool was also developed based on the findings and the literature of Study One. Study Two used a non-randomised, quasi-experimental approach using a non-equivalent post-test only comparison group design used to investigate the effect of the patient education brochure (Stommel and Wills, 2004). The brochure was disseminated to one of two study groups (the intervention group). The effect of the brochure was assessed by comparing the data obtained from both the intervention and control group. These two groups consisted of 150 participants each. It was expected that any differences in the relevant domains selected for examination would indicate the effect of the brochure both on expectation and potentially satisfaction. Results: Study One revealed several areas of common ground between patients and nurses in terms of relevant content for the written intervention, including the need for information on the triage system and waiting times. Areas of difference were also found with patients emphasizing communication issues, whereas focus group members expressed concern that patients were often unable to assimilate verbal information. The findings suggested the potential utility of written material to reinforce verbal communication particularly in terms of the triage process and other ED protocols. This material was synthesized within the final version of the written intervention. Overall the results of Study Two indicated no significant differences between the two groups. The intervention group did indicate a significant number of participants who viewed the brochure of having changed their expectations. The effect of the brochure may have been obscured by a lack of parity between the two groups as the control group presented with statistically significantly higher levels of acuity and experienced significantly shorter waiting times. In terms of disconfirmation theory this would suggest expectations that had been met or exceeded. The results confirmed the correlation of expectations with satisfaction. Several domains also indicated age as a significant predictor with older patients tending to score higher satisfaction results. Other significant predictors of satisfaction established were waiting time and care from nurses, reinforcing the combination of efficient service and positive interpersonal experiences as being valued by patients. Conclusions: Information presented in written form appears to benefit a significant number of ED users in terms of orientation and explaining systems and procedures. The degree to which these effects may interact with other dimensions of satisfaction however is likely to be limited. Waiting time and interpersonal behaviours from staff also provide influential cues in determining satisfaction. Written material is likely to be one element in a series of coordinated strategies to improve patient satisfaction during periods of peak demand.

A prioritised fuzzy constraint satisfaction approach to model agent negotiation for railway scheduling

With the recent regulatory reforms in a number of countries, railways resources are no longer managed by a single party but are distributed among different stakeholders. To facilitate the operation of train services, a train service provider (SP) has to negotiate with the infrastructure provider (IP) for a train schedule and the associated track access charge. This paper models the SP and IP as software agents and the negotiation as a prioritized fuzzy constraint satisfaction (PFCS) problem. Computer simulations have been conducted to demonstrate the effects on the train schedule when the SP has different optimization criteria. The results show that by assigning different priorities on the fuzzy constraints, agents can represent SPs with different operational objectives.

Case management educational intervention with public health nurses : cluster randomized controlled trial

Aim: This paper is a report of a study conducted to determine the effectiveness of a community case management collaborative education intervention in terms of satisfaction, learning and performance among public health nurses. Background: Previous evaluation studies of case management continuing professional education often failed to demonstrate effectiveness across a range of outcomes and had methodological weaknesses such as small convenience samples and lack of control groups. Method: A cluster randomised controlled trial was conducted between September 2005 and February 2006. Ten health centre clusters (5 control, 5 intervention) recruited 163 public health nurses in Taiwan to the trial. After pre-tests for baseline measurements, public health nurses in intervention centres received an educational intervention of four half-day workshops. Post-tests for both groups were conducted after the intervention. Two-way repeated measures analysis of variance was performed to evaluate the effect of the intervention on target outcomes. Results: A total of 161 participants completed the pre- and post-intervention measurements. This was almost a 99% response rate. Results revealed that 97% of those in the experimental group were satisfied with the programme. There were statistically significant differences between the two groups in knowledge (p = 0.001), confidence in case management skills (p = 0.001), preparedness for case manager role activities (p = 0.001), self-reported frequency in using skills (p = 0.001), and role activities (p = 0.004). Conclusion: Collaboration between academic and clinical nurses is an effective strategy to prepare nurses for rapidly-changing roles.

Sustainability performance of construction : conceptual models of satisfaction levels in construction projects

In general, the performance of construction projects, including their sustainability performance, does not meet optimal expectations. One aspect of this is the performance of the participants who are independent and make a significance impact on overall project outcomes. Of these participants, the client is traditionally the owner of the project, the architect or engineer is engaged as the lead designer and a contractor is selected to construct the facilities. Generally, the performance of the participants is gauged by considering three main factors, namely, time, cost and quality. As the level of satisfaction is a subjective issue, it is rarely used in the performance evaluation of construction work. Recently, various approaches to the measurement of satisfaction have been made in an attempt to determine the performance of construction project outcomes - for instance, client satisfaction, customer satisfaction, contractor satisfaction, occupant satisfaction and home buyer satisfaction. These not only identify the performance of the construction project but are also used to improve and maintain relationships. In addition, these assessments are necessary for the continuous improvement and enhanced cooperation of participants. The measurement of satisfaction levels primarily involves expectations and perceptions. An expectation can be regarded as a comparative standard of different needs, motives and beliefs, while a perception is a subjective interpretation that is influenced by moods, experiences and values. This suggests that the disparity between perceptions and expectations may possibly be used to represent different levels of satisfaction. However, this concept is rather new and in need of further investigation. This chapter examines the methods commonly practised in measuring satisfaction levels today and the advantages of promoting these methods. The results provide a preliminary review of the advantages of satisfaction measurement in the construction industry and recommendations are made concerning the most appropriate methods to use in identifying the performance of project outcomes.

Longitudinal evaluations of student satisfaction with a postgraduate unit using importance-performance analysis

The Australian tourism tertiary education sector operates in a competitive and dynamic environment, which necessitates a market orientation to be successful. Academic staff and management in the sector must regularly assess the perceptions of prospective and current students and monitor the satisfaction levels of current students. This study is concerned with the setting and monitoring of satisfaction levels of current students, reporting the results of three longitudinal investigations of student satisfaction in a postgraduate unit. The study also addresses a limitation of a university’s generic teaching evaluation instrument. Importance-Performance Analysis (IPA) has been recommended as a simple but effective tool for overcoming the deficiencies of many student evaluation studies, which have generally measured only attribute performance at the end of a semester. IPA was used to compare student expectations of the unit at the beginning of a semester with their perceptions of performance 10 weeks later. The first stage documented key benchmarks for which amendments to the unit based on student feedback could be evaluated during subsequent teaching periods.

Conceptual models of satisfaction level in construction

It is recognized that, in general, the performance of construction projects does not meet optimal expectations. One aspect of this is the performance of each participant, which is interdependent and makes a significance impact on overall project outcomes. Of these, the client is traditionally the owner of the project, the architect or engineer is engaged as the lead designer and a contractor is selected to construct the facilities. Generally, the performance of the participants is gauged by considering three main factors, namely time, cost and quality. As the level of satisfaction is a subjective measurement, it is rarely used in the performance evaluation of construction work. Recently, various approaches to the measurement of satisfaction have been made in attempting to determine the performance of construction project outcomes – for instance client satisfaction, consultant satisfaction, contractor satisfaction, customer satisfaction and home buyer satisfaction. These not only identify the performance of the construction project, but are also used to improve and maintain relationships. In addition, these assessments are necessary for continuous improvement and enhanced cooperation between participants. The measurement of satisfaction levels primarily involves expectations and perceptions. An expectation can be regarded as a comparison standard of different needs, motives and beliefs, while a perception is a subjective interpretation that is influenced by moods, experiences and values. This suggests that the disparity between perceptions and expectations may be used to represent different levels of satisfaction. However, this concept is rather new and in need of further investigation. This paper examines the current methods commonly practiced in measuring satisfaction level and the advantages of promoting these methods. The results provided are a preliminary review of the advantages of satisfaction measurement in the construction industry and recommendations are made concerning the most appropriate methods for use in identifying the performance of project outcomes.