Real or imagined? A study exploring the existence of common method variance effects in road safety research

Common method variance (CMV) has received little attention within the field of road safety research despite a heavy reliance on self-report data. Two surveys were completed by 214 motorists over a two-month period, allowing associations between social desirability and key road safety variables and relationships between scales across the two survey waves to be examined. Social desirability was found to have a strong negative correlation with the Driver Behaviour Questionnaire (DBQ) sub-scales as well as age, but not with crashes and offences. Drivers who scored higher on the social desirability scale were also less likely to report aberrant driving behaviours as measured by the DBQ. Controlling for social desirability did not substantially alter the predictive relationship between the DBQ and the crash and offences variables. The strength of the correlations within and between the two waves were also compared with the results strongly suggesting that effects associated with CMV were present. Identification of CMV would be enhanced by the replication of this study with a larger sample size and comparing self-report data with official sources.

The effects of method variance within measures

Despite concern about method variance between measures as a bias in survey research, scholars have overlooked or ignored the effects of method variance within measures (i.e., covariation among items from the same scale that may be attributed to the method of measurement employed). Not only do few commonly used survey instruments reflect efforts to control for method variance, but guides to scale construction encourage researchers to implement strategies that enhance the effects of method variance within measures. In this article, we have argued that when method variance inflates relationships between questionnaire items, traditional psychometric indices overestimate the amount of true or construct variance that scales capture. Implications for survey research that uses fixed alternative questionnaire measures are delineated.

How large are actor and partner effects of personality on relationship satisfaction? The importance of controlling for shared method variance

Previous research suggests that the personality of a relationship partner predicts not only the individual’s own satisfaction with the relationship but also the partner’s satisfaction. Based on the actor–partner interdependence model, the present research tested whether actor and partner effects of personality are biased when the same method (e.g., self-report) is used for the assessment of personality and relationship satisfaction and, consequently, shared method variance is not controlled for. Data came from 186 couples, of whom both partners provided self- and partner reports on the Big Five personality traits. Depending on the research design, actor effects were larger than partner effects (when using only self-reports), smaller than partner effects (when using only partner reports), or of about the same size as partner effects (when using self- and partner reports). The findings attest to the importance of controlling for shared method variance in dyadic data analysis.

Estimating the effect of common method variance: the method-method pair technique with an illustration from tam research

This paper presents a meta-analysis-based technique to estimate the effect of common method variance on the validity of individual theories. The technique explains between-study variance in observed correlations as a function of the susceptibility to common method variance of the methods employed in individual studies. The technique extends to mono-method studies the concept of method variability underpinning the classic multitrait-multimethod technique. The application of the technique is demonstrated by analyzing the effect of common method variance on the observed correlations between perceived usefulness and usage in the technology acceptance model literature. Implications of the technique and the findings for future research are discussed.

Validating the is-impact model in the Malaysian public sector

This research is one of several ongoing studies conducted within the IT Professional Services (ITPS) research programme at Queensland University of Technology (QUT). In 2003, ITPS introduced the IS-Impact model, a measurement model for measuring information systems success from the viewpoint of multiple stakeholders. The model, along with its instrument, is robust, simple, yet generalisable, and yields results that are comparable across time, stakeholders, different systems and system contexts. The IS-Impact model is defined as “a measure at a point in time, of the stream of net benefits from the Information System (IS), to date and anticipated, as perceived by all key-user-groups”. The model represents four dimensions, which are ‘Individual Impact’, ‘Organizational Impact’, ‘Information Quality’ and ‘System Quality’. The two Impact dimensions measure the up-to-date impact of the evaluated system, while the remaining two Quality dimensions act as proxies for probable future impacts (Gable, Sedera & Chan, 2008). To fulfil the goal of ITPS, “to develop the most widely employed model” this research re-validates and extends the IS-Impact model in a new context. This method/context-extension research aims to test the generalisability of the model by addressing known limitations of the model. One of the limitations of the model relates to the extent of external validity of the model. In order to gain wide acceptance, a model should be consistent and work well in different contexts. The IS-Impact model, however, was only validated in the Australian context, and packaged software was chosen as the IS understudy. Thus, this study is concerned with whether the model can be applied in another different context. Aiming for a robust and standardised measurement model that can be used across different contexts, this research re-validates and extends the IS-Impact model and its instrument to public sector organisations in Malaysia. The overarching research question (managerial question) of this research is “How can public sector organisations in Malaysia measure the impact of information systems systematically and effectively?” With two main objectives, the managerial question is broken down into two specific research questions. The first research question addresses the applicability (relevance) of the dimensions and measures of the IS-Impact model in the Malaysian context. Moreover, this research question addresses the completeness of the model in the new context. Initially, this research assumes that the dimensions and measures of the IS-Impact model are sufficient for the new context. However, some IS researchers suggest that the selection of measures needs to be done purposely for different contextual settings (DeLone & McLean, 1992, Rai, Lang & Welker, 2002). Thus, the first research question is as follows, “Is the IS-Impact model complete for measuring the impact of IS in Malaysian public sector organisations?” [RQ1]. The IS-Impact model is a multidimensional model that consists of four dimensions or constructs. Each dimension is represented by formative measures or indicators. Formative measures are known as composite variables because these measures make up or form the construct, or, in this case, the dimension in the IS-Impact model. These formative measures define different aspects of the dimension, thus, a measurement model of this kind needs to be tested not just on the structural relationship between the constructs but also the validity of each measure. In a previous study, the IS-Impact model was validated using formative validation techniques, as proposed in the literature (i.e., Diamantopoulos and Winklhofer, 2001, Diamantopoulos and Siguaw, 2006, Petter, Straub and Rai, 2007). However, there is potential for improving the validation testing of the model by adding more criterion or dependent variables. This includes identifying a consequence of the IS-Impact construct for the purpose of validation. Moreover, a different approach is employed in this research, whereby the validity of the model is tested using the Partial Least Squares (PLS) method, a component-based structural equation modelling (SEM) technique. Thus, the second research question addresses the construct validation of the IS-Impact model; “Is the IS-Impact model valid as a multidimensional formative construct?” [RQ2]. This study employs two rounds of surveys, each having a different and specific aim. The first is qualitative and exploratory, aiming to investigate the applicability and sufficiency of the IS-Impact dimensions and measures in the new context. This survey was conducted in a state government in Malaysia. A total of 77 valid responses were received, yielding 278 impact statements. The results from the qualitative analysis demonstrate the applicability of most of the IS-Impact measures. The analysis also shows a significant new measure having emerged from the context. This new measure was added as one of the System Quality measures. The second survey is a quantitative survey that aims to operationalise the measures identified from the qualitative analysis and rigorously validate the model. This survey was conducted in four state governments (including the state government that was involved in the first survey). A total of 254 valid responses were used in the data analysis. Data was analysed using structural equation modelling techniques, following the guidelines for formative construct validation, to test the validity and reliability of the constructs in the model. This study is the first research that extends the complete IS-Impact model in a new context that is different in terms of nationality, language and the type of information system (IS). The main contribution of this research is to present a comprehensive, up-to-date IS-Impact model, which has been validated in the new context. The study has accomplished its purpose of testing the generalisability of the IS-Impact model and continuing the IS evaluation research by extending it in the Malaysian context. A further contribution is a validated Malaysian language IS-Impact measurement instrument. It is hoped that the validated Malaysian IS-Impact instrument will encourage related IS research in Malaysia, and that the demonstrated model validity and generalisability will encourage a cumulative tradition of research previously not possible. The study entailed several methodological improvements on prior work, including: (1) new criterion measures for the overall IS-Impact construct employed in ‘identification through measurement relations’; (2) a stronger, multi-item ‘Satisfaction’ construct, employed in ‘identification through structural relations’; (3) an alternative version of the main survey instrument in which items are randomized (rather than blocked) for comparison with the main survey data, in attention to possible common method variance (no significant differences between these two survey instruments were observed); (4) demonstrates a validation process of formative indexes of a multidimensional, second-order construct (existing examples mostly involved unidimensional constructs); (5) testing the presence of suppressor effects that influence the significance of some measures and dimensions in the model; and (6) demonstrates the effect of an imbalanced number of measures within a construct to the contribution power of each dimension in a multidimensional model.

When non-significance maybe significant: lessons learned from a study into the development, implementation and evaluation of a risk assessment tool for fleet settings

Introduction This study reports on the development of a self report assessment tool to increase the efficacy of crash prediction within Australian Fleet settings Over last 20 years an array of measures have been produced (Driver anger scale, Driving Skill Inventory, Manchester Driver Behaviour Questionnaire, Driver Attitude Questionnaire, Driver Stress Inventory, Safety Climate Questionnaire) While these tools are useful, research has demonstrated limited ability to accurately identify individuals most likely to be involved in a crash. Reasons cited include; - Crashes are relatively rare - Other competing factors may influence crash event - Ongoing questions regarding the validity of self report measures (common method variance etc) - Lack of contemporary issues relating to fleet driving performance

Multiphase experiments with at least one later laboratory phase. I. orthogonal designs

The paper provides a systematic approach to designing the laboratory phase of a multiphase experiment, taking into account previous phases. General principles are outlined for experiments in which orthogonal designs can be employed. Multiphase experiments occur widely, although their multiphase nature is often not recognized. The need to randomize the material produced from the first phase in the laboratory phase is emphasized. Factor-allocation diagrams are used to depict the randomizations in a design and the use of skeleton analysis-of-variance (ANOVA) tables to evaluate their properties discussed. The methods are illustrated using a scenario and a case study. A basis for categorizing designs is suggested. This article has supplementary material online.

The Driver Behaviour Questionnaire as accident predictor: A methodological re-meta-analysis

Problem The Manchester Driver Behaviour Questionnaire (DBQ) is the most commonly used self-report tool in traffic safety research and applied settings. It has been claimed that the violation factor of this instrument predicts accident involvement, which was supported by a previous meta-analysis. However, that analysis did not test for methodological effects, or include contacting researchers to obtain unpublished results. Method The present study re-analysed studies on prediction of accident involvement from DBQ factors, including lapses, and many unpublished effects. Tests of various types of dissemination bias and common method variance were undertaken. Results Outlier analysis showed that some effects were probably not reliable data, but excluding them did not change the results. For correlations between violations and crashes, tendencies for published effects to be larger than unpublished ones and for effects to decrease over time were observed, but were not significant. Also, analysis using the proxy of the mean of accidents in studies indicated that studies where effects for violations are unknown have smaller effect sizes. These differences indicate dissemination bias. Studies using self-reported accidents as dependent variables had much larger effects than those using recorded accident data. Also, zero-order correlations were larger than partial correlations that controlled for exposure. Similarly, violations/accidents effects were strong only when there was also a strong correlation between accidents and exposure. Overall, the true effect is probably very close to zero (r<.07) for violations versus traffic accident involvement, depending upon which systematic tendencies in the data are controlled for. Conclusions: Methodological factors and dissemination bias have inflated the mean effect size of the DBQ in the published literature. Strong evidence of various artefactual effects is apparent. Practical Applications A greater level of care should be taken if the DBQ continues to be used in traffic safety research. Also, validation of self-reports should be more comprehensive in the future, taking into account the possibility of common method variance.

Predicting crashes using traffic offences. A meta-analysis that examines potential bias between self-report and archival data

Background Traffic offences have been considered an important predictor of crash involvement, and have often been used as a proxy safety variable for crashes. However the association between crashes and offences has never been meta-analysed and the population effect size never established. Research is yet to determine the extent to which this relationship may be spuriously inflated through systematic measurement error, with obvious implications for researchers endeavouring to accurately identify salient factors predictive of crashes. Methodology and Principal Findings Studies yielding a correlation between crashes and traffic offences were collated and a meta-analysis of 144 effects drawn from 99 road safety studies conducted. Potential impact of factors such as age, time period, crash and offence rates, crash severity and data type, sourced from either self-report surveys or archival records, were considered and discussed. After weighting for sample size, an average correlation of r = .18 was observed over the mean time period of 3.2 years. Evidence emerged suggesting the strength of this correlation is decreasing over time. Stronger correlations between crashes and offences were generally found in studies involving younger drivers. Consistent with common method variance effects, a within country analysis found stronger effect sizes in self-reported data even controlling for crash mean. Significance The effectiveness of traffic offences as a proxy for crashes may be limited. Inclusion of elements such as independently validated crash and offence histories or accurate measures of exposure to the road would facilitate a better understanding of the factors that influence crash involvement.

Why Are Some Subsidiaries of Multinationals the Source of Novel Practices while Others Are Not? National, Corporate and Functional Influences

It has frequently been argued that multinational companies are moving towards network forms whereby subsidiaries share different practices with the rest of the company. This paper presents large-scale empirical evidence concerning the extent to which subsidiaries input novel practices into the rest of the multinational. We investigate this in the field of human resources through analysis of a unique international data set in four host countries - Canada, Ireland, Spain and the UK - and address the question of how we can explain variation between subsidiaries in terms of whether they initiate the diffusion of practices to other subsidiaries. The data support the argument that multiple, rather than single, factor explanations are required to more effectively understand the factors promoting or retarding the diffusion of human resource practices within multinational companies. It emerges that national, corporate and functional contexts all matter. More specifically, actors at subsidiary level who seek to initiate diffusion appear to be differentially placed according to their national context, their place within corporate structures and the extent to which the human resource function is internationally networked.

Liderança, virtuosidade e criatividade explicando o desempenho de equipes

As organizações estão se tornando cada vez mais competitivas e buscando, constantemente, formas de aumentar o seu desempenho através da compreensão dos fatores que o antecedem. A nossa investigação incorpora dois estudos, ambos incidentes sobre as lojas de uma grande cadeia de varejo brasileira. O primeiro estudo, envolvendo 68 lojas, investiga como a liderança autêntica explica o desempenho das lojas através do efeito mediador da virtuosidade e da potência das equipes (uma espécie de sentido de eficácia coletiva). O segundo estudo, abrangendo 46 lojas, testa como a potência das equipes explica o respetivo desempenho através do efeito mediador da criatividade. Os principais resultados são: (a) a relação entre a liderança autêntica e o desempenho das lojas é totalmente mediada pela virtuosidade e pela potência das equipes; (b) a criatividade medeia a relação entre a potência das equipes e o desempenho das lojas; (c) a criatividade não explica diretamente o desempenho, mas parece influenciá-lo através do efeito mediador da potência. Os cuidados metodológicos destinados a diminuir os riscos de variância do método comum aumentam a validade dos estudos. Ao focar variáveis relacionadas com a virtuosidade, a liderança autêntica, e as forças das equipes, e ao incluir dados de desempenho objetivo, a nossa investigação é um contributo relevante para o movimento dos “Estudos Organizacionais Positivos”.

Business level strategy and performance: the moderating effects of environment and structure

Purpose – This study aims to examine the moderating effects of external environment and organisational structure in the relationship between business-level strategy and organisational performance. Design/methodology/approach – The focus of the study is on manufacturing firms in the UK belonging to the electrical and mechanical engineering sectors, and respondents were CEOs. Both objective and subjective measures were used to assess performance. Non-response bias was assessed statistically and appropriate measures taken to minimise the impact of common method variance (CMV). Findings – The results indicate that environmental dynamism and hostility act as moderators in the relationship between business-level strategy and relative competitive performance. In low-hostility environments a cost-leadership strategy and in high-hostility environments a differentiation strategy lead to better performance compared with competitors. In highly dynamic environments a cost-leadership strategy and in low dynamism environments a differentiation strategy are more helpful in improving financial performance. Organisational structure moderates the relationship of both the strategic types with ROS. However, in the case of ROA, the moderating effect of structure was found only in its relationship with cost-leadership strategy. A mechanistic structure is helpful in improving the financial performance of organisations adopting either a cost-leadership or a differentiation strategy. Originality/value – Unlike many other empirical studies, the study makes an important contribution to the literature by examining the moderating effects of both environment and structure on the relationship between business-level strategy and performance in a detailed manner, using moderated regression analysis.

Generic strategies and performance: evidence from manufacturing firms

Purpose – The purpose of this study is to examine the relationship between business-level strategy and organisational performance and to test the applicability of Porter's generic strategies in explaining differences in the performance of organisations. Design/methodology/approach – The study was focussed on manufacturing firms in the UK belonging to the electrical and mechanical engineering sectors. Data were collected through a postal survey using the survey instrument from 124 organisations and the respondents were all at CEO level. Both objective and subjective measures were used to assess performance. Non-response bias was assessed statistically and it was not found to be a major problem affecting this study. Appropriate measures were taken to ensure that common method variance (CMV) does not affect the results of this study. Statistical tests indicated that CMV problem does not affect the results of this study. Findings – The results of this study indicate that firms adopting one of the strategies, namely cost-leadership or differentiation, perform better than “stuck-in-the-middle” firms which do not have a dominant strategic orientation. The integrated strategy group has lower performance compared with cost-leaders and differentiators in terms of financial performance measures. This provides support for Porter's view that combination strategies are unlikely to be effective in organisations. However, the cost-leadership and differentiation strategies were not strongly correlated with the financial performance measures indicating the limitations of Porter's generic strategies in explaining performance heterogeneity in organisations. Originality/value – This study makes an important contribution to the literature by identifying some of the gaps in the literature through a systematic literature review and addressing those gaps.

The relationship between challenging parenting behaviour and childhood anxiety disorders

Background: This research investigates the relationship between challenging parenting behaviour and childhood anxiety disorders proposed by Bögels and Phares (2008). Challenging parenting behaviour involves the playful encouragement of children to go beyond their own limits, and may decrease children’s risk for anxiety (Bögels & Phares, 2008). Method: Parents (n = 164 mothers, 144 fathers) of 164 children aged between 3.4 and 4.8 years participated in the current study. A multi-method, multi-informant assessment of anxiety was used, incorporating data from diagnostic interviews as well as questionnaire measures. Parents completed self-report measures of their parenting behaviour (n = 147 mothers, 138 fathers) and anxiety (n = 154 mothers, 143 fathers). Mothers reported on their child’s anxiety via questionnaire as well as diagnostic interview (n = 156 and 164 respectively). Of these children, 74 met criteria for an anxiety disorder and 90 did not. Results: Fathers engaged in challenging parenting behaviour more often than mothers. Both mothers’ and fathers’ challenging parenting behaviour was associated with lower report of child anxiety symptoms. However, only mothers’ challenging parenting behaviour was found to predict child clinical anxiety diagnosis. Limitations: Shared method variance from mothers confined the interpretation of these results. Moreover, due to study design, it is not possible to delineate cause and effect. Conclusions: The finding with respect to maternal challenging parenting behaviour was not anticipated, prompting replication of these results. Future research should investigate the role of challenging parenting behaviour by both caregivers as this may have implications for parenting interventions for anxious children.

Validez de constructo de tres escalas de satisfacción del paciente mediante la estrategia de matrices multirrasgo-multimétodo

Se emplea el diseño de las matrices multirrasgo-multimétodo (MTMM) en la evaluación de la satisfacción del paciente. La muestra, extraída al azar simple, fue de 254 pacientes ingresados en tres hospitales del Servei Valencià de Salut de la provincia de Alicante, mayores de 16 años, conscientes y orientados. Los instrumentos de medida fueron tres escalas de satisfacción, dos de carácter general y una específica con los cuidados de enfermería, todas autoinformes. Los rasgos evaluados fueron varias dimensiones de satisfacción, y los métodos tres tipos de formulación de items y escalas de respuesta. Se ha empleado el análisis factorial confirmatorio, siguiéndose la estrategia de constrastar varios modelos alternativos (Widaman, 1985; Marsh, 1989). Los resulta dos indican que: la varianza de método es elevada, superior a la de rasgos; existe validez convergente; los rasgos están altamente correlacionados, pero hay evidencia de validez discriminante; dos métodos están altamente correlacionados; y no se ha podido estimar el modelo general de matrices MTMM.