A proposed validation framework for expert elicited Bayesian Networks

The popularity of Bayesian Network modelling of complex domains using expert elicitation has raised questions of how one might validate such a model given that no objective dataset exists for the model. Past attempts at delineating a set of tests for establishing confidence in an entirely expert-elicited model have focused on single types of validity stemming from individual sources of uncertainty within the model. This paper seeks to extend the frameworks proposed by earlier researchers by drawing upon other disciplines where measuring latent variables is also an issue. We demonstrate that even in cases where no data exist at all there is a broad range of validity tests that can be used to establish confidence in the validity of a Bayesian Belief Network.

Validation of scales measuring self-efficacy and outcome expectancy in evidence-based practice

Background: Evidence-based practice (EBP) is embraced internationally as an ideal approach to improve patient outcomes and provide cost-effective care. However, despite the support for and apparent benefits of evidence-based practice, it has been shown to be complex and difficult to incorporate into the clinical setting. Research exploring implementation of evidence-based practice has highlighted many internal and external barriers including clinicians’ lack of knowledge and confidence to integrate EBP into their day-to-day work. Nurses in particular often feel ill-equipped with little confidence to find, appraise and implement evidence. Aims: The following study aimed to undertake preliminary testing of the psychometric properties of tools that measure nurses’ self-efficacy and outcome expectancy in regard to evidence-based practice. Methods: A survey design was utilised in which nurses who had either completed an EBP unit or were randomly selected from a major tertiary referral hospital in Brisbane, Australia were sent two newly developed tools: 1) Self-efficacy in Evidence-Based Practice (SE-EBP) scale and 2) Outcome Expectancy for Evidence-Based Practice (OE-EBP) scale. Results: Principal Axis Factoring found three factors with eigenvalues above one for the SE-EBP explaining 73% of the variance and one factor for the OE-EBP scale explaining 82% of the variance. Cronbach’s alpha for SE-EBP, three SE-EBP factors and OE-EBP were all >.91 suggesting some item redundancy. The SE-EBP was able to distinguish between those with no prior exposure to EBP and those who completed an introductory EBP unit. Conclusions: While further investigation of the validity of these tools is needed, preliminary testing indicates that the SE-EBP and OE-EBP scales are valid and reliable instruments for measuring health professionals’ confidence in the process and the outcomes of basing their practice on evidence.

Validation of a new hand-held electronic data capture method for continuous monitoring of subjective appetite sensations

Background When large scale trials are investigating the effects of interventions on appetite, it is paramount to efficiently monitor large amounts of human data. The original hand-held Electronic Appetite Ratings System (EARS) was designed to facilitate the administering and data management of visual analogue scales (VAS) of subjective appetite sensations. The purpose of this study was to validate a novel hand-held method (EARS II (HP® iPAQ)) against the standard Pen and Paper (P&P) method and the previously validated EARS. Methods Twelve participants (5 male, 7 female, aged 18-40) were involved in a fully repeated measures design. Participants were randomly assigned in a crossover design, to either high fat (>48% fat) or low fat (<28% fat) meal days, one week apart and completed ratings using the three data capture methods ordered according to Latin Square. The first set of appetite sensations was completed in a fasted state, immediately before a fixed breakfast. Thereafter, appetite sensations were completed every thirty minutes for 4h. An ad libitum lunch was provided immediately before completing a final set of appetite sensations. Results Repeated measures ANOVAs were conducted for ratings of hunger, fullness and desire to eat. There were no significant differences between P&P compared with either EARS or EARS II (p > 0.05). Correlation coefficients between P&P and EARS II, controlling for age and gender, were performed on Area Under the Curve ratings. R2 for Hunger (0.89), Fullness (0.96) and Desire to Eat (0.95) were statistically significant (p < 0.05). Conclusions EARS II was sensitive to the impact of a meal and recovery of appetite during the postprandial period and is therefore an effective device for monitoring appetite sensations. This study provides evidence and support for further validation of the novel EARS II method for monitoring appetite sensations during large scale studies. The added versatility means that future uses of the system provides the potential to monitor a range of other behavioural and physiological measures often important in clinical and free living trials.

Validation of de-identified record linkage to ascertain hospital admissions in a cohort study

Background Cohort studies can provide valuable evidence of cause and effect relationships but are subject to loss of participants over time, limiting the validity of findings. Computerised record linkage offers a passive and ongoing method of obtaining health outcomes from existing routinely collected data sources. However, the quality of record linkage is reliant upon the availability and accuracy of common identifying variables. We sought to develop and validate a method for linking a cohort study to a state-wide hospital admissions dataset with limited availability of unique identifying variables. Methods A sample of 2000 participants from a cohort study (n = 41 514) was linked to a state-wide hospitalisations dataset in Victoria, Australia using the national health insurance (Medicare) number and demographic data as identifying variables. Availability of the health insurance number was limited in both datasets; therefore linkage was undertaken both with and without use of this number and agreement tested between both algorithms. Sensitivity was calculated for a sub-sample of 101 participants with a hospital admission confirmed by medical record review. Results Of the 2000 study participants, 85% were found to have a record in the hospitalisations dataset when the national health insurance number and sex were used as linkage variables and 92% when demographic details only were used. When agreement between the two methods was tested the disagreement fraction was 9%, mainly due to "false positive" links when demographic details only were used. A final algorithm that used multiple combinations of identifying variables resulted in a match proportion of 87%. Sensitivity of this final linkage was 95%. Conclusions High quality record linkage of cohort data with a hospitalisations dataset that has limited identifiers can be achieved using combinations of a national health insurance number and demographic data as identifying variables.

Preliminary validation of a novel high-resolution melt-based typing method based on the multilocus sequence typing scheme of Streptococcus pyogenes

The major limitation of current typing methods for Streptococcus pyogenes, such as emm sequence typing and T typing, is that these are based on regions subject to considerable selective pressure. Multilocus sequence typing (MLST) is a better indicator of the genetic backbone of a strain but is not widely used due to high costs. The objective of this study was to develop a robust and cost-effective alternative to S. pyogenes MLST. A 10-member single nucleotide polymorphism (SNP) set that provides a Simpson’s Index of Diversity (D) of 0.99 with respect to the S. pyogenes MLST database was derived. A typing format involving high-resolution melting (HRM) analysis of small fragments nucleated by each of the resolution-optimized SNPs was developed. The fragments were 59–119 bp in size and, based on differences in G+C content, were predicted to generate three to six resolvable HRM curves. The combination of curves across each of the 10 fragments can be used to generate a melt type (MelT) for each sequence type (ST). The 525 STs currently in the S. pyogenes MLST database are predicted to resolve into 298 distinct MelTs and the method is calculated to provide a D of 0.996 against the MLST database. The MelTs are concordant with the S. pyogenes population structure. To validate the method we examined clinical isolates of S. pyogenes of 70 STs. Curves were generated as predicted by G+C content discriminating the 70 STs into 65 distinct MelTs.

Measuring literalism and symbolism in advertisements : scale development and validation

A basic element in advertising strategy is the choice of an appeal. Many researchers have studied communication message form and specifically forms of literalism and symbolism, or some variation. The motives for such study are grounded in increasing the effectiveness of commercial communication messages, especially advertising messages. Advertising research studies typically use forms of literalism (e.g. informativeness) or symbolism (e.g. metaphoric, tropes, schemes figures of speech, and rhetorical figures) as independent variables and compare these against one or more of the traditional advertising effectiveness measures as dependent variable(s). The main challenge in assessing the effectiveness of literalism or symbolism in message content is the discreet identification of the construct. However, no standard, empirically-tested measure was located in the literature.

An assessment tool for selection of appropriate daylighting solutions for buildings in tropical and subtropical regions : Validation using radiance simulation

A new decision-making tool that will assist designers in the selection of appropriate daylighting solutions for buildings in tropical locations has been previously proposed by the authors. Through an evaluation matrix that prioritizes the parameters that best respond to the needs of tropical climates (e.g. reducing solar gain and protection from glare) the tool determines the most appropriate devices for specific climate and building inputs. The tool is effective in demonstrating the broad benefits and limitations of the different daylight strategies for buildings in the tropics. However for thorough analysis and calibration of the tool, validation is necessary. This paper presents a first step in the validation process. RADIANCE simulations were conducted to compare simulation performance with the performance predicted by the tool. To this end, an office building case study in subtropical Brisbane, Australia, and five different daylighting devices including openings, light guiding systems and light transport systems were simulated. Illuminance, light uniformity, daylight penetration and glare analysis were assessed for each device. The results indicate the tool can appropriately rank and recommend daylighting strategies based on specific building inputs for tropical and subtropical regions, making it a useful resource for designers.

Development and validation of the consumer disillusionment toward marketing activity scale

This study examines disillusioned consumers. The theory proposes that this is a group learning to lower their expectations of firm integrity and who, to avoid being let down, ignore marketing activity directly from the firm. This kind of exchange orientation develops as a response to consistent failure in perceptions of firm integrity. The research includes six studies, including over 600 adult consumers, to outline the development and validation of a measure of consumer disillusionment toward marketing activity. Completing the process provides a valid and reliable four-item measure. In addition, the study includes the assessment of the nomological validity of the construct. The nomological validation includes using cue utilization theory to predict that disillusioned consumers favor advertising that provides evidence of verifiable integrity. The validation experiment uses print advertising containing high and low verifiable integrity stimuli. Results confirm the theory with disillusioned consumers focusing less on the firm as source of information. Further, these consumers respond more favorably than non-disillusioned consumers to third party endorsers who serve to verify the firm's attempts to show integrity.

Open design of high pressure ratio radial-inflow turbine for academic validation

Computational Fluid Dynamics (CFD) simulations are widely used in mechanical engineering. Although achieving a high level of confidence in numerical modelling is of crucial importance in the field of turbomachinery, verification and validation of CFD simulations are very tricky especially for complex flows encountered in radial turbines. Comprehensive studies of radial machines are available in the literature. Unfortunately, none of them include enough detailed geometric data to be properly reproduced and so cannot be considered for academic research and validation purposes. As a consequence, design improvements of such configurations are difficult. Moreover, it seems that well-developed analyses of radial turbines are used in commercial software but are not available in the open literature especially at high pressure ratios. It is the purpose of this paper to provide a fully open set of data to reproduce the exact geometry of the high pressure ratio single stage radial-inflow turbine used in the Sundstrand Power Systems T-100 Multipurpose Small Power Unit. First, preliminary one-dimensional meanline design and analysis are performed using the commercial software RITAL from Concepts-NREC in order to establish a complete reference test case available for turbomachinery code validation. The proposed design of the existing turbine is then carefully and successfully checked against the geometrical and experimental data partially published in the literature. Then, three-dimensional Reynolds-Averaged Navier-Stokes simulations are conducted by means of the Axcent-PushButton CFDR CFD software. The effect of the tip clearance gap is investigated in detail for a wide range of operating conditions. The results confirm that the 3D geometry is correctly reproduced. It also reveals that the turbine is shocked while designed to give a high-subsonic flow and highlight the importance of the diffuser.

A framework for a dense matching algorithm : Results using matching metrics and order statistic filters

An algorithm for computing dense correspondences between images of a stereo pair or image sequence is presented. The algorithm can make use of both standard matching metrics and the rank and census filters, two filters based on order statistics which have been applied to the image matching problem. Their advantages include robustness to radiometric distortion and amenability to hardware implementation. Results obtained using both real stereo pairs and a synthetic stereo pair with ground truth were compared. The rank and census filters were shown to significantly improve performance in the case of radiometric distortion. In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics. For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small.

Order statistic filters for image matching

The rank and census are two filters based on order statistics which have been applied to the image matching problem for stereo pairs. Advantages of these filters include their robustness to radiometric distortion and small amounts of random noise, and their amenability to hardware implementation. In this paper, a new matching algorithm is presented, which provides an overall framework for matching, and is used to compare the rank and census techniques with standard matching metrics. The algorithm was tested using both real stereo pairs and a synthetic pair with ground truth. The rank and census filters were shown to significantly improve performance in the case of radiometric distortion. In all cases, the results obtained were comparable to, if not better than, those obtained using standard matching metrics. Furthermore, the rank and census have the additional advantage that their computational overhead is less than these metrics. For all techniques tested, the difference between the results obtained for the synthetic stereo pair, and the ground truth results was small.

Assessing the overuse of antibiotics in children in Saudi Arabia : validation of the parental perception on antibiotics scale (PAPA scale)

Background Antibiotics overuse is a global public health issue influenced by several factors, of which some are parent-related psychosocial factors that can only be measured using valid and reliable psychosocial measurement instruments. The PAPA scale was developed to measure these factors and the content validity of this instrument was assessed. Aim This study further validated the recently developed instrument in terms of (1) face validity and (2) construct validity including: deciding the number and nature of factors, and item selection. Methods Questionnaires were self-administered to parents of children between the ages of 0 and 12 years old. Parents were conveniently recruited from schools’ parental meetings in the Eastern Province, Saudi Arabia. Face validity was assessed with regards to questionnaire clarity and unambiguity. Construct validity and item selection processes were conducted using Exploratory factor analysis. Results Parallel analysis and Exploratory factor analysis using principal axis factoring produced six factors in the developed instrument: knowledge and beliefs, behaviours, sources of information, adherence, awareness about antibiotics resistance, and parents’ perception regarding doctors’ prescribing behaviours. Reliability was assessed (Cronbach’s alpha = 0.78) which demonstrates the instrument as being reliable. Conclusion The ‘factors’ produced in this study coincide with the constructs contextually identified in the development phase of other instruments used to study antibiotic use. However, no other study considering perceptions of antibiotic use had gone beyond content validation of such instruments. This study is the first to constructively validate the factors underlying perceptions regarding antibiotic use in any population and in parents in particular.

Validation of the Educational Stress Scale for Adolescents (ESSA) in Vietnam

Introduction To date, there has been little systematic, quantitative research on the links between academic pressure and mental health among adolescents in Asia, and none in Vietnam. In part, this is because of a lack of appropriate tools to measure this complex phenomenon. This study was to validate the Educational Stress Scale for Adolescents (ESSA), developed and tested in China, with the aim of fostering further research in Asia. Methods A total of 1283 students were recruited in 3 secondary schools and 3 high schools in Ho Chi Minh City, Vietnam. Anonymous, selfreport questionnaires included the ESSA and previously validated measures of mental health. Results Among the 1226 questionnaires available, 54% of respondents were female. The mean age was 15.3 years. Students reported substantial study burden. The ESSA had good internal consistency, and factorial validity and concurrent validity were established. Conclusion The ESSA is a suitable measure for school-based mental health research in Asia.

Experimental and computer simulation validation of ultrasound phase interference created by lateral inhomogeneity of transit time in replica bone marrow composite models

Purpose: The measurement of broadband ultrasonic attenuation (BUA) in cancellous bone for the assessment of osteoporosis follows a parabolic-type dependence with bone volume fraction; having minima values corresponding to both entire bone and entire marrow. Langton has recently proposed that the primary BUA mechanism may be significant phase interference due to variations in propagation transit time through the test sample as detected over the phase-sensitive surface of the receive ultrasound transducer. This fundamentally simple concept assumes that the propagation of ultrasound through a complex solid : liquid composite sample such as cancellous bone may be considered by an array of parallel ‘sonic rays’. The transit time of each ray is defined by the proportion of bone and marrow propagated, being a minimum (tmin) solely through bone and a maximum (tmax) solely through marrow. A Transit Time Spectrum (TTS), ranging from tmin to tmax, may be defined describing the proportion of sonic rays having a particular transit time, effectively describing lateral inhomogeneity of transit time over the surface of the receive ultrasound transducer. Phase interference may result from interaction of ‘sonic rays’ of differing transit times. The aim of this study was to test the hypothesis that there is a dependence of phase interference upon the lateral inhomogenity of transit time by comparing experimental measurements and computer simulation predictions of ultrasound propagation through a range of relatively simplistic solid:liquid models exhibiting a range of lateral inhomogeneities. Methods: A range of test models was manufactured using acrylic and water as surrogates for bone and marrow respectively. The models varied in thickness in one dimension normal to the direction of propagation, hence exhibiting a range of transit time lateral inhomogeneities, ranging from minimal (single transit time) to maximal (wedge; ultimately the limiting case where each sonic ray has a unique transit time). For the experimental component of the study, two unfocused 1 MHz ¾” broadband diameter transducers were utilized in transmission mode; ultrasound signals were recorded for each of the models. The computer simulation was performed with Matlab, where the transit time and relative amplitude of each sonic ray was calculated. The transit time for each sonic ray was defined as the sum of transit times through acrylic and water components. The relative amplitude considered the reception area for each sonic ray along with absorption in the acrylic. To replicate phase-sensitive detection, all sonic rays were summed and the output signal plotted in comparison with the experimentally derived output signal. Results: From qualtitative and quantitative comparison of the experimental and computer simulation results, there is an extremely high degree of agreement of 94.2% to 99.0% between the two approaches, supporting the concept that propagation of an ultrasound wave, for the models considered, may be approximated by a parallel sonic ray model where the transit time of each ray is defined by the proportion of ‘bone’ and ‘marrow’. Conclusions: This combined experimental and computer simulation study has successfully demonstrated that lateral inhomogeneity of transit time has significant potential for phase interference to occur if a phase-sensitive ultrasound receive transducer is implemented as in most commercial ultrasound bone analysis devices.