An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education.

BACKGROUND We evaluated a newly designed electronic portfolio (e-Portfolio) that provided quantitative evaluation of surgical skills. Medical students at the University of Seville used the e-Portfolio on a voluntary basis for evaluation of their performance in undergraduate surgical subjects. METHODS Our new web-based e-Portfolio was designed to evaluate surgical practical knowledge and skills targets. Students recorded each activity on a form, attached evidence, and added their reflections. Students self-assessed their practical knowledge using qualitative criteria (yes/no), and graded their skills according to complexity (basic/advanced) and participation (observer/assistant/independent). A numerical value was assigned to each activity, and the values of all activities were summated to obtain the total score. The application automatically displayed quantitative feedback. We performed qualitative evaluation of the perceived usefulness of the e-Portfolio and quantitative evaluation of the targets achieved. RESULTS Thirty-seven of 112 students (33%) used the e-Portfolio, of which 87% reported that they understood the methodology of the portfolio. All students reported an improved understanding of their learning objectives resulting from the numerical visualization of progress, all students reported that the quantitative feedback encouraged their learning, and 79% of students felt that their teachers were more available because they were using the e-Portfolio. Only 51.3% of students reported that the reflective aspects of learning were useful. Individual students achieved a maximum of 65% of the total targets and 87% of the skills targets. The mean total score was 345 ± 38 points. For basic skills, 92% of students achieved the maximum score for participation as an independent operator, and all achieved the maximum scores for participation as an observer and assistant. For complex skills, 62% of students achieved the maximum score for participation as an independent operator, and 98% achieved the maximum scores for participation as an observer or assistant. CONCLUSIONS Medical students reported that use of an electronic portfolio that provided quantitative feedback on their progress was useful when the number and complexity of targets were appropriate, but not when the portfolio offered only formative evaluations based on reflection. Students felt that use of the e-Portfolio guided their learning process by indicating knowledge gaps to themselves and teachers.

How to Measure Scope Variables when no Metrics Exist : Application to Landscape Quality Measurement and Hedonic Price Evaluation

Landscape is an example of a non-market good where no metrics exist to measure its quality. The paper proposes an original methodology to nevertheless estimate scope variables in those circumstances, allowing then to better test if people's willingnesstopay for such good is sensitive to the scope. The methodology is based on techniques developed in the context of multicriteria decision analysis. It is applied to assess the quality of the landscape of several Swiss alpine resorts. This assessment is then used as an explanatory variable in a hedonic price function to explain the rent of apartments and to derive an implicit price of the landscape quality.

Bladder cancer index: cross-cultural adaptation into Spanish and psychometric evaluation.

BACKGROUND The Bladder Cancer Index (BCI) is so far the only instrument applicable across all bladder cancer patients, independent of tumor infiltration or treatment applied. We developed a Spanish version of the BCI, and assessed its acceptability and metric properties. METHODS For the adaptation into Spanish we used the forward and back-translation method, expert panels, and cognitive debriefing patient interviews. For the assessment of metric properties we used data from 197 bladder cancer patients from a multi-center prospective study. The Spanish BCI and the SF-36 Health Survey were self-administered before and 12 months after treatment. Reliability was estimated by Cronbach's alpha. Construct validity was assessed through the multi-trait multi-method matrix. The magnitude of change was quantified by effect sizes to assess responsiveness. RESULTS Reliability coefficients ranged 0.75-0.97. The validity analysis confirmed moderate associations between the BCI function and bother subscales for urinary (r = 0.61) and bowel (r = 0.53) domains; conceptual independence among all BCI domains (r ≤ 0.3); and low correlation coefficients with the SF-36 scores, ranging 0.14-0.48. Among patients reporting global improvement at follow-up, pre-post treatment changes were statistically significant for the urinary domain and urinary bother subscale, with effect sizes of 0.38 and 0.53. CONCLUSIONS The Spanish BCI is well accepted, reliable, valid, responsive, and similar in performance compared to the original instrument. These findings support its use, both in Spanish and international studies, as a valuable and comprehensive tool for assessing quality of life across a wide range of bladder cancer patients.

Infornut® Process; improves accessibility to diagnosis and nutritional support for the malnourished hospitalized patient; impact on management indicators; two-year assessment.

Introduction: The high prevalence of disease-related hospital malnutrition justifies the need for screening tools and early detection in patients at risk for malnutrition, followed by an assessment targeted towards diagnosis and treatment. At the same time there is clear undercoding of malnutrition diagnoses and the procedures to correct it Objectives: To describe the INFORNUT program/ process and its development as an information system. To quantify performance in its different phases. To cite other tools used as a coding source. To calculate the coding rates for malnutrition diagnoses and related procedures. To show the relationship to Mean Stay, Mortality Rate and Urgent Readmission; as well as to quantify its impact on the hospital Complexity Index and its effect on the justification of Hospitalization Costs. Material and methods: The INFORNUT® process is based on an automated screening program of systematic detection and early identification of malnourished patients on hospital admission, as well as their assessment, diagnoses, documentation and reporting. Of total readmissions with stays longer than three days incurred in 2008 and 2010, we recorded patients who underwent analytical screening with an alert for a medium or high risk of malnutrition, as well as the subgroup of patients in whom we were able to administer the complete INFORNUT® process, generating a report for each.

Liver safety assessment: required data elements and best practices for data collection and standardization in clinical trials.

A workshop was convened to discuss best practices for the assessment of drug-induced liver injury (DILI) in clinical trials. In a breakout session, workshop attendees discussed necessary data elements and standards for the accurate measurement of DILI risk associated with new therapeutic agents in clinical trials. There was agreement that in order to achieve this goal the systematic acquisition of protocol-specified clinical measures and lab specimens from all study subjects is crucial. In addition, standard DILI terms that address the diverse clinical and pathologic signatures of DILI were considered essential. There was a strong consensus that clinical and lab analyses necessary for the evaluation of cases of acute liver injury should be consistent with the US Food and Drug Administration (FDA) guidance on pre-marketing risk assessment of DILI in clinical trials issued in 2009. A recommendation that liver injury case review and management be guided by clinicians with hepatologic expertise was made. Of note, there was agreement that emerging DILI signals should prompt the systematic collection of candidate pharmacogenomic, proteomic and/or metabonomic biomarkers from all study subjects. The use of emerging standardized clinical terminology, CRFs and graphic tools for data review to enable harmonization across clinical trials was strongly encouraged. Many of the recommendations made in the breakout session are in alignment with those made in the other parallel sessions on methodology to assess clinical liver safety data, causality assessment for suspected DILI, and liver safety assessment in special populations (hepatitis B, C, and oncology trials). Nonetheless, a few outstanding issues remain for future consideration.

An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education.

BACKGROUND We evaluated a newly designed electronic portfolio (e-Portfolio) that provided quantitative evaluation of surgical skills. Medical students at the University of Seville used the e-Portfolio on a voluntary basis for evaluation of their performance in undergraduate surgical subjects. METHODS Our new web-based e-Portfolio was designed to evaluate surgical practical knowledge and skills targets. Students recorded each activity on a form, attached evidence, and added their reflections. Students self-assessed their practical knowledge using qualitative criteria (yes/no), and graded their skills according to complexity (basic/advanced) and participation (observer/assistant/independent). A numerical value was assigned to each activity, and the values of all activities were summated to obtain the total score. The application automatically displayed quantitative feedback. We performed qualitative evaluation of the perceived usefulness of the e-Portfolio and quantitative evaluation of the targets achieved. RESULTS Thirty-seven of 112 students (33%) used the e-Portfolio, of which 87% reported that they understood the methodology of the portfolio. All students reported an improved understanding of their learning objectives resulting from the numerical visualization of progress, all students reported that the quantitative feedback encouraged their learning, and 79% of students felt that their teachers were more available because they were using the e-Portfolio. Only 51.3% of students reported that the reflective aspects of learning were useful. Individual students achieved a maximum of 65% of the total targets and 87% of the skills targets. The mean total score was 345 ± 38 points. For basic skills, 92% of students achieved the maximum score for participation as an independent operator, and all achieved the maximum scores for participation as an observer and assistant. For complex skills, 62% of students achieved the maximum score for participation as an independent operator, and 98% achieved the maximum scores for participation as an observer or assistant. CONCLUSIONS Medical students reported that use of an electronic portfolio that provided quantitative feedback on their progress was useful when the number and complexity of targets were appropriate, but not when the portfolio offered only formative evaluations based on reflection. Students felt that use of the e-Portfolio guided their learning process by indicating knowledge gaps to themselves and teachers.

Dummy run and conformity indices in the ongoing EORTC low-grade glioma trial 22033-26033: First evaluation of quality of radiotherapy planning.

PURPOSE: Early assessment of radiotherapy (RT) quality in the ongoing EORTC trial comparing primary temozolomide versus RT in low-grade gliomas. MATERIALS AND METHODS: RT plans provided for dummy cases were evaluated and compared against expert plans. We analysed: (1) tumour and organs-at-risk delineation, (2) geometric and dosimetric characteristics, (3) planning parameters, compliance with dose prescription and Dmax for OAR (4) indices: RTOG conformity index (CI), coverage factor (CF), tissue protection factor (PF); conformity number (CN = PF x CF); dose homogeneity in PTV (U). RESULTS: Forty-one RT plans were evaluated. Only two (5%) centres were requested to repeat CTV-PTV delineations. Three (7%) plans had a significant under-dosage and dose homogeneity in one deviated > 10%. Dose distribution was good with mean values of 1.5, 1, 0.68, and 0.68 (ideal values = 1) for CI, CF, PF, and CN, respectively. CI and CN strongly correlated with PF and they correlated with PTV. Planning with more beams seems to increase PTV(Dmin), improving CF. U correlated with PTV(Dmax). CONCLUSION: Preliminary results of the dummy run procedure indicate that most centres conformed to protocol requirements. To quantify plan quality we recommend systematic calculation of U and either CI or CN, both of which measure the amount of irradiated normal brain tissue.

Assessment of statistical iterative reconstructions to optimize CT examination protocols for children and young adults

La tomodensitométrie (CT) est une technique d'imagerie dont l'intérêt n'a cessé de croître depuis son apparition dans le début des années 70. Dans le domaine médical, son utilisation est incontournable à tel point que ce système d'imagerie pourrait être amené à devenir victime de son succès si son impact au niveau de l'exposition de la population ne fait pas l'objet d'une attention particulière. Bien évidemment, l'augmentation du nombre d'examens CT a permis d'améliorer la prise en charge des patients ou a rendu certaines procédures moins invasives. Toutefois, pour assurer que le compromis risque - bénéfice soit toujours en faveur du patient, il est nécessaire d'éviter de délivrer des doses non utiles au diagnostic.¦Si cette action est importante chez l'adulte elle doit être une priorité lorsque les examens se font chez l'enfant, en particulier lorsque l'on suit des pathologies qui nécessitent plusieurs examens CT au cours de la vie du patient. En effet, les enfants et jeunes adultes sont plus radiosensibles. De plus, leur espérance de vie étant supérieure à celle de l'adulte, ils présentent un risque accru de développer un cancer radio-induit dont la phase de latence peut être supérieure à vingt ans. Partant du principe que chaque examen radiologique est justifié, il devient dès lors nécessaire d'optimiser les protocoles d'acquisitions pour s'assurer que le patient ne soit pas irradié inutilement. L'avancée technologique au niveau du CT est très rapide et depuis 2009, de nouvelles techniques de reconstructions d'images, dites itératives, ont été introduites afin de réduire la dose et améliorer la qualité d'image.¦Le présent travail a pour objectif de déterminer le potentiel des reconstructions itératives statistiques pour réduire au minimum les doses délivrées lors d'examens CT chez l'enfant et le jeune adulte tout en conservant une qualité d'image permettant le diagnostic, ceci afin de proposer des protocoles optimisés.¦L'optimisation d'un protocole d'examen CT nécessite de pouvoir évaluer la dose délivrée et la qualité d'image utile au diagnostic. Alors que la dose est estimée au moyen d'indices CT (CTDIV0| et DLP), ce travail a la particularité d'utiliser deux approches radicalement différentes pour évaluer la qualité d'image. La première approche dite « physique », se base sur le calcul de métriques physiques (SD, MTF, NPS, etc.) mesurées dans des conditions bien définies, le plus souvent sur fantômes. Bien que cette démarche soit limitée car elle n'intègre pas la perception des radiologues, elle permet de caractériser de manière rapide et simple certaines propriétés d'une image. La seconde approche, dite « clinique », est basée sur l'évaluation de structures anatomiques (critères diagnostiques) présentes sur les images de patients. Des radiologues, impliqués dans l'étape d'évaluation, doivent qualifier la qualité des structures d'un point de vue diagnostique en utilisant une échelle de notation simple. Cette approche, lourde à mettre en place, a l'avantage d'être proche du travail du radiologue et peut être considérée comme méthode de référence.¦Parmi les principaux résultats de ce travail, il a été montré que les algorithmes itératifs statistiques étudiés en clinique (ASIR?, VEO?) ont un important potentiel pour réduire la dose au CT (jusqu'à-90%). Cependant, par leur fonctionnement, ils modifient l'apparence de l'image en entraînant un changement de texture qui pourrait affecter la qualité du diagnostic. En comparant les résultats fournis par les approches « clinique » et « physique », il a été montré que ce changement de texture se traduit par une modification du spectre fréquentiel du bruit dont l'analyse permet d'anticiper ou d'éviter une perte diagnostique. Ce travail montre également que l'intégration de ces nouvelles techniques de reconstruction en clinique ne peut se faire de manière simple sur la base de protocoles utilisant des reconstructions classiques. Les conclusions de ce travail ainsi que les outils développés pourront également guider de futures études dans le domaine de la qualité d'image, comme par exemple, l'analyse de textures ou la modélisation d'observateurs pour le CT.¦-¦Computed tomography (CT) is an imaging technique in which interest has been growing since it first began to be used in the early 1970s. In the clinical environment, this imaging system has emerged as the gold standard modality because of its high sensitivity in producing accurate diagnostic images. However, even if a direct benefit to patient healthcare is attributed to CT, the dramatic increase of the number of CT examinations performed has raised concerns about the potential negative effects of ionizing radiation on the population. To insure a benefit - risk that works in favor of a patient, it is important to balance image quality and dose in order to avoid unnecessary patient exposure.¦If this balance is important for adults, it should be an absolute priority for children undergoing CT examinations, especially for patients suffering from diseases requiring several follow-up examinations over the patient's lifetime. Indeed, children and young adults are more sensitive to ionizing radiation and have an extended life span in comparison to adults. For this population, the risk of developing cancer, whose latency period exceeds 20 years, is significantly higher than for adults. Assuming that each patient examination is justified, it then becomes a priority to optimize CT acquisition protocols in order to minimize the delivered dose to the patient. Over the past few years, CT advances have been developing at a rapid pace. Since 2009, new iterative image reconstruction techniques, called statistical iterative reconstructions, have been introduced in order to decrease patient exposure and improve image quality.¦The goal of the present work was to determine the potential of statistical iterative reconstructions to reduce dose as much as possible without compromising image quality and maintain diagnosis of children and young adult examinations.¦The optimization step requires the evaluation of the delivered dose and image quality useful to perform diagnosis. While the dose is estimated using CT indices (CTDIV0| and DLP), the particularity of this research was to use two radically different approaches to evaluate image quality. The first approach, called the "physical approach", computed physical metrics (SD, MTF, NPS, etc.) measured on phantoms in well-known conditions. Although this technique has some limitations because it does not take radiologist perspective into account, it enables the physical characterization of image properties in a simple and timely way. The second approach, called the "clinical approach", was based on the evaluation of anatomical structures (diagnostic criteria) present on patient images. Radiologists, involved in the assessment step, were asked to score image quality of structures for diagnostic purposes using a simple rating scale. This approach is relatively complicated to implement and also time-consuming. Nevertheless, it has the advantage of being very close to the practice of radiologists and is considered as a reference method.¦Primarily, this work revealed that the statistical iterative reconstructions studied in clinic (ASIR? and VECO have a strong potential to reduce CT dose (up to -90%). However, by their mechanisms, they lead to a modification of the image appearance with a change in image texture which may then effect the quality of the diagnosis. By comparing the results of the "clinical" and "physical" approach, it was showed that a change in texture is related to a modification of the noise spectrum bandwidth. The NPS analysis makes possible to anticipate or avoid a decrease in image quality. This project demonstrated that integrating these new statistical iterative reconstruction techniques can be complex and cannot be made on the basis of protocols using conventional reconstructions. The conclusions of this work and the image quality tools developed will be able to guide future studies in the field of image quality as texture analysis or model observers dedicated to CT.

Activity assessment of eosinophilic esophagitis.

The activity of eosinophilic esophagitis (EoE) can be assessed with patient-reported outcomes and biologic measures. Patient-reported outcomes include symptoms and quality of life, whereas biologic measures refer to endoscopic, histologic, and biochemical activity (e.g. blood biomarkers). So far, a validated tool to assess EoE activity in the above-mentioned dimensions is lacking. Given the lack of a standardized way to assess EoE activity in the various dimensions, the results of different clinical trials may be difficult to compare. For symptom assessment in adult patients, the symptom 'dysphagia' should be evaluated according to different standardized food consistencies. Furthermore, symptom assessment should take into account the following items: avoidance of specific food categories, food modification, and time to eat a regular meal. A distinct symptom recall period (e.g. 2 weeks) has to be defined for symptom assessment. Performing an 'esophageal stress test' with ingestion of a standardized meal to measure symptom severity bears the potential risk of acute food bolus impaction and should therefore be avoided. The description of endoscopic findings in EoE has meanwhile been standardized. Histologic evaluation of EoE activity should report either the size of the high-power field used or count the eosinophils per mm(2). There is a current lack of blood biomarkers demonstrating a good correlation with histologic activity in esophageal biopsies. The development and validation of an adult and pediatric EoE activity index is urgently needed not only for clinical trials and observational studies, but also for daily practice.

Familial evaluation in catecholaminergic polymorphic ventricular tachycardia: disease penetrance and expression in cardiac ryanodine receptor mutation-carrying relatives.

BACKGROUND: Catecholaminergic polymorphic ventricular tachycardia (CPVT) is an inherited arrhythmia syndrome associated with mutations in the cardiac ryanodine receptor gene (Ryr2) in the majority of patients. Previous studies of CPVT patients mainly involved probands, so current insight into disease penetrance, expression, genotype-phenotype correlations, and arrhythmic event rates in relatives carrying the Ryr2 mutation is limited. METHODS AND RESULTS: One-hundred sixteen relatives carrying the Ryr2 mutation from 15 families who were identified by cascade screening of the Ryr2 mutation causing CPVT in the proband were clinically characterized, including 61 relatives from 1 family. Fifty-four of 108 antiarrhythmic drug-free relatives (50%) had a CPVT phenotype at the first cardiological examination, including 27 (25%) with nonsustained ventricular tachycardia. Relatives carrying a Ryr2 mutation in the C-terminal channel-forming domain showed an increased odds of nonsustained ventricular tachycardia (odds ratio, 4.1; 95% CI, 1.5-11.5; P=0.007, compared with N-terminal domain) compared with N-terminal domain. Sinus bradycardia was observed in 19% of relatives, whereas other supraventricular dysrhythmias were present in 16%. Ninety-eight (most actively treated) relatives (84%) were followed up for a median of 4.7 years (range, 0.3-19.0 years). During follow-up, 2 asymptomatic relatives experienced exercise-induced syncope. One relative was not being treated, whereas the other was noncompliant. None of the 116 relatives died of CPVT during a 6.7-year follow-up (range, 1.4-20.9 years). CONCLUSIONS: Relatives carrying an Ryr2 mutation show a marked phenotypic diversity. The vast majority do not have signs of supraventricular disease manifestations. Mutation location may be associated with severity of the phenotype. The arrhythmic event rate during follow-up was low.

Importance of databases for technology assessment.

The data indispensable for carrying out the comprehensive, multi-faceted process of medical technology assessment (MTA) should be collected from a variety of sources. The authors distinguish between type "A" general data, useful for assessment but collected without this specific aim, and type "B" data. Registries of health care procedures or of diseases, as well as clinical data bases are quoted as examples of type "B" data, specifically relating to MTA. Since demographic methods are of importance for the evaluation of long-term effects of medical technologies, examples of sources of type "A" data are presented. Their significance for health policy making is discussed.

Adaptive integral assessment package for the A2UN@ project

This paper presents a first approach of Evaluation Engine Architecture (EEA) as proposal to support adaptive integral assessment, in the context of a virtual learning environment. The goal of our research is design an evaluation engine tool to assist in the whole assessment process within the A2UN@ project, linking that tool with the other key elements of a learning design (learning task, learning resources and learning support). The teachers would define the relation between knowledge, competencies, activities, resources and type of assessment. Providing this relation is possible obtain more accurate estimations of student's knowledge for adaptive evaluations and future recommendations. The process is supported by usage of educational standards and specifications and for an integral user modelling

Tele-education and competencies assessment to Brazilian's auxiliary nurse

This paper presents Brazilian's experience with the organization of methods and strategies for the assessment of competencies for technical level of nursing workers. The evaluative process proposed includes the creation of a learning-oriented and distance-based virtual assessment environment. The proposed methodology for professional competencies assessment adopted a critical-emancipatory perspective. A tele-education environment was deployed, involving software development - a virtual man - and an assessment cybertutor. Learning modules for the cybertutor were developed and videos of clinical simulations, structured around assessment in cognitive, behavioral, and simulation areas. The evaluation modules considered aspects of competencies in know-know, know-how and know-act professional ethics. Also the variability of practices of nursing - hospitals and primary health care units - was considered. This instrument showed as an important strategy for the optimization of assessment procedures that are widely used across Brazil and it is a powerful tool for incorporation into the continuing professional education.

Image quality assessment in digital mammography: part I. Technical characterization of the systems.

In many European countries, image quality for digital x-ray systems used in screening mammography is currently specified using a threshold-detail detectability method. This is a two-part study that proposes an alternative method based on calculated detectability for a model observer: the first part of the work presents a characterization of the systems. Eleven digital mammography systems were included in the study; four computed radiography (CR) systems, and a group of seven digital radiography (DR) detectors, composed of three amorphous selenium-based detectors, three caesium iodide scintillator systems and a silicon wafer-based photon counting system. The technical parameters assessed included the system response curve, detector uniformity error, pre-sampling modulation transfer function (MTF), normalized noise power spectrum (NNPS) and detective quantum efficiency (DQE). Approximate quantum noise limited exposure range was examined using a separation of noise sources based upon standard deviation. Noise separation showed that electronic noise was the dominant noise at low detector air kerma for three systems; the remaining systems showed quantum noise limited behaviour between 12.5 and 380 µGy. Greater variation in detector MTF was found for the DR group compared to the CR systems; MTF at 5 mm(-1) varied from 0.08 to 0.23 for the CR detectors against a range of 0.16-0.64 for the DR units. The needle CR detector had a higher MTF, lower NNPS and higher DQE at 5 mm(-1) than the powder CR phosphors. DQE at 5 mm(-1) ranged from 0.02 to 0.20 for the CR systems, while DQE at 5 mm(-1) for the DR group ranged from 0.04 to 0.41, indicating higher DQE for the DR detectors and needle CR system than for the powder CR phosphor systems. The technical evaluation section of the study showed that the digital mammography systems were well set up and exhibiting typical performance for the detector technology employed in the respective systems.

Evaluation de la sarcopénie de la personne agée par la bio-impédance spectroscopique [Multispectral bioimpedance and sarcopenia in the elderly].

The definition of sarcopenia includes both a loss of muscle strength and a decline in functional quality in addition to the loss of muscle protein mass. Multispectral bioimpendance allows bedside assessment of muscle mass. Using this new tool, we performed a pilot study to look for a possible correlation between muscle mass and various tests of muscle strength (grip strength, key-pitch, tip-pinch) and with functional tests (walk speed on 10 meters and Tinetti test). Our study demonstrates a good correlation between muscle mass determined by spectroscopic bioimpendance and muscle strength assessment, but no correlation with functional tests.