Definizione e validazione di uno strumento per la misurazione dell'impatto delle variabili redazionali di un item matematico sulla performance degli studenti

Questa tesi descrive la ricerca condotta tra l'autunno e l'inverno di quest'anno da un gruppo di ricercatori in didattica della matematica relativamente all'influenza che le variazioni redazionali di un quesito matematico hanno sulle performance degli studenti. Lo scopo della ricerca è quella di strutturare e validare una metodologia e uno strumento che permettano di individuare e quantificare l'influenza delle variazioni del testo sulle prestazioni dello studente. Si è sentita l'esigenza di condurre uno studio di questo tipo poichè è sempre più evidente il profondo legame tra il linguaggio e l'apprendimento della matematica. La messa a punto di questo strumento aprirebbe le porte a una serie di ricerche più approfondite sulle varie tipologie di variazioni numeriche e/o linguistiche finora individuate. Nel primo capitolo è presentato il quadro teorico di riferimento relativo agli studi condotti fino ad ora nell'ambito della didattica della matematica, dai quali emerge la grossa influenza che la componente linguistica ha sulla comprensione e la trasmissione della matematica. Si farà quindi riferimento alle ricerche passate volte all'individuazione e alla schematizzazione delle variazioni redazionali dei Word Problems. Nel secondo capitolo, invece si passerà alla descrizione teorica relativa allo strumento statistico utilizzato. Si tratta del modello di Rasch appartenente alla famiglia dei modelli statistici dell'Item Response Theory, particolarmente utilizzato nella ricerca in didattica. Il terzo capitolo sarà dedicato alla descrizione dettagliata della sperimentazione svolta. Il quarto capitolo sarà il cuore di questa tesi; in esso infatti verrà descritta e validata la nuova metodologia utilizzata. Nel quinto sarà eseguita un analisi puntuale di come lo strumento ha messo in evidenza le differenze per ogni item variato. Infine verranno tratte le conclusioni complessive dello studio condotto.

Measuring nursing competencies in the operating theatre: instrument development and psychometric analysis using Item Response Theory

BACKGROUND: Concern about the process of identifying underlying competencies that contribute to effective nursing performance has been debated with a lack of consensus surrounding an approved measurement instrument for assessing clinical performance. Although a number of methodologies are noted in the development of competency-based assessment measures, these studies are not without criticism. RESEARCH AIM: The primary aim of the study was to develop and validate a Performance Based Scoring Rubric, which included both analytical and holistic scales. The aim included examining the validity and reliability of the rubric, which was designed to measure clinical competencies in the operating theatre. RESEARCH METHOD: The fieldwork observations of 32 nurse educators and preceptors assessing the performance of 95 instrument nurses in the operating theatre were used in the calibration of the rubric. The Rasch model, a particular model among Item Response Models, was used in the calibration of each item in the rubric in an attempt at improving the measurement properties of the scale. This is done by establishing the 'fit' of the data to the conditions demanded by the Rasch model. RESULTS: Acceptable reliability estimates, specifically a high Cronbach's alpha reliability coefficient (0.940), as well as empirical support for construct and criterion validity for the rubric were achieved. Calibration of the Performance Based Scoring Rubric using Rasch model revealed that the fit statistics for most items were acceptable. CONCLUSION: The use of the Rasch model offers a number of features in developing and refining healthcare competency-based assessments, improving confidence in measuring clinical performance. The Rasch model was shown to be useful in developing and validating a competency-based assessment for measuring the competence of the instrument nurse in the operating theatre with implications for use in other areas of nursing practice.

Rasch analysis of the routines-based interview implementation checklist

The Routines-Based Interview (RBI) is useful for developing functional outcomes/goals, for establishing strong relationships with families, and for assessing the family’s true needs. In this study, the authors investigated the psychometric properties of the RBI Implementation Checklist, conducted by 120 early intervention professionals, specifically looking at the probability of correct responses on the items as a logistic function of the difference between the person and the item parameters.We selected Rasch analysis (Rasch, 1980) for this study so we could answer questions about both how themeasure performed and how the interviewers performed and we related these performances to one another. Results indicate that scores on the RBI Implementation Checklist were reliable. The checklist could possibly benefit from more difficult items to measure the true performance of the few people who had scores higher than the most difficult items and also from additional items that focus on the family.

Deriving a preference-based measure for cancer using the EORTC QLQ-C30 : a confirmatory versus exploratory approach

Background: To derive preference-based measures from various condition-specific descriptive health-related quality of life (HRQOL) measures. A general 2-stage method is evolved: 1) an item from each domain of the HRQOL measure is selected to form a health state classification system (HSCS); 2) a sample of health states is valued and an algorithm derived for estimating the utility of all possible health states. The aim of this analysis was to determine whether confirmatory or exploratory factor analysis (CFA, EFA) should be used to derive a cancer-specific utility measure from the EORTC QLQ-C30. Methods: Data were collected with the QLQ-C30v3 from 356 patients receiving palliative radiotherapy for recurrent or metastatic cancer (various primary sites). The dimensional structure of the QLQ-C30 was tested with EFA and CFA, the latter based on a conceptual model (the established domain structure of the QLQ-C30: physical, role, emotional, social and cognitive functioning, plus several symptoms) and clinical considerations (views of both patients and clinicians about issues relevant to HRQOL in cancer). The dimensions determined by each method were then subjected to item response theory, including Rasch analysis. Results: CFA results generally supported the proposed conceptual model, with residual correlations requiring only minor adjustments (namely, introduction of two cross-loadings) to improve model fit (increment χ2(2) = 77.78, p < .001). Although EFA revealed a structure similar to the CFA, some items had loadings that were difficult to interpret. Further assessment of dimensionality with Rasch analysis aligned the EFA dimensions more closely with the CFA dimensions. Three items exhibited floor effects (>75% observation at lowest score), 6 exhibited misfit to the Rasch model (fit residual > 2.5), none exhibited disordered item response thresholds, 4 exhibited DIF by gender or cancer site. Upon inspection of the remaining items, three were considered relatively less clinically important than the remaining nine. Conclusions: CFA appears more appropriate than EFA, given the well-established structure of the QLQ-C30 and its clinical relevance. Further, the confirmatory approach produced more interpretable results than the exploratory approach. Other aspects of the general method remain largely the same. The revised method will be applied to a large number of data sets as part of the international and interdisciplinary project to develop a multi-attribute utility instrument for cancer (MAUCa).

Development and validation of the ankle fracture outcome of rehabilitation measure (A-FORM)

Study Design Delphi panel and cohort study. Objective To develop and refine a condition-specific, patient-reported outcome measure, the Ankle Fracture Outcome of Rehabilitation Measure (A-FORM), and to examine its psychometric properties, including factor structure, reliability, and validity, by assessing item fit with the Rasch model. Background To our knowledge, there is no patient-reported outcome measure specific to ankle fracture with a robust content foundation. Methods A 2-stage research design was implemented. First, a Delphi panel that included patients and health professionals developed the items and refined the item wording. Second, a cohort study (n = 45) with 2 assessment points was conducted to permit preliminary maximum-likelihood exploratory factor analysis and Rasch analysis. Results The Delphi panel reached consensus on 53 potential items that were carried forward to the cohort phase. From the 2 time points, 81 questionnaires were completed and analyzed; 38 potential items were eliminated on account of greater than 10% missing data, factor loadings, and uniqueness. The 15 unidimensional items retained in the scale demonstrated appropriate person and item reliability after (and before) removal of 1 item (anxious about footwear) that had a higher-than-ideal outfit statistic (1.75). The “anxious about footwear” item was retained in the instrument, but only the 14 items with acceptable infit and outfit statistics (range, 0.5–1.5) were included in the summary score. Conclusion This investigation developed and refined the A-FORM (Version 1.0). The A-FORM items demonstrated favorable psychometric properties and are suitable for conversion to a single summary score. Further studies utilizing the A-FORM instrument are warranted. J Orthop Sports Phys Ther 2014;44(7):488–499. Epub 22 May 2014. doi:10.2519/jospt.2014.4980

Analisi di un impianto solare termico a circolazione forzata per la produzione di acqua calda sanitaria in un condominio

Il presente elaborato propone un’analisi in regime dinamico mediante il software di simulazione TRNSYS di un impianto solare termico per la produzione di acqua calda sanitaria istallato in un condominio a Bologna. Le simulazioni rappresentano una particolare tipologia di esperimento numerico e possono fornire lo stesso tipo di informazioni sulle performance termiche dei comuni esperimenti fisici. Esse possono valutare gli effetti delle variabili di progetto sulle performance del sistema attraverso una serie di esperimenti, eseguiti tutti in identiche condizioni metereologiche e di carico. Le variabili di progetto esaminate in questo documento mediante simulazione sono l’inclinazione e orientamento del collettore, la superficie captante totale istallata, il volume di accumulo e la portata massima del circuito solare. Le analisi esposte sono un esempio dei risultati che un approccio simulativo può garantire; nel nostro caso, in particolare, le simulazioni hanno confermato la bontà dei criteri di dimensionamento degli impianti solari termici che è possibile incontrare in letteratura. Il presente elaborato si propone inoltre di discutere i benefici, ambientali ed economici, legati all’investimento proposto. L’analisi economica, in particolare, è stata eseguita a seconda dell’impianto tradizionale da affiancare, del diverso costo dell’impianto, di differenti valori di superficie captante e considerando o meno gli incentivi fiscali offerti: per questa valutazione è stato nuovamente necessario l’ausilio del software di simulazione.

The use of scoring rubrics to determine clinical performance in the operating suite

Summary : This research evolved out of the need to examine the validity and inter-rater reliability of a set of performance-based scoring rubrics designed to measure competencies within the operating suite.

Method : Both holistic and analytical rubrics were developed aligned to the ACORN Standard [Australian College of Operating Room Nurses Standard NR4, 2004. ACORN Competency Standards for Perioperative Nurses: Standard NR4: The Instrument Nurse in the Perioperative Environment. Australian College of Operating Room Nurses Ltd, Adelaide] and underpinned by the Dreyfus model (1981). Three video clips that captured varying performance of nurses performing as instrument nurses in the operating suite were recorded and used as prompts by expert raters, who judged the performance using the rubrics.

Results : The study found that the holistic rubrics led to more consistent judgments than the analytical rubrics, yet the latter provided more diagnostic information for intervention purposes. Despite less consistency, the Analytical Observation Form had sufficient construct validity to satisfy the requirements of criterion referencing as determined by the Item Separation Index (Rasch, 1960), including high internal consistency and greater inter-rater reliability when average ratings were used.

Conclusion : The study was an empirical investigation of the use of concomitant Analytical and Holistic Rubrics to determine various levels of performance in the operating suite including inter-rater reliability. The methodology chosen was theoretically sound and sufficiently flexible to be used to develop other competencies within the operating suite.

Performance of the assessment of quality of life measure in people with hip and knee joint disease and implications for research and clinical use

Objective
To comprehensively evaluate the performance of the Assessment of Quality of Life (AQoL) instrument for measuring health-related quality of life (HRQOL) in people with hip and knee joint disease (arthritis or osteoarthritis).

Methods
Data from 237 individuals were available for analysis from a national cross-sectional, population-based study of hip and knee joint disease in Australia. AQoL-4D data were evaluated using Rasch analysis. A range of measurement properties was explored, including model and item fit, threshold ordering, differential item functioning, and targeting.

Results
Good overall fit of the AQoL with the Rasch model was demonstrated across a range of tests, supporting internal validity. Only 1 item (relating to hearing) showed evidence of misfit. Most AQoL items showed logical sequencing of response option categories, with threshold disordering evident for only 2 of the 12 items (items 4 and 9). Minor issues with potential clinical and research implications include limited options for reporting pain and some evidence of measurement bias between demographic subgroups (including age and sex). Participants' HRQOL was generally better than that represented by the AQoL items (mean ± SD for person abilities −2.15 ± 1.39, mean ± SD for item difficulties 0.00 ± 0.67), indicating ceiling effects that could impact the instrument's ability to detect HRQOL improvement in population-based studies.

Conclusion
The AQoL is a competent tool for assessing HRQOL in people with hip and knee joint disease, although researchers and clinicians should consider the caveats identified when selecting appropriate HRQOL measures for future outcome assessment involving this patient group.

L’effetto “età della Terra”. Contratto didattico e principi regolativi dell’azione degli studenti in matematica.

L’oggetto di questa tesi è un fenomeno didattico osservato in due valutazioni standardizzate nazionali INVALSI, legato all’atteggiamento degli studenti mentre svolgono task di matematica. L’effetto, che abbiamo denotato effetto “età della Terra”, è stato interpretato in questa ricerca attraverso l’interazione e il confronto di diversi costrutti teorici che spiegano come questo effetto, che può essere considerato come una tipica situazione di contratto didattico, è generato dalla relazione studente-insegnante ma può diventare più strettamente legato al rapporto che hanno gli studenti con la matematica. Inizialmente abbiamo condotto uno studio dei risultati statistici delle valutazioni standardizzate nazionali (Rash Analysis). Il primo step della sperimentazione è consistito nella preparazione, validazione e somministrazione di 612 questionari a studenti di diversi livelli scolastici e basandoci sui risultati dei questionari abbiamo condotto interviste di gruppo. L’analisi quantitativa e qualitativa dei risultati ha confermato la presenza dell’effetto “età della Terra” e ha mostrato che questo effetto è indipendente dal livello scolastico e dall’età degli studenti, dal contenuto matematico e dal contesto dei task proposti. La seconda parte della ricerca è stata volta ad indagare la cause di questo effetto. Abbiamo infatti individuato un principio regolativo che condizione l’azione degli studenti mentre fanno attività matematica e abbiamo condotto molte interviste individuali per indagarlo. Il comportamento degli studenti intervistati è stato così studiato e classificato con i costrutti del quadro teorico.

The eight-item modified Medical Outcomes Study Social Support Survey: psychometric evaluation showed excellent performance

Evaluation and validation of the psychometric properties of the eight-item modified Medical Outcomes Study Social Support Survey (mMOS-SS).

Validation of the Action Research Arm Test using item response theory in patients after stroke

Objective: To validate the unidimensionality of the Action Research Arm Test (ARAT) using Mokken analysis and to examine whether scores of the ARAT can be transformed into interval scores using Rasch analysis. Subjects and methods: A total of 351 patients with stroke were recruited from 5 rehabilitation departments located in 4 regions of Taiwan. The 19-item ARAT was administered to all the subjects by a physical therapist. The data were analysed using item response theory by non-parametric Mokken analysis followed by Rasch analysis. Results: The results supported a unidimensional scale of the 19-item ARAT by Mokken analysis, with the scalability coefficient H = 0.95. Except for the item pinch ball bearing 3rd finger and thumb'', the remaining 18 items have a consistently hierarchical order along the upper extremity function's continuum. In contrast, the Rasch analysis, with a stepwise deletion of misfit items, showed that only 4 items (grasp ball'', grasp block 5 cm(3)'', grasp block 2.5 cm(3)'', and grip tube 1 cm(3)'') fit the Rasch rating scale model's expectations. Conclusion: Our findings indicated that the 19-item ARAT constituted a unidimensional construct measuring upper extremity function in stroke patients. However, the results did not support the premise that the raw sum scores of the ARAT can be transformed into interval Rasch scores. Thus, the raw sum scores of the ARAT can provide information only about order of patients on their upper extremity functional abilities, but not represent each patient's exact functioning.

X-ray your data with Rasch

By using the Rasch model, much detailed diagnostic information is available to developers of survey and assessment instruments and to the researchers who use them. We outline an approach to the analysis of data obtained from the administration of survey instruments that can enable researchers to recognise and diagnose difficulties with those instruments and then to suggest remedial actions that can improve the measurement properties of the scales included in questionnaires. We illustrate the approach using examples drawn from recent research and demonstrate how the approach can be used to generate figures that make the results of Rasch analyses accessible to non-specialists.

The role of fluency in a mathematics item with an embedded graphic: Interpreting a pie chart.

The purpose of this study was to identify the pedagogical knowledge relevant to the successful completion of a pie chart item. This purpose was achieved through the identification of the essential fluencies that 12–13-year-olds required for the successful solution of a pie chart item. Fluency relates to ease of solution and is particularly important in mathematics because it impacts on performance. Although the majority of students were successful on this multiple choice item, there was considerable divergence in the strategies they employed. Approximately two-thirds of the students employed efficient multiplicative strategies, which recognised and capitalised on the pie chart as a proportional representation. In contrast, the remaining one-third of students used a less efficient additive strategy that failed to capitalise on the representation of the pie chart. The results of our investigation of students’ performance on the pie chart item during individual interviews revealed that five distinct fluencies were involved in the solution process: conceptual (understanding the question), linguistic (keywords), retrieval (strategy selection), perceptual (orientation of a segment of the pie chart) and graphical (recognising the pie chart as a proportional representation). In addition, some students exhibited mild disfluencies corresponding to the five fluencies identified above. Three major outcomes emerged from the study. First, a model of knowledge of content and students for pie charts was developed. This model can be used to inform instruction about the pie chart and guide strategic support for students. Second, perceptual and graphical fluency were identified as two aspects of the curriculum, which should receive a greater emphasis in the primary years, due to their importance in interpreting pie charts. Finally, a working definition of fluency in mathematics was derived from students’ responses to the pie chart item.

Dramatic teaching : focussing on drama, theatre, and performance in higher education

This 90 minute panel session is designed to explore issues relating to the teaching of drama, performance studies, and theatre studies within Higher Education. Some of the issues that will be raised include: developing an understanding of the learning that students believe they are experiencing through performance; contemporary models for teaching; and the suggestion that the body can be an important site for acquiring a variety of different knowledges. Paul Makeham will present a general position paper to commence the session (15 minutes). Maryrose Casey, Gillian Kehoul, and Delyse Ryan will each speak briefly (15 minutes) about aspects of their research into Higher Education teaching before opening the floor for a round-table discussion of issues affecting the teaching of these disciplines.