Time Efficient Assessment and Feedback Methods for Large Computer-Aided-Design Cohorts

Timely and individualized feedback on coursework is desirable from a student perspective as it facilitates formative development and encourages reflective learning practice. Faculty however are faced with a significant and potentially time consuming challenge when teaching larger cohorts if they are to provide feedback which is timely, individualized and detailed. Additionally, for subjects which assess non-traditional submissions, such as Computer-Aided-Design (CAD), the methods for assessment and feedback tend not to be so well developed or optimized. Issues can also arise over the consistency of the feedback provided. Evaluations of Computer-Assisted feedback in other disciplines (Denton et al, 2008), (Croft et al, 2001) have shown students prefer this method of feedback to traditional “red pen” marking and also that such methods can be more time efficient for faculty.
Herein, approaches are described which make use of technology and additional software tools to speed up, simplify and automate assessment and the provision of feedback for large cohorts of first and second year engineering students studying modules where CAD files are submitted electronically. A range of automated methods are described and compared with more “manual” approaches. Specifically one method uses an application programming interface (API) to interrogate SolidWorks models and extract information into an Excel spreadsheet, which is then used to automatically send feedback emails. Another method describes the use of audio recordings made during model interrogation which reduces the amount of time while increasing the level of detail provided as feedback.
Limitations found with these methods and problems encountered are discussed along with a quantified assessment of time saving efficiencies made.

A Perspective on Computer Assisted Assessment Techniques for Short Free-Text Answers

Computer Assisted Assessment (CAA) has been existing for several years now. While some forms of CAA do not require sophisticated text understanding (e.g., multiple choice questions), there are also student answers that consist of free text and require analysis of text in the answer. Research towards the latter till date has concentrated on two main sub-tasks: (i) grading of essays, which is done mainly by checking the style, correctness of grammar, and coherence of the essay and (ii) assessment of short free-text answers. In this paper, we present a structured view of relevant research in automated assessment techniques for short free-text answers. We review papers spanning the last 15 years of research with emphasis on recent papers. Our main objectives are two folds. First we present the survey in a structured way by segregating information on dataset, problem formulation, techniques, and evaluation measures. Second we present a discussion on some of the potential future directions in this domain which we hope would be helpful for researchers.

Factors associated with suboptimal adherence to antiretroviral therapy in Viet Nam: A cross-sectional study using audio computer-assisted self-interview (ACASI)

Background: Optimal adherence to antiretroviral therapy (ART) is necessary for people living with HIV/AIDS (PLHIV). There have been relatively few systematic analyses of factors that promote or inhibit adherence to antiretroviral therapy among PLHIV in Asia. This study assessed ART adherence and examined factors associated with suboptimal adherence in northern Viet Nam. Methods: Data from 615 PLHIV on ART in two urban and three rural outpatient clinics were collected by medical record extraction and from patient interviews using audio computer-assisted self-interview (ACASI). Results: The prevalence of suboptimal adherence was estimated to be 24.9% via a visual analogue scale (VAS) of past-month dose-missing and 29.1% using a modified Adult AIDS Clinical Trial Group scale for on-time dose-taking in the past 4 days. Factors significantly associated with the more conservative VAS score were: depression (p < 0.001), side-effect experiences (p < 0.001), heavy alcohol use (p = 0.001), chance health locus of control (p = 0.003), low perceived quality of information from care providers (p = 0.04) and low social connectedness (p = 0.03). Illicit drug use alone was not significantly associated with suboptimal adherence, but interacted with heavy alcohol use to reduce adherence (p < 0.001). Conclusions: This is the largest survey of ART adherence yet reported from Asia and the first in a developing country to use the ACASI method in this context. The evidence strongly indicates that ART services in Viet Nam should include screening and treatment for depression, linkage with alcohol and/or drug dependence treatment, and counselling to address the belief that chance or luck determines health outcomes.

Problem solving and computer-assisted instruction in science education : analysis of research findings and the research process /

The quantitative component of this study examined the effect of computerassisted instruction (CAI) on science problem-solving performance, as well as the significance of logical reasoning ability to this relationship. I had the dual role of researcher and teacher, as I conducted the study with 84 grade seven students to whom I simultaneously taught science on a rotary-basis. A two-treatment research design using this sample of convenience allowed for a comparison between the problem-solving performance of a CAI treatment group (n = 46) versus a laboratory-based control group (n = 38). Science problem-solving performance was measured by a pretest and posttest that I developed for this study. The validity of these tests was addressed through critical discussions with faculty members, colleagues, as well as through feedback gained in a pilot study. High reliability was revealed between the pretest and the posttest; in this way, students who tended to score high on the pretest also tended to score high on the posttest. Interrater reliability was found to be high for 30 randomly-selected test responses which were scored independently by two raters (i.e., myself and my faculty advisor). Results indicated that the form of computer-assisted instruction (CAI) used in this study did not significantly improve students' problem-solving performance. Logical reasoning ability was measured by an abbreviated version of the Group Assessment of Lx)gical Thinking (GALT). Logical reasoning ability was found to be correlated to problem-solving performance in that, students with high logical reasoning ability tended to do better on the problem-solving tests and vice versa. However, no significant difference was observed in problem-solving improvement, in the laboratory-based instruction group versus the CAI group, for students varying in level of logical reasoning ability.Insignificant trends were noted in results obtained from students of high logical reasoning ability, but require further study. It was acknowledged that conclusions drawn from the quantitative component of this study were limited, as further modifications of the tests were recommended, as well as the use of a larger sample size. The purpose of the qualitative component of the study was to provide a detailed description ofmy thesis research process as a Brock University Master of Education student. My research journal notes served as the data base for open coding analysis. This analysis revealed six main themes which best described my research experience: research interests, practical considerations, research design, research analysis, development of the problem-solving tests, and scoring scheme development. These important areas ofmy thesis research experience were recounted in the form of a personal narrative. It was noted that the research process was a form of problem solving in itself, as I made use of several problem-solving strategies to achieve desired thesis outcomes.

Adaptive computer assisted assessment of free-text students answers : an approach to automatically generate students' conceptual models.

Las formas de evaluación basadas en el uso de tests no pueden identificar muchos errores conceptuales de los estudiantes. Esta investigación tiene como objetivo facilitar un nuevo procedimiento capaz de generar los modelos conceptuales de los estudiantes de forma automática a partir de respuestas en texto libre.. Este trabajo se organiza en tres apartados. En primer lugar, se procede a la revisión del estado de la cuestión. A continuación se describen el procedimiento para generar automáticamente los modelos conceptuales de los estudiantes y los sistemas que implementan dicho procedimiento. Por último se ofrecen: una explicación de los experimentos realizados y sus resultados, las conclusiones obtenidas y las líneas de trabajo futuro. Además se proporciona información para aplicar el procedimiento en otro idioma y/o área de conocimiento.. Se propone un procedimiento para generar automáticamente modelos conceptuales de cada estudiante y de una clase a partir de las respuestas facilitadas al sistema de evaluación automático y adaptativo. Estos sistemas son la evolución de los actuales de evaluación de respuestas en texto libre, que evalúan respuestas en texto libre automáticamente y de forma adaptada al modelo de cada estudiante..

The impact of computer-assisted learning on academic grades: an assessment of students' perceptions

This study examines student perceptions of the usefulness of Computer-Assisted Learning (CAL) packages in learning accounting concepts in terms of the influence on academic performance. Various additional factors  affecting academic performance [such as gender, prior studies of  accounting, and computer systems, together with entry background] are incorporated in the development of a multiple regression model, together with perceptions of CAL. The study uses a sample of 280 second-year undergraduate accounting students from an Australian university to test the model. In contrast to prior studies (e.g. Lane and Porch, 2002, Accounting Education: an international journal, 11(3), pp. 217-233), this study showed that positive perceptions of the usefulness of CAL significantly influenced performance. Additionally, it was found that international students, many of whom enter university at the second year level having obtained advanced standing credits, had significantly poorer performance than local students. The findings show that gender, prior studies of accounting and computing systems were not significant influences on academic performance. Overall, the results have implications for accounting educators utilising CAL in courses as a means of improving students' understanding of accounting concepts and academic performance.

Correlation properties of spontaneous motor activity in healthy infants: a new computer-assisted method to evaluate neurological maturation

Qualitative assessment of spontaneous motor activity in early infancy is widely used in clinical practice. It enables the description of maturational changes of motor behavior in both healthy infants and infants who are at risk for later neurological impairment. These assessments are, however, time-consuming and are dependent upon professional experience. Therefore, a simple physiological method that describes the complex behavior of spontaneous movements (SMs) in infants would be helpful. In this methodological study, we aimed to determine whether time series of motor acceleration measurements at 40-44 weeks and 50-55 weeks gestational age in healthy infants exhibit fractal-like properties and if this self-affinity of the acceleration signal is sensitive to maturation. Healthy motor state was ensured by General Movement assessment. We assessed statistical persistence in the acceleration time series by calculating the scaling exponent α via detrended fluctuation analysis of the time series. In hand trajectories of SMs in infants we found a mean α value of 1.198 (95 % CI 1.167-1.230) at 40-44 weeks. Alpha changed significantly (p = 0.001) at 50-55 weeks to a mean of 1.102 (1.055-1.149). Complementary multilevel regression analysis confirmed a decreasing trend of α with increasing age. Statistical persistence of fluctuation in hand trajectories of SMs is sensitive to neurological maturation and can be characterized by a simple parameter α in an automated and observer-independent fashion. Future studies including children at risk for neurological impairment should evaluate whether this method could be used as an early clinical screening tool for later neurological compromise.

Lung cancer screening with CT: evaluation of radiologists and different computer assisted detection software (CAD) as first and second readers for lung nodule detection at different dose levels

OBJECTIVES To find the best pairing of first and second reader at highest sensitivity for detecting lung nodules with CT at various dose levels. MATERIALS AND METHODS An anthropomorphic lung phantom and artificial lung nodules were used to simulate screening CT-examination at standard dose (100 mAs, 120 kVp) and 8 different low dose levels, using 120, 100 and 80 kVp combined with 100, 50 and 25 mAs. At each dose level 40 phantoms were randomly filled with 75 solid and 25 ground glass nodules (5-12 mm). Two radiologists and 3 different computer aided detection softwares (CAD) were paired to find the highest sensitivity. RESULTS Sensitivities at standard dose were 92%, 90%, 84%, 79% and 73% for reader 1, 2, CAD1, CAD2, CAD3, respectively. Combined sensitivity for human readers 1 and 2 improved to 97%, (p1=0.063, p2=0.016). Highest sensitivities--between 97% and 99.0%--were achieved by combining any radiologist with any CAD at any dose level. Combining any two CADs, sensitivities between 85% and 88% were significantly lower than for radiologists combined with CAD (p<0.03). CONCLUSIONS Combination of a human observer with any of the tested CAD systems provide optimal sensitivity for lung nodule detection even at reduced dose at 25 mAs/80 kVp.

Accuracy of computer-aided design/computer-assisted manufacture (CAD/CAM) fabricated dental restorations: a comparative study

Introduction: Computer-Aided-Design (CAD) and Computer-Aided-Manufacture (CAM) has been developed to fabricate fixed dental restorations accurately, faster and improve cost effectiveness of manufacture when compared to the conventional method. Two main methods exist in dental CAD/CAM technology: the subtractive and additive methods. While fitting accuracy of both methods has been explored, no study yet has compared the fabricated restoration (CAM output) to its CAD in terms of accuracy. The aim of this present study was to compare the output of various dental CAM routes to a sole initial CAD and establish the accuracy of fabrication. The internal fit of the various CAM routes were also investigated. The null hypotheses tested were: 1) no significant differences observed between the CAM output to the CAD and 2) no significant differences observed between the various CAM routes. Methods: An aluminium master model of a standard premolar preparation was scanned with a contact dental scanner (Incise, Renishaw, UK). A single CAD was created on the scanned master model (InciseCAD software, V2.5.0.140, UK). Twenty copings were then fabricated by sending the single CAD to a multitude of CAM routes. The copings were grouped (n=5) as: Laser sintered CoCrMo (LS), 5-axis milled CoCrMo (MCoCrMo), 3-axis milled zirconia (ZAx3) and 4-axis milled zirconia (ZAx4). All copings were micro-CT scanned (Phoenix X-Ray, Nanotom-S, Germany, power: 155kV, current: 60µA, 3600 projections) to produce 3-Dimensional (3D) models. A novel methodology was created to superimpose the micro-CT scans with the CAD (GOM Inspect software, V7.5SR2, Germany) to indicate inaccuracies in manufacturing. The accuracy in terms of coping volume was explored. The distances from the surfaces of the micro-CT 3D models to the surfaces of the CAD model (CAD Deviation) were investigated after creating surface colour deviation maps. Localised digital sections of the deviations (Occlusal, Axial and Cervical) and selected focussed areas were then quantitatively measured using software (GOM Inspect software, Germany). A novel methodology was also explored to digitally align (Rhino software, V5, USA) the micro-CT scans with the master model to investigate internal fit. Fifty digital cross sections of the aligned scans were created. Point-to-point distances were measured at 5 levels at each cross section. The five levels were: Vertical Marginal Fit (VF), Absolute Marginal Fit (AM), Axio-margin Fit (AMF), Axial Fit (AF) and Occlusal Fit (OF). Results: The results of the volume measurement were summarised as: VM-CoCrMo (62.8mm3 ) > VZax3 (59.4mm3 ) > VCAD (57mm3 ) > VZax4 (56.1mm3 ) > VLS (52.5mm3 ) and were all significantly different (p presented as areas with different colour. No significant differences were observed at the internal aspect of the cervical aspect between all groups of copings. Significant differences (p< M-CoCrMo Internal Occlusal, Internal Axial and External Axial 2 ZAx3 > ZAx4 External Occlusal, External Cervical 3 ZAx3 < ZAx4 Internal Occlusal 4 M-CoCrMo > ZAx4 Internal Occlusal and Internal Axial The mean values of AMF and AF were significantly (p M-CoCrMo and CAD > ZAx4. Only VF of M-CoCrMo was comparable with the CAD Internal Fit. All VF and AM values were within the clinically acceptable fit (120µm). Conclusion: The investigated CAM methods reproduced the CAD accurately at the internal cervical aspect of the copings. However, localised deviations at axial and occlusal aspects of the copings may suggest the need for modifications in these areas prior to fitting and veneering with porcelain. The CAM groups evaluated also showed different levels of Internal Fit thus rejecting the null hypotheses. The novel non-destructive methodologies for CAD/CAM accuracy and internal fit testing presented in this thesis may be a useful evaluation tool for similar applications.

Computer-assisted measurements of coronal knee joint laxity in vitro are related to low-stress behavior rather than structural properties of the collateral ligaments

The relationship between coronal knee laxity and the restraining properties of the collateral ligaments remains unknown. This study investigated correlations between the structural properties of the collateral ligaments and stress angles used in computer-assisted total knee arthroplasty (TKA), measured with an optically based navigation system. Ten fresh-frozen cadaveric knees (mean age: 81 ± 11 years) were dissected to leave the menisci, cruciate ligaments, posterior joint capsule and collateral ligaments. The resected femur and tibia were rigidly secured within a test system which permitted kinematic registration of the knee using a commercially available image-free navigation system. Frontal plane knee alignment and varus-valgus stress angles were acquired. The force applied during varus-valgus testing was quantified. Medial and lateral bone-collateral ligament-bone specimens were then prepared, mounted within a uni-axial materials testing machine, and extended to failure. Force and displacement data were used to calculate the principal structural properties of the ligaments. The mean varus laxity was 4 ± 1° and the mean valgus laxity was 4 ± 2°. The corresponding mean manual force applied was 10 ± 3 N and 11 ± 4 N, respectively. While measures of knee laxity were independent of the ultimate tensile strength and stiffness of the collateral ligaments, there was a significant correlation between the force applied during stress testing and the instantaneous stiffness of the medial (r = 0.91, p = 0.001) and lateral (r = 0.68, p = 0.04) collateral ligaments. These findings suggest that clinicians may perceive a rate of change of ligament stiffness as the end-point during assessment of collateral knee laxity.