Automated Assessment of Student Activity in Moodle

Report published in the Proceedings of the National Conference on "Education and Research in the Information Society", Plovdiv, May, 2014

Adaptive computer assisted assessment of free-text students answers : an approach to automatically generate students' conceptual models.

Las formas de evaluación basadas en el uso de tests no pueden identificar muchos errores conceptuales de los estudiantes. Esta investigación tiene como objetivo facilitar un nuevo procedimiento capaz de generar los modelos conceptuales de los estudiantes de forma automática a partir de respuestas en texto libre.. Este trabajo se organiza en tres apartados. En primer lugar, se procede a la revisión del estado de la cuestión. A continuación se describen el procedimiento para generar automáticamente los modelos conceptuales de los estudiantes y los sistemas que implementan dicho procedimiento. Por último se ofrecen: una explicación de los experimentos realizados y sus resultados, las conclusiones obtenidas y las líneas de trabajo futuro. Además se proporciona información para aplicar el procedimiento en otro idioma y/o área de conocimiento.. Se propone un procedimiento para generar automáticamente modelos conceptuales de cada estudiante y de una clase a partir de las respuestas facilitadas al sistema de evaluación automático y adaptativo. Estos sistemas son la evolución de los actuales de evaluación de respuestas en texto libre, que evalúan respuestas en texto libre automáticamente y de forma adaptada al modelo de cada estudiante..

Integrating eco-efficiency assessment of commercial buildings into the design process : LCADesign

The ability to assess a commercial building for its impact on the environment at the earliest stage of design is a goal which is achievable by integrating several approaches into a single procedure directly from the 3D CAD representation. Such an approach enables building design professionals to make informed decisions on the environmental impact of building and its alternatives during the design development stage instead of at the post-design stage where options become limited. The indicators of interest are those which relate to consumption of resources and energy, contributions to pollution of air, water and soil, and impacts on the health and wellbeing of people in the built environment as a result of constructing and operating buildings. 3D object-oriented CAD files contain a wealth of building information which can be interrogated for details required for analysis of the performance of a design. The quantities of all components in the building can be automatically obtained from the 3D CAD objects and their constituent materials identified to calculate a complete list of the amounts of all building products such as concrete, steel, timber, plastic etc. When this information is combined with a life cycle inventory database, key internationally recognised environmental indicators can be estimated. Such a fully integrated tool known as LCADesign has been created for automated ecoefficiency assessment of commercial buildings direct from 3D CAD. This paper outlines the key features of LCADesign and its application to environmental assessment of commercial buildings.

LCADesign : an integrated approach to automatic eco-efficiency assessment of commercial buildings

Buildings consume resources and energy, contribute to pollution of our air, water and soil, impact the health and well-being of populations and constitute an important part of the built environment in which we live. The ability to assess their design with a view to reducing that impact automatically from their 3D CAD representations enables building design professionals to make informed decisions on the environmental impact of building structures. Contemporary 3D object-oriented CAD files contain a wealth of building information. LCADesign has been designed as a fully integrated approach for automated eco-efficiency assessment of commercial buildings direct from 3D CAD. LCADesign accesses the 3D CAD detail through Industry Foundation Classes (IFCs) - the international standard file format for defining architectural and constructional CAD graphic data as 3D real-world objects - to permit construction professionals to interrogate these intelligent drawing objects for analysis of the performance of a design. The automated take-off provides quantities of all building components whose specific production processes, logistics and raw material inputs, where necessary, are identified to calculate a complete list of quantities for all products such as concrete, steel, timber, plastic etc and combines this information with the life cycle inventory database, to estimate key internationally recognised environmental indicators such as CML, EPS and Eco-indicator 99. This paper outlines the key modules of LCADesign and their role in delivering an automated eco-efficiency assessment for commercial buildings.

Assessment of commercial network RTK user positioning performance over long inter-station distances

The paper provides an assessment of the performance of commercial Real Time Kinematic (RTK) systems over longer than recommended inter-station distances. The experiments were set up to test and analyse solutions from the i-MAX, MAX and VRS systems being operated with three triangle shaped network cells, each having an average inter-station distance of 69km, 118km and 166km. The performance characteristics appraised included initialization success rate, initialization time, RTK position accuracy and availability, ambiguity resolution risk and RTK integrity risk in order to provide a wider perspective of the performance of the testing systems. ----- ----- The results showed that the performances of all network RTK solutions assessed were affected by the increase in the inter-station distances to similar degrees. The MAX solution achieved the highest initialization success rate of 96.6% on average, albeit with a longer initialisation time. Two VRS approaches achieved lower initialization success rate of 80% over the large triangle. In terms of RTK positioning accuracy after successful initialisation, the results indicated a good agreement between the actual error growth in both horizontal and vertical components and the accuracy specified in the RMS and part per million (ppm) values by the manufacturers. ----- ----- Additionally, the VRS approaches performed better than the MAX and i-MAX when being tested under the standard triangle network with a mean inter-station distance of 69km. However as the inter-station distance increases, the network RTK software may fail to generate VRS correction and then may turn to operate in the nearest single-base RTK (or RAW) mode. The position uncertainty reached beyond 2 meters occasionally, showing that the RTK rover software was using an incorrect ambiguity fixed solution to estimate the rover position rather than automatically dropping back to using an ambiguity float solution. Results identified that the risk of incorrectly resolving ambiguities reached 18%, 20%, 13% and 25% for i-MAX, MAX, Leica VRS and Trimble VRS respectively when operating over the large triangle network. Additionally, the Coordinate Quality indicator values given by the Leica GX1230 GG rover receiver tended to be over-optimistic and not functioning well with the identification of incorrectly fixed integer ambiguity solutions. In summary, this independent assessment has identified some problems and failures that can occur in all of the systems tested, especially when being pushed beyond the recommended limits. While such failures are expected, they can offer useful insights into where users should be wary and how manufacturers might improve their products. The results also demonstrate that integrity monitoring of RTK solutions is indeed necessary for precision applications, thus deserving serious attention from researchers and system providers.

A hierarchical security assessment model for object-oriented programs

We present a hierarchical model for assessing an object-oriented program's security. Security is quantified using structural properties of the program code to identify the ways in which `classified' data values may be transferred between objects. The model begins with a set of low-level security metrics based on traditional design characteristics of object-oriented classes, such as data encapsulation, cohesion and coupling. These metrics are then used to characterise higher-level properties concerning the overall readability and writability of classified data throughout the program. In turn, these metrics are then mapped to well-known security design principles such as `assigning the least privilege' and `reducing the size of the attack surface'. Finally, the entire program's security is summarised as a single security index value. These metrics allow different versions of the same program, or different programs intended to perform the same task, to be compared for their relative security at a number of different abstraction levels. The model is validated via an experiment involving five open source Java programs, using a static analysis tool we have developed to automatically extract the security metrics from compiled Java bytecode.

Preliminary development of a sewerage infrastructure buffer assessment tool for engineering risk and strategic land use planning

Urban expansion continues to encroach on once isolated sewerage infrastructure. In this context,legislation and guidelines provide limited direction to the amenity allocation of appropriate buffer distances for land use planners and infrastructure providers. Topography, wind speed and direction,temperature, humidity, existing land uses and vegetation profiles are some of the factors that require investigation in analytically determining a basis for buffer separations. This paper discusses the compilation and analysis of six years of Logan sewerage odour complaint data. Graphically,relationships between the complaints, topographical features and meteorological data are presented. Application of a buffer sizing process could assist planners and infrastructure designers alike, whilst automatically providing extra green spaces. Establishing a justifiable criterion for buffer zone allocations can only assist in promoting manageable growth for healthier and more sustainable communities.

Quantitative fit assessment of a precontoured fracture fixation plate : its automation and an investigation on the borderline cases

Virtual methods to assess the fitting of a fracture fixation plate were proposed recently, however with limitations such as simplified fit criteria or manual data processing. This study aims to automate a fit analysis procedure using clinical-based criteria, and then to analyse the results further for borderline fit cases. Three dimensional (3D) models of 45 bones and of a precontoured distal tibial plate were utilized to assess the fitting of the plate automatically. A Matlab program was developed to automatically measure the shortest distance between the bone and the plate at three regions of interest and a plate-bone angle. The measured values including the fit assessment results were recorded in a spreadsheet as part of the batch-process routine. An automated fit analysis procedure will enable the processing of larger bone datasets in a significantly shorter time, which will provide more representative data of the target population for plate shape design and validation. As a result, better fitting plates can be manufactured and made available to surgeons, thereby reducing the risk and cost associated with complications or corrective procedures. This in turn, is expected to translate into improving patients' quality of life.

Security assessment of code refactoring rules

Refactoring is a common approach to producing better quality software. Its impact on many software quality properties, including reusability, maintainability and performance, has been studied and measured extensively. However, its impact on the information security of programs has received relatively little attention. In this work, we assess the impact of a number of the most common code-level refactoring rules on data security, using security metrics that are capable of measuring security from the viewpoint of potential information flow. The metrics are calculated for a given Java program using a static analysis tool we have developed to automatically analyse compiled Java bytecode. We ran our Java code analyser on various programs which were refactored according to each rule. New values of the metrics for the refactored programs then confirmed that the code changes had a measurable effect on information security.

Can we safely deform a plate to fit every bone? Population-based fit assessment and finite element deformation of a distal tibial plate

Anatomically precontoured plates are commonly used to treat periarticular fractures. A well-fitting plate can be used as a tool for anatomical reduction of the fractured bone. Recent studies highlighted that some plates fit poorly for many patients due to considerable shape variations between bones of the same anatomical site. While it is impossible to design one shape that fits all, it is also burdensome for the manufacturers and hospitals to produce, store and manage multiple plate shapes without the certainty of utilization by a patient population. In this study, we investigated the number of shapes required for maximum fit within a given dataset, and if they could be obtained by manually deforming the original plate. A distal medial tibial plate was automatically positioned on 45 individual tibiae, and the optimal deformation was determined iteratively using finite element analysis simulation. Within the studied dataset, we found that: (i) 89% fit could be achieved with four shapes, (ii) 100% fit was impossible through mechanical deformation, and (iii) the deformations required to obtain the four plate shapes were safe for the stainless steel plate for further clinical use. The proposed framework is easily transferable to other orthopaedic plates.

Machine vision based concrete surface quality assessment

Manually inspecting concrete surface defects (e.g., cracks and air pockets) is not always reliable. Also, it is labor-intensive. In order to overcome these limitations, automated inspection using image processing techniques was proposed. However, the current work can only detect defects in an image without the ability of evaluating them. This paper presents a novel approach for automatically assessing the impact of two common surface defects (i.e., air pockets and discoloration). These two defects are first located using the developed detection methods. Their attributes, such as the number of air pockets and the area of discoloration regions, are then retrieved to calculate defects’ visual impact ratios (VIRs). The appropriate threshold values for these VIRs are selected through a manual rating survey. This way, for a given concrete surface image, its quality in terms of air pockets and discoloration can be automatically measured by judging whether their VIRs are below the threshold values or not. The method presented in this paper was implemented in C++ and a database of concrete surface images was tested to validate its performance. Read More: http://ascelibrary.org/doi/abs/10.1061/%28ASCE%29CO.1943-7862.0000126?journalCode=jcemd4

Detecting air pockets for architectural concrete quality assessment using visual sensing

Air pockets, one kind of concrete surface defects, are often created on formed concrete surfaces during concrete construction. Their existence undermines the desired appearance and visual uniformity of architectural concrete. Therefore, measuring the impact of air pockets on the concrete surface in the form of air pockets is vital in assessing the quality of architectural concrete. Traditionally, such measurements are mainly based on in-situ manual inspections, the results of which are subjective and heavily dependent on the inspectors’ own criteria and experience. Often, inspectors may make different assessments even when inspecting the same concrete surface. In addition, the need for experienced inspectors costs owners or general contractors more in inspection fees. To alleviate these problems, this paper presents a methodology that can measure air pockets quantitatively and automatically. In order to achieve this goal, a high contrast, scaled image of a concrete surface is acquired from a fixed distance range and then a spot filter is used to accurately detect air pockets with the help of an image pyramid. The properties of air pockets (the number, the size, and the occupation area of air pockets) are subsequently calculated. These properties are used to quantify the impact of air pockets on the architectural concrete surface. The methodology is implemented in a C++ based prototype and tested on a database of concrete surface images. Comparisons with manual tests validated its measuring accuracy. As a result, the methodology presented in this paper can increase the reliability of concrete surface quality assessment

Defects Detection and Assessment in Concrete Surfaces

Aside from cracks, the impact of other surface defects, such as air pockets and discoloration, can be detrimental to the quality of concrete in terms of strength, appearance and durability. For this reason, local and national codes provide standards for quantifying the quality impact of these concrete surface defects and owners plan for regular visual inspections to monitor surface conditions. However, manual visual inspection of concrete surfaces is a qualitative (and subjective) process with often unreliable results due to its reliance on inspectors’ own criteria and experience. Also, it is labor intensive and time-consuming. This paper presents a novel, automated concrete surface defects detection and assessment approach that addresses these issues by automatically quantifying the extent of surface deterioration. According to this approach, images of the surface shot from a certain angle/distance can be used to automatically detect the number and size of surface air pockets, and the degree of surface discoloration. The proposed method uses histogram equalization and filtering to extract such defects and identify their properties (e.g. size, shape, location). These properties are used to quantify the degree of impact on the concrete surface quality and provide a numerical tool to help inspectors accurately evaluate concrete surfaces. The method has been implemented in C++ and results that validate its performance are presented.

Computer vision tools for the non-invasive assessment of autism-related behavioral markers

The early detection of developmental disorders is key to child outcome, allowing interventions to be initiated that promote development and improve prognosis. Research on autism spectrum disorder (ASD) suggests behavioral markers can be observed late in the first year of life. Many of these studies involved extensive frame-by-frame video observation and analysis of a child's natural behavior. Although non-intrusive, these methods are extremely time-intensive and require a high level of observer training; thus, they are impractical for clinical and large population research purposes. Diagnostic measures for ASD are available for infants but are only accurate when used by specialists experienced in early diagnosis. This work is a first milestone in a long-term multidisciplinary project that aims at helping clinicians and general practitioners accomplish this early detection/measurement task automatically. We focus on providing computer vision tools to measure and identify ASD behavioral markers based on components of the Autism Observation Scale for Infants (AOSI). In particular, we develop algorithms to measure three critical AOSI activities that assess visual attention. We augment these AOSI activities with an additional test that analyzes asymmetrical patterns in unsupported gait. The first set of algorithms involves assessing head motion by tracking facial features, while the gait analysis relies on joint foreground segmentation and 2D body pose estimation in video. We show results that provide insightful knowledge to augment the clinician's behavioral observations obtained from real in-clinic assessments.

The second round of Critical Assessment of Automated Structure Determination of Proteins by NMR: CASD-NMR-2013

The second round of the community-wide initiative Critical Assessment of automated Structure Determination of Proteins by NMR (CASD-NMR-2013) comprised ten blind target datasets, consisting of unprocessed spectral data, assigned chemical shift lists and unassigned NOESY peak and RDC lists, that were made available in both curated (i.e. manually refined) or un-curated (i.e. automatically generated) form. Ten structure calculation programs, using fully automated protocols only, generated a total of 164 three-dimensional structures (entries) for the ten targets, sometimes using both curated and un-curated lists to generate multiple entries for a single target. The accuracy of the entries could be established by comparing them to the corresponding manually solved structure of each target, which was not available at the time the data were provided. Across the entire data set, 71 % of all entries submitted achieved an accuracy relative to the reference NMR structure better than 1.5 Å. Methods based on NOESY peak lists achieved even better results with up to 100 % of the entries within the 1.5 Å threshold for some programs. However, some methods did not converge for some targets using un-curated NOESY peak lists. Over 90 % of the entries achieved an accuracy better than the more relaxed threshold of 2.5 Å that was used in the previous CASD-NMR-2010 round. Comparisons between entries generated with un-curated versus curated peaks show only marginal improvements for the latter in those cases where both calculations converged.