Assessing the Use of Flexible Assessment

A course which has a large student enrolment consequently puts a heavy load on instructors both in the presentation and the assessment areas. In the School of Economics at the University of Queensland, this is the case for the quantitative analysis subjects. Assessment for many years has been through mid-semester and end of semester exams, as well as Computer Managed Learning (CML) assignments. In 2000 it was decided to incorporate a system of flexible assessment where neither the CML nor the mid-semester exam was compulsory. The outcomes are assessed and the advantages and disadvantages discussed.

Enabling computer access: Introduction to the Test of Mouse Proficiency

As the use of technological devices in everyday environments becomes more prevalent, it is clear that access to these devices has become an important aspect of occupational performance. Children are increasingly required to competently manipulate technology such as the computer to fulfil occupational roles of student and player. Occupational therapists are in a position to facilitate the successful interface between children and standard computer technologies. The literature has supported the use of direct manipulation interfaces in computing that requires mastery of devices such as the mouse. Identification of children likely to experience difficulties with mouse use will inform the development of appropriate methods of intervention promoting mouse skill and further enhance participation in occupational tasks. The aim of this paper is to discuss the development of an assessment of mouse proficiency for children. It describes the construction of the assessment, the content of the test, and its content validity.

Linguistic evaluation method for the environmental assessment of infrastructure projects

A soft linguistic evaluation method is proposed for the environmental assessment of physical infrastructure projects based on fuzzy relations. Infrastructure projects are characterized in terms of linguistic expressions of 'performance' with respect to factors or impacts and the 'importance' of those factors/impacts. A simple example is developed to illustrate the method in the context of three road infrastructure projects assessed against five factors/impacts. In addition, a means to include hard or crisp factors is presented and illustrated with respect to a sixth factor.

Assessing childrens competence in computer interactions: Preliminary reliability and validity of the test of mouse proficiency

The Test of Mouse Proficiency (TOMP) was developed to assist occupational therapists and education professionals assess computer mouse competency skills in children from preschool to upper primary (elementary) school age. The preliminary reliability and validity of TOMP are reported in this paper. Methods used to examine the internal consistency, test-retest reliability, and criterion- and construct-related validity of the test are elaborated. In the continuing process of test refinement, these preliminary studies support to varying degrees the reliability and validity of TOMP. Recommendations for further validation of the assessment are discussed along with indications for potential clinical application.

An evaluation of active lecturing in a computer networks course

The article describes an attempt to improve student learning outcomes in a computer networks course by making lectures more active learning experiences. Quick quizzes, group and individual exercises, the review of student questions, as well as multiple breaks, were incorporated into the weekly three-hour lectures. Student responses to the modified lectures was overwhelmingly positive: over 85% of respondents agreed that the lectures aided understanding, with large majorities of the respondents finding the individual activities useful to their learning. Although student examination performance improved over the previous year, performance on an examination question that was designed to examine deep understanding remained unchanged.

Use of web-based formative assessment to support student learning in metabolism/nutrition unit

In 2002, an integrated basic science course was introduced into the Bachelor of Dental Sciences programme at the University of Queensland, Australia. Learning activities for the Metabolism and Nutrition unit within this integrated course included lectures, problem-based learning tutorials, computer-based self-directed learning exercises and practicals. To support student learning and assist students to develop the skills necessary to become lifelong learners, an extensive bank of formative assessment questions was set up using the commercially available package, WebCT®. Questions included short-answer, multiple-choice and extended matching questions. As significant staff time was involved in setting up the question database, the extent to which students used the formative assessment and their perceptions of its usefulness to their learning were evaluated to determine whether formative assessment should be extended to other units within the course. More than 90% of the class completed formative assessment tasks associated with learning activities scheduled in the first two weeks of the block, but this declined to less than 50% by the fourth and final week of the block. Patterns of usage of the formative assessment were also compared in students who scored in the top 10% for all assessment for the semester with those who scored in the lowest 10%. High-performing students accessed the Web-based formative assessment about twice as often as those who scored in the lowest band. However, marks for the formative assessment tests did not differ significantly between the two groups. In a questionnaire that was administered at the completion of the block, students rated the formative assessment highly, with 80% regarding it as being helpful for their learning. In conclusion, although substantial staff time was required to set up the question database, this appeared to be justified by the positive responses of the students.

Data-driven fuzzy analysis in quantitative mineral resource assessment

The integration of geo-information from multiple sources and of diverse nature in developing mineral favourability indexes (MFIs) is a well-known problem in mineral exploration and mineral resource assessment. Fuzzy set theory provides a convenient framework to combine and analyse qualitative and quantitative data independently of their source or characteristics. A novel, data-driven formulation for calculating MFIs based on fuzzy analysis is developed in this paper. Different geo-variables are considered fuzzy sets and their appropriate membership functions are defined and modelled. A new weighted average-type aggregation operator is then introduced to generate a new fuzzy set representing mineral favourability. The membership grades of the new fuzzy set are considered as the MFI. The weights for the aggregation operation combine the individual membership functions of the geo-variables, and are derived using information from training areas and L, regression. The technique is demonstrated in a case study of skarn tin deposits and is used to integrate geological, geochemical and magnetic data. The study area covers a total of 22.5 km(2) and is divided into 349 cells, which include nine control cells. Nine geo-variables are considered in this study. Depending on the nature of the various geo-variables, four different types of membership functions are used to model the fuzzy membership of the geo-variables involved. (C) 2002 Elsevier Science Ltd. All rights reserved.

Validation and patient acceptance of a computer touch screen version of the WOMAC 3.1 osteoarthritis index

Objectives: To validate the WOMAC 3.1 in a touch screen computer format, which applies each question as a cartoon in writing and in speech (QUALITOUCH method), and to assess patient acceptance of the computer touch screen version. Methods: The paper and computer formats of WOMAC 3.1 were applied in random order to 53 subjects with hip or knee osteoarthritis. The mean age of the subjects was 64 years ( range 45 to 83), 60% were male, 53% were 65 years or older, and 53% used computers at home or at work. Agreement between formats was assessed by intraclass correlation coefficients (ICCs). Preferences were assessed with a supplementary questionnaire. Results: ICCs between formats were 0.92 (95% confidence interval, 0.87 to 0.96) for pain; 0.94 (0.90 to 0.97) for stiffness, and 0.96 ( 0.94 to 0.98) for function. ICCs were similar in men and women, in subjects with or without previous computer experience, and in subjects below or above age 65. The computer format was found easier to use by 26% of the subjects, the paper format by 8%, and 66% were undecided. Overall, 53% of subjects preferred the computer format, while 9% preferred the paper format, and 38% were undecided. Conclusion: The computer format of the WOMAC 3.1 is a reliable assessment tool. Agreement between computer and paper formats was independent of computer experience, age, or sex. Thus the computer format may help improve patient follow up by meeting patients' preferences and providing immediate results.

Screening for physical activity in family practice - Evaluation of two brief assessment tools

Background: Physical activity (PA) is relevant to the prevention and management of many health conditions in family practice. There is a need for an efficient, reliable, and valid assessment tool to identify patients in need of PA interventions. Methods: Twenty-eight family physicians in three Australian cities assessed the PA of their adult patients during 2004 using either a two- (2Q) or three-question (3Q) assessment. This was administered again approximately 3 days later to evaluate test-retest reliability. Concurrent validity was evaluated by measuring agreement with the Active Australia Questionnaire, and criterion validity by comparison with 7-day Computer Science Applications, Inc. (CSA) accelerometer counts. Results: A total of 509 patients participated, with 428 (84%) completing a repeat assessment, and 415 (82%) accelerometer monitoring. The brief assessments had moderate test-retest reliability (2Q k = 58.0%, 95% confidence interval [CI] = 47.2-68.8%; 3Q k = 55.6%, 95% CI = 43.8-67.4%); fair to moderate concurrent validity (2Q k = 46.7%, 95% CI = 35.657.9%; 3Q k = 38.7%, 95% CI = 26.4-51.1%); and poor to fair criterion validity (2Q k = 18.2%, 95% CI = 3.9-32.6%; 3Q k = 24.3%, 95% CI = 11.6-36.9%) for identifying patients as sufficiently active. A four-level scale of PA derived from the PA assessments was significantly correlated with accelerometer minutes (2Q rho = 0.39, 95% CI = 0.28-0.49; 3Q rho = 0.31, 95% CI = 0.18-0.43). Physicians reported that the assessments took I to 2 minutes to complete. Conclusions: Both PA assessments were feasible to use in family practice, and were suitable for identifying the least active patients. The 2Q assessment was preferred by clinicians and may be most appropriate for dissemination.

Reliability and validity of a brief physical activity assessment for use by family doctors

Objective: To evaluate the reliability and validity of a brief physical activity assessment tool suitable for doctors to use to identify inactive patients in the primary care setting. Methods: Volunteer family doctors (n = 8) screened consenting patients (n = 75) for physical activity participation using a brief physical activity assessment tool. Inter-rater reliability was assessed within one week (n = 71). Validity was assessed against an objective physical activity monitor (computer science and applications accelerometer; n = 42). Results: The brief physical activity assessment tool produced repeatable estimates of sufficient total physical activity, correctly classifying over 76% of cases (kappa 0.53, 95% confidence interval (CI) 0.33 to 0.72). The validity coefficient was reasonable (kappa 0.40, 95% CI 0.12 to 0.69), with good percentage agreement (71%). Conclusions: The brief physical activity assessment tool is a reliable instrument, with validity similar to that of more detailed self report measures of physical activity. It is a tool that can be used efficiently in routine primary healthcare services to identify insufficiently active patients who may need physical activity advice.

Fitness assessment of document model

Document classification is a supervised machine learning process, where predefined category labels are assigned to documents based on the hypothesis derived from training set of labelled documents. Documents cannot be directly interpreted by a computer system unless they have been modelled as a collection of computable features. Rogati and Yang [M. Rogati and Y. Yang, Resource selection for domain-specific cross-lingual IR, in SIGIR 2004: Proceedings of the 27th annual international conference on Research and Development in Information Retrieval, ACM Press, Sheffied: United Kingdom, pp. 154-161.] pointed out that the effectiveness of document classification system may vary in different domains. This implies that the quality of document model contributes to the effectiveness of document classification. Conventionally, model evaluation is accomplished by comparing the effectiveness scores of classifiers on model candidates. However, this kind of evaluation methods may encounter either under-fitting or over-fitting problems, because the effectiveness scores are restricted by the learning capacities of classifiers. We propose a model fitness evaluation method to determine whether a model is sufficient to distinguish positive and negative instances while still competent to provide satisfactory effectiveness with a small feature subset. Our experiments demonstrated how the fitness of models are assessed. The results of our work contribute to the researches of feature selection, dimensionality reduction and document classification.

Formal analysis of human-computer interaction using model checking

Experiments with simulators allow psychologists to better understand the causes of human errors and build models of cognitive processes to be used in human reliability assessment (HRA). This paper investigates an approach to task failure analysis based on patterns of behaviour, by contrast to more traditional event-based approaches. It considers, as a case study, a formal model of an air traffic control (ATC) system which incorporates controller behaviour. The cognitive model is formalised in the CSP process algebra. Patterns of behaviour are expressed as temporal logic properties. Then a model-checking technique is used to verify whether the decomposition of the operator's behaviour into patterns is sound and complete with respect to the cognitive model. The decomposition is shown to be incomplete and a new behavioural pattern is identified, which appears to have been overlooked in the analysis of the data provided by the experiments with the simulator. This illustrates how formal analysis of operator models can yield fresh insights into how failures may arise in interactive systems.

A qualitative assessment of communicating spatial concepts in virtual and physical environments via a text-based medium

The pedagogical exercise described here was used to investigate how spatial communication about the manipulation of objects in a virtual and physical space is communicated between remote partners. It continues work done by others. Where it differs from previous research in this area is in its use of a qualitative methodology to study how these types of interactions are structured, communicated and interpreted via text-based media. What emerged from the qualitative analysis are new insights over the previous quantitative investigations. This paper reports on completed research.