Can computer-based testing achieve quality and efficiency in assessment?

This paper reports findings of a two year study concerning the development and implementation of a general-purpose computer-based assessment (CBA) system at a UK University. Data gathering took place over a period of nineteen months, involving a number of formative and summative assessments. Approximately 1,000 students, drawn from undergraduate courses, were involved in the exercise. The techniques used in gathering data included questionnaires, observation, interviews and an analysis of student scores in both conventional examinations and computer-based assessments. Comparisons with conventional assessment methods suggest that the use of CBA techniques may improve the overall performance of students. However it is clear that the technique must not be seen as a "quick fix" for problems such as rising student numbers. If one accepts that current systems test only a relatively narrow range of skills, then the hasty implementation of CBA systems will result in a distorted and inaccurate view of student performance. In turn, this may serve to reduce the overall quality of courses and - ultimately - detract from the student learning experience. On the other hand, if one adopts a considered and methodical approach to computer-based assessment, positive benefits might include increased efficiency and quality, leading to improved student learning.

Computer based monitoring in U.K. service organizations : a comparative study

The majority of the literature about CBM is American in origin, and (inter alia) notes that there were differing uses of similar technology, indicating that context has an important role to play in the use of CBM. The literature maps the psychological effects of CBM in considerable detail, but only two published studies examine the context of CBM. These grounded results provide scant support for any systematic, quantitative, large scale analysis of computer based monitoring in the UK context. This thesis thus aims to systemically examine the context of CBM using discourse analysis. Forty four interviewees were theoretically sampled using a structured sample technique in four organizations. All were national or multinational enterprises. The interviews were semi structured in nature and divided into three sections. The first addressed the respondents' thoughts and perceptions about CBM, the second elicited talk about the departmental context (focusing the management - worker relationship), and the final section addressed the organizational context. The cases demonstrated variation in the use of CBM, measured according to the criteria of Westin (1987, 1988) and according to the interpretive repertoires used by the respondents in each case. Seven analytical categories of talk emerged from the data: three at the organizational level and four at the departmental level of analysis. Discourse analysis revealed two discrete interpretive repertories - the procedural and the substantive repertoires - in respondents' talk whose main variation occurred at the departmental level of analysis. Furthermore, patterns were found in the use of these repertories within cases and between categories. Between the cases, variation in the use of the repertories matched the between case variation according to the criteria of Westin. It would thus appear that the source of variation in the use of CBM lies in its context, more specifically in the relative emphasis of humanistic, interpersonal and idiosyncratic values within the management worker relationship.

Computer based approaches to managerial decision making:optimism, heuristics and simulation

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Learner centred control in computer based training

DUE TO COPYRIGHT RESTRICTIONS ONLY AVAILABLE FOR CONSULTATION AT ASTON UNIVERSITY LIBRARY AND INFORMATION SERVICES WITH PRIOR ARRANGEMENT

Assessing the motivation of MDs to use computer-based support at the point-of-care in the emergency department

A significant body of research investigates the acceptance of computer-based support (including devices and applications ranging from e-mail to specialized clinical systems, like PACS) among clinicians. Much of this research has focused on measuring the usability of systems using characteristics related to the clarity of interactions and ease of use. We propose that an important attribute of any clinical computer-based support tool is the intrinsic motivation of the end-user (i.e. a clinician) to use the system in practice. In this paper we present the results of a study that investigated factors motivating medical doctors (MDs) to use computer-based support. Our results demonstrate that MDs value computer-based support, find it useful and easy to use, however, uptake is hindered by perceived incompetence, and pressure and tension associated with using technology.

The development of a multi-function computer-assisted instruction system using a hierarchical database structure as the lesson compendium.

Computer-Based Learning systems of one sort or another have been in existence for almost 20 years, but they have yet to achieve real credibility within Commerce, Industry or Education. A variety of reasons could be postulated for this, typically: - cost - complexity - inefficiency - inflexibility - tedium Obviously different systems deserve different levels and types of criticism, but it still remains true that Computer-Based Learning (CBL) is falling significantly short of its potential. Experience of a small, but highly successful CBL system within a large, geographically distributed industry (the National Coal Board) prompted an investigation into currently available packages, the original intention being to purchase the most suitable software and run it on existing computer hardware, alongside existing software systems. It became apparent that none of the available CBL packages were suitable, and a decision was taken to develop an in-house Computer-Assisted Instruction system according to the following criteria: - cheap to run; - easy to author course material; - easy to use; - requires no computing knowledge to use (as either an author or student) ; - efficient in the use of computer resources; - has a comprehensive range of facilities at all levels. This thesis describes the initial investigation, resultant observations and the design, development and implementation of the SCHOOL system. One of the principal characteristics c£ SCHOOL is that it uses a hierarchical database structure for the storage of course material - thereby providing inherently a great deal of the power, flexibility and efficiency originally required. Trials using the SCHOOL system on IBM 303X series equipment are also detailed, along with proposed and current development work on what is essentially an operational CBL system within a large-scale Industrial environment.

An Investigation into the adoption and implementation of computer aided production management systems:using the Rogers model of adoption and implementation and a social constuctionist account of technology

The research investigates the processes of adoption and implementation, by organisations, of computer aided production management systems (CAPM). It is organised around two different theoretical perspectives. The first part is informed by the Rogers model of the diffusion, adoption and implementation of innovations, and the second part by a social constructionist approach to technology. Rogers' work is critically evaluated and a model of adoption and implementation is distilled from it and applied to a set of empirical case studies. In the light of the case study data, strengths and weaknesses of the model are identified. It is argued that the model is too rational and linear to provide an adequate explanation of adoption processes. It is useful for understanding processes of implementation but requires further development. The model is not able to adequately encompass complex computer based technologies. However, the idea of 'reinvention' is identified as Roger's key concept but it needs to be conceptually extended. Both Roger's model and definition of CAPM found in the literature from production engineering tend to treat CAPM in objectivist terms. The problems with this view are addressed through a review of the literature on the sociology of technology, and it is argued that a social constructionist approach offers a more useful framework for understanding CAPM, its nature, adoption, implementation, and use. CAPM it is argued, must be understood on terms of the ways in which it is constituted in discourse, as part of a 'struggle for meaning' on the part of academics, professional engineers, suppliers, and users.

Individual differences in human-computer interaction

This thesis initially presents an 'assay' of the literature pertaining to individual differences in human-computer interaction. A series of experiments is then reported, designed to investigate the association between a variety of individual characteristics and various computer task and interface factors. Predictor variables included age, computer expertise, and psychometric tests of spatial visualisation, spatial memory, logical reasoning, associative memory, and verbal ability. These were studied in relation to a variety of computer-based tacks, including: (1) word processing and its component elements; (ii) the location of target words within passages of text; (iii) the navigation of networks and menus; (iv) command generation using menus and command line interfaces; (v) the search and selection of icons and text labels; (vi) information retrieval. A measure of self-report workload was also included in several of these experiments. The main experimental findings included: (i) an interaction between spatial ability and the manipulation of semantic but not spatial interface content; (ii) verbal ability being only predictive of certain task components of word processing; (iii) age differences in word processing and information retrieval speed but not accuracy; (iv) evidence of compensatory strategies being employed by older subjects; (v) evidence of performance strategy differences which disadvantaged high spatial subjects in conditions of low spatial information content; (vi) interactive effects of associative memory, expertise and command strategy; (vii) an association between logical reasoning and word processing but not information retrieval; (viii) an interaction between expertise and cognitive demand; and (ix) a stronger association between cognitive ability and novice performance than expert performance.