Can computer-based testing achieve quality and efficiency in assessment?

This paper reports findings of a two year study concerning the development and implementation of a general-purpose computer-based assessment (CBA) system at a UK University. Data gathering took place over a period of nineteen months, involving a number of formative and summative assessments. Approximately 1,000 students, drawn from undergraduate courses, were involved in the exercise. The techniques used in gathering data included questionnaires, observation, interviews and an analysis of student scores in both conventional examinations and computer-based assessments. Comparisons with conventional assessment methods suggest that the use of CBA techniques may improve the overall performance of students. However it is clear that the technique must not be seen as a "quick fix" for problems such as rising student numbers. If one accepts that current systems test only a relatively narrow range of skills, then the hasty implementation of CBA systems will result in a distorted and inaccurate view of student performance. In turn, this may serve to reduce the overall quality of courses and - ultimately - detract from the student learning experience. On the other hand, if one adopts a considered and methodical approach to computer-based assessment, positive benefits might include increased efficiency and quality, leading to improved student learning.

Approaches to Developing Educational Resources Using AI Embedded in Computer Game Technology. A Case Study: Primary School Literacy

In this extended abstract, we discuss recent research at Worcester into the inclusion of AI into ‘Serious Games’. Serious Games research intends to harness the power of computer game technology to produce educational and training materials. We prefer the name ‘Immersive Environments’ (IEs) since this emphasises the human psychological dimension. Creation of compelling and convincing learning software requires a rich engagement of the learner, and a convincing learning experience. We believe that various aspects of the AI tradition can inform the production of such learning.

Creating Educational Immersive Environments Using a Computer Game Engine: Case Studies and Design Theory

This paper aims to crystallize recent research performed at the University of Worcester to investigate the feasibility of using the commercial game engine ‘Unreal Tournament 2004’ (UT2004) to produce ‘Educational Immersive Environments’ (EIEs) suitable for education and training. Our research has been supported by the UK Higher Education Academy. We discuss both practical and theoretical aspects of EIEs. The practical aspects include the production of EIEs to support high school physics education, the education of architects, and the learning of literacy by primary school children. This research is based on the development of our novel instructional medium, ‘UnrealPowerPoint’. Our fundamental guiding principles are that, first, pedagogy must inform technology, and second, that both teachers and pupils should be empowered to produce educational materials. Our work is informed by current educational theories such as constructivism, experiential learning and socio-cultural approaches as well as elements of instructional design and game principles.

Deconstruction-reconstruction of Computer Game and Immersive Environment Semiotics

Computer game technology produces compelling ‘immersive environments’ where our digitally-native youth play and explore. Players absorb visual, auditory and other signs and process these in real time, making rapid choices on how to move through the game-space to experience ‘meaningful play’. How can immersive environments be designed to elicit perception and understanding of signs? In this paper we explore game design and gameplay from a semiotic perspective, focusing on the creation of meaning for players as they play the game. We propose a theory of game design based on semiotics.

The Role of Art in Computer Game Design

Computer games are significant since they embody our youngsters’ engagement with contemporary culture, including both play and education. These games rely heavily on visuals, systems of sign and expression based on concepts and principles of Art and Architecture. We are researching a new genre of computer games, ‘Educational Immersive Environments’ (EIEs) to provide educational materials suitable for the school classroom. Close collaboration with subject teachers is necessary, but we feel a specific need to engage with the practicing artist, the art theoretician and historian. Our EIEs are loaded with multimedia (but especially visual) signs which act to direct the learner and provide the ‘game-play’ experience forming semiotic systems. We suggest the hypothesis that computer games are a space of deconstruction and reconstruction (DeRe): When players enter the game their physical world and their culture is torn apart; they move in a semiotic system which serves to reconstruct an alternate reality where disbelief is suspended. The semiotic system draws heavily on visuals which direct the players’ interactions and produce motivating gameplay. These can establish a reconstructed culture and emerging game narrative. We have recently tested our hypothesis and have used this in developing design principles for computer game designers. Yet there are outstanding issues concerning the nature of the visuals used in computer games, and so questions for contemporary artists. Currently, the computer game industry employs artists in a ‘classical’ role in production of concept sketches, storyboards and 3D content. But this is based on a specification from the client which restricts the artist in intellectual freedom. Our DeRe hypothesis places the artist at the generative centre, to inform the game designer how art may inform our DeRe semiotic spaces. This must of course begin with the artists’ understanding of DeRe in this time when our ‘identities are becoming increasingly fractured, networked, virtualized and distributed’ We hope to persuade artists to engage with the medium of computer game technology to explore these issues. In particular, we pose several questions to the artist: (i) How can particular ‘periods’ in art history be used to inform the design of computer games? (ii) How can specific artistic elements or devices be used to design ‘signs’ to guide the player through the game? (iii) How can visual material be integrated with other semiotic strata such as text and audio?

The Development of Educational Materials for the Study of Architecture using Computer Game Technology

Computer game technology provides us with the tools to create web-based educational materials for autonomous and collaborative learning. At Worcester, we have researched the use of this technology in various educational contexts. This paper reports one such study; the use of the commercial game engine “Unreal Tournament 2004” (UT2004) to produce materials suitable for education of Architects. We map the concepts and principles of Architectural Design onto the affordances (development tools) provided by UT2004, leading to a systematic procedure for the realization of buildings and urban environments using this game engine. A theory for the production of web-based learning materials which supports both autonomous and collaborative learning is developed. A heuristic evaluation of our materials, used with second-year students is presented. Associated web-pages provide on-line materials for delegates.

Unreal PowerPoint™: Immersing PowerPoint Presentations in a Virtual Computer Game Engine World

At a recent conference on games in education, we made a radical decision to transform our standard presentation of PowerPoint slides and computer game demonstrations into a unified whole, inserting the PowerPoint presentation to the computer game. This opened up various questions relating to learning and teaching theories, which were debated by the conference delegates. In this paper, we reflect on these discussions, we present our initial experiment, and relate this to various theories of learning and teaching. In particular, we consider the applicability of “concept maps” to inform the construction of educational materials, especially their topological, geometrical and pedagogical significance. We supplement this “spatial” dimension with a theory of the dynamic, temporal dimension, grounded in a context of learning processes, such as Kolb’s learning cycle. Finally, we address the multi-player aspects of computer games, and relate this to the theories of social and collaborative learning. This paper attempts to explore various theoretical bases, and so support the development of a new learning and teaching virtual reality approach.

Effect of Playing Interactive Computer Game on Distress of Insulin Injection Among Type 1 Diabetic Children

Background: Diabetic children and their families experience high level stress because of daily insulin injection. Objectives: This study was conducted to investigate the impact of an interactive computer game on behavioral distress due to insulin injection among diabetic children. Patients and Methods: In this clinical trial, thirty children (3-12 years) with type 1 diabetes who needed daily insulin injection were recruited and allocated randomly into two groups. Children in intervention groups received an interactive computer game and asked to play at home for a week. No special intervention was done for control group. The behavioral distress of groups was assessed before, during and after the intervention by Observational Scale of Behavioral Distress–Revised (OSBD-R). Results: Repeated measure ANOVA test showed no significantly difference of OSBD-R over time for control group (P = 0.08), but this changes is signification in the study group (P = 0.001). Comparison mean score of distress were significantly different between two groups (P = 0.03). Conclusions: According to the findings, playing interactive computer game can decrease behavioral distress induced by insulin injection in type 1 diabetic children. It seems this game can be beneficial to be used alongside other interventions.

Measuring Visual Consistency in 3D Rendering Systems

One of the major challenges facing a present day game development company is the removal of bugs from such complex virtual environments. This work presents an approach for measuring the correctness of synthetic scenes generated by a rendering system of a 3D application, such as a computer game. Our approach builds a database of labelled point clouds representing the spatiotemporal colour distribution for the objects present in a sequence of bug-free frames. This is done by converting the position that the pixels take over time into the 3D equivalent points with associated colours. Once the space of labelled points is built, each new image produced from the same game by any rendering system can be analysed by measuring its visual inconsistency in terms of distance from the database. Objects within the scene can be relocated (manually or by the application engine); yet the algorithm is able to perform the image analysis in terms of the 3D structure and colour distribution of samples on the surface of the object. We applied our framework to the publicly available game RacingGame developed for Microsoft(R) Xna(R). Preliminary results show how this approach can be used to detect a variety of visual artifacts generated by the rendering system in a professional quality game engine.

Computational approaches to the visual validation of 3D virtual environments

Virtual environments can provide, through digital games and online social interfaces, extremely exciting forms of interactive entertainment. Because of their capability in displaying and manipulating information in natural and intuitive ways, such environments have found extensive applications in decision support, education and training in the health and science domains amongst others. Currently, the burden of validating both the interactive functionality and visual consistency of a virtual environment content is entirely carried out by developers and play-testers. While considerable research has been conducted in assisting the design of virtual world content and mechanics, to date, only limited contributions have been made regarding the automatic testing of the underpinning graphics software and hardware. The aim of this thesis is to determine whether the correctness of the images generated by a virtual environment can be quantitatively defined, and automatically measured, in order to facilitate the validation of the content. In an attempt to provide an environment-independent definition of visual consistency, a number of classification approaches were developed. First, a novel model-based object description was proposed in order to enable reasoning about the color and geometry change of virtual entities during a play-session. From such an analysis, two view-based connectionist approaches were developed to map from geometry and color spaces to a single, environment-independent, geometric transformation space; we used such a mapping to predict the correct visualization of the scene. Finally, an appearance-based aliasing detector was developed to show how incorrectness too, can be quantified for debugging purposes. Since computer games heavily rely on the use of highly complex and interactive virtual worlds, they provide an excellent test bed against which to develop, calibrate and validate our techniques. Experiments were conducted on a game engine and other virtual worlds prototypes to determine the applicability and effectiveness of our algorithms. The results show that quantifying visual correctness in virtual scenes is a feasible enterprise, and that effective automatic bug detection can be performed through the techniques we have developed. We expect these techniques to find application in large 3D games and virtual world studios that require a scalable solution to testing their virtual world software and digital content.

Neural network-based detection of virtual environment anomalies

The increasingly widespread use of large-scale 3D virtual environments has translated into an increasing effort required from designers, developers and testers. While considerable research has been conducted into assisting the design of virtual world content and mechanics, to date, only limited contributions have been made regarding the automatic testing of the underpinning graphics software and hardware. In the work presented in this paper, two novel neural network-based approaches are presented to predict the correct visualization of 3D content. Multilayer perceptrons and self-organizing maps are trained to learn the normal geometric and color appearance of objects from validated frames and then used to detect novel or anomalous renderings in new images. Our approach is general, for the appearance of the object is learned rather than explicitly represented. Experiments were conducted on a game engine to determine the applicability and effectiveness of our algorithms. The results show that the neural network technology can be effectively used to address the problem of automatic and reliable visual testing of 3D virtual environments.

A method for measuring the creative potential of computer games

This paper describes a method for measuring the creative potential of computer games. The research approach applies a behavioral and verbal protocol to analyze the factors that influence the creative processes used by people as they play computer games from the puzzle genre. Creative potential is measured by examining task motivation and domain-relevant and creativity-relevant skills. This paper focuses on the reliability of the factors used for measurement, determining those factors that are more strongly related to creativity. The findings show that creative potential may be determined by examining the relationship between skills required and the effect of intrinsic motivation within game play activities.

Testing the simulation capability of APSIM-ORYZA under different levels of nitrogen fertiliser and transplanting time regimes in Korea

APSIM-ORYZA is a new functionality developed in the APSIM framework to simulate rice production while addressing management issues such as fertilisation and transplanting, which are particularly important in Korean agriculture. To validate the model for Korean rice varieties and field conditions, the measured yields and flowering times from three field experiments conducted by the Gyeonggi Agricultural Research and Extension Services (GARES) in Korea were compared against the simulated outputs for different management practices and rice varieties. Simulated yields of early-, mid- and mid-to-late-maturing varieties of rice grown in a continuous rice cropping system from 1997 to 2004 showed close agreement with the measured data. Similar results were also found for yields simulated under seven levels of nitrogen application. When different transplanting times were modelled, simulated flowering times ranged from within 3 days of the measured values for the early-maturing varieties, to up to 9 days after the measured dates for the mid- and especially mid-to-late-maturing varieties. This was associated with highly variable simulated yields which correlated poorly with the measured data. This suggests the need to accurately calibrate the photoperiod sensitivity parameters of the model for the photoperiod-sensitive rice varieties in Korea.