Integrating kinect and haptics for interactive STEM education in local and distributed environments

Skill shortage is a realistic social problem that Australia is currently facing, especially in the fields of Science, Technology, Engineering and Mathematics (STEM). Various approaches have been proposed to soften this issue. By now the most successful approach is to attract pre-university youth and university freshmen into those fields before they make a decision on future subjects by introducing them with interactive, modifiable and inspiring virtual environments, which incorporates most essential knowledge of STEM. We propose to design a comprehensive virtual reality platform with immersive interactions, pluggable components and flexible configurations. It also involves haptics, motion capture and gesture recognition, and could be deployed in both local and distributed environments. The platform utilizes off the shelf low cost haptics and motion capture products, however the fidelity can be maintained at a good level. The proposed platform has been implemented with different configurations and has been tested on a group of users. Preliminary test results show that the interactivity, flexibility and fidelity of the platform are highly appreciated by users. User surveys also indicate that the proposed platform could help pre-university students and university freshmen build an overview of various aspects of STEM education. Besides, users are also positive on the fact that the platform enabled them to identify the challenges for higher education in STEM by providing them opportunities to interactively modify system configurations and instantly experience the corresponding results both visually and haptically.

Multitasking: multiple, domain-specific cognitive functions in a virtual environment

Multitasking among three or more different tasks is a ubiquitous requirement of everyday cognition, yet rarely is it addressed in research on healthy adults who have had no specific training in multitasking skills. Participants completed a set of diverse subtasks within a simulated shopping mall and office environment, the Edinburgh Virtual Errands Test (EVET). The aim was to investigate how different cognitive functions, such as planning, retrospective and prospective memory, and visuospatial and verbal working memory, contribute to everyday multitasking. Subtasks were chosen to be diverse, and predictions were derived from a statistical model of everyday multitasking impairments associated with frontal-lobe lesions (Burgess, Veitch, de Lacy Costello, & Shallice, 2000b). Multiple regression indicated significant independent contributions from measures of retrospective memory, visuospatial working memory, and online planning, but not from independent measures of prospective memory or verbal working memory. Structural equation modelling showed that the best fit to the data arose from three underlying constructs, with Memory and Planning having a weak link, but with both having a strong directional pathway to an Intent construct that reflected implementation of intentions. Participants who followed their preprepared plan achieved higher scores than those who altered their plan during multitask performance. This was true regardless of whether the plan was efficient or poor. These results substantially develop and extend the Burgess et al. (2000b) model to healthy adults and yield new insight into the poorly understood area of everyday multitasking. The findings also point to the utility of using virtual environments for investigating this form of complex human cognition.

Second life first hand: a teacher perspective

The professional landscape in public relations is changing as new communication and social networking technologies are integrated into day-to-day professional practice. Whilst adoption of such technologies by public relations practitioners is certainly on the increase, their use can still be regarded as limited and application experimental to some degree. However, few could argue that these technologies will be increasingly important to public relations practice in coming years.
In this context, public relations educators must strive to deliver a contemporary curriculum reflective of industry expectations and best practice principles but which also provides students with exposure to new communication contexts and technologies.

The advent of persistent virtual worlds generated by Massively Multiplayer Online Role Playing Games (MMORPGs) and Collaborative Virtual Environments (CVEs) offer new realms for public relations practitioners and educators alike. Virtual worlds potentially provide public relations educators with novel but relevant training grounds for their students. These 3D worlds offer dynamic and authentic learning environments which have the capability to foster deep learning and engender a sense of community within a student cohort in a way that many computer-mediated classrooms sadly lack.

This paper will present the experiences of two tertiary educators’ journey towards a conceptual understanding of the persistent virtual world, Second Life, from a teacher perspective. The paper argues that the successful adoption of new online technologies like Second Life need not be inhibited by preferences for technology or prior ICT skills as long as teaching staff are given the necessary support and training by their institutions coupled with opportunity for familiarisation and experimentation.

Success factors for blended learning

There is now a new landscape in educational technology where physical and virtual environments are blended to support learning in university courses. Blended learning has potential to improve learning but there are also challenges, especially in responding to the complexity of two environments and embedding it as a legitimate learning environment. Owing to the newness of the blended learning concept in higher education, little is known about what makes a successful blended learning experience and this paper provides an overview of success factors that are starting to emerge from the literature, research and our practice. It also offers some suggestions for future research.

Computer-simulated environment for training : challenge of efficacy evaluation

Computer-assisted instruction has been around for decades. There has been much speculation about the benefits of computer-mediated learning. Numerous applications have been developed in different domains incorporated with emerging technologies. In recently years, advanced technologies, such as Augmented Reality (AR) and Virtual Reality (VR), have received much attention in their potential of creating interactive learning experience for the users. However, related literature and empirical studies indicated that learning effects in computer-simulated environments or Virtual Environments (VEs) are not systematically tested. Furthermore, the performance and learning in computer-simulated learning environment need to be evaluated through more rigorous methods. This paper suggests that 1) the efficacy of VEs is subject to a close examination, not only in terms of how VE-based training systems are easy of use, but also in terms of how effective learning is; 2) evaluation of learning in computer simulated learning environments is required to be reconsidered in terms of theoretical basis and evaluation methodologies that are relevant to the measurement of training effectiveness in computer-simulated virtual learning environment. This paper explains on how learning can be assessed in VEs through the lens of training evaluation.

Enriching student experience and inter-professional learning of inclusive design with Second Life

This position paper reflects rapid advances in immersive 2D and 3D eLearning technologies and the expanding pool of ideas and applications in higher education across two professions. Inspiration has been drawn from examples in design learning, and various multidisciplinary collaborative projects through developmental research in Multi-User Virtual Environments (MUVEs). Linden Lab's Second Life (SL) is the most mature and popular of the ‘persistent’ virtual worlds. The study described in this paper aims to increase the authenticity of student learning through a range of SL simulated ‘life experiences’ relating to accessibility and mobility in the built environment. Significantly, the successes of such initiatives lie in several elements: teaching champions with vision and courage; detailed scripting of precise role-play encounters for first-time users to provide supportive ‘blended learning’ contexts; careful and vigilant strategic management of facilities and resources, and a robust design program. This paper focuses on the crucial alignment of these elements to the specific challenges of designing and navigating conception and development processes, to enable the execution and delivery of a tightly defined script for meaningful and memorable learning outcomes. This innovative pedagogical approach lacks time-tested outcomes, but is recognised equally as opportunity and challenge; risk and reward.

An empirical examination of feedback : user control and performance in a hapto-audio-visual training environment

Utilising advanced technologies, such as virtual environments (VEs), is of importance to training and education. The need to develop and effectively apply interactive, immersive 3D VEs continues to grow. As with any emerging technology, user acceptance of new software and hardware devices is often difficult to measure and guidelines to introduce and ensure adequate and correct usage of such technologies are lacking. It is therefore imperative to obtain a solid understanding of the important elements that play a role in effective learning through VEs. In particular, 3D VEs may present unusual and varied interaction and adoption considerations. The major contribution of this study is to investigate a complex set of interrelated factors in the relatively new sphere of VEs for training and education. Although many of the factors appears to be important from past research, researcher have not explicitly studied a comprehensive set of inter-dependant, empirically validated factors in order to understand how VEs aid complex procedural knowledge and motor skill learning. By integrating theory from research on training, human computer interaction (HCI), ergonomics and cognitive psychology, this research proposes and validates a model that contributes to application-specific VE efficacy formation. The findings of this study show visual feedback has a significant effect on performance. For tactile/force feedback and auditory feedback, no significant effect were found. For satisfaction, user control is salient for performance. Other factors such as interactivity and system comfort, as well as level of task difficulty, also showed effects on performance.

Simulating crowd phenomena in African markets

Crowd simulation is an important feature in the computer graphics field. Typical implementations simulate battle scenes, emergency situations, safety issues or add content to virtual environments. The problem stated in this paper falls in the last category. We present a crowd simulation behavioural model which allows us to simulate identified phenomena in popular local African markets such as narrow street flows and crowd formation around street performances. We propose a three-tier architecture model enable to produce intentions, perform path planning and control movement. We demonstrate that this approach produces the desired behaviour associated with crowds in an African market, which includes navigation, flow formation and circle creation.

Mechanisms for multimodality : taking fiction to another dimension

We present methods for automatically constructing representations of fiction books in a range of modalities: audibly, graphically and as 3D virtual environments. The correspondence between the sequential ordering of events against the order of events presented in the text is used to correctly resolve the dynamic interactions for each representation. Synthesised audio created from the fiction text is used to calibrate the base time-line against which the other forms of media are correctly aligned. The audio stream is based on speech synthesis using the text of the book, and is enhanced using distinct voices for the different characters in a book. Sound effects are included automatically. The graphical representation represents the text (as subtitles), identifies active characters and provides visual feedback of the content of the story. Dynamic virtual environments conform to the constraints implied by the story, and are used as a source of further visual content. These representations are all aligned to a common time-line, and combined using sequencing facilities to provide a multimodal version of the original text.

User-centered design and evaluation of an interactive visual-haptic-auditory interface : a user study on assembly

Utilizing user-centred system design and evaluation method has become an increasingly important tool to foster better usability in the field of virtual environments (VEs). In recent years, although it is still the norm that designers and developers are concerning the technological advancement and striving for designing impressive multimodal multisensory interfaces, more and more awareness are aroused among the development team that in order to produce usable and useful interfaces, it is essential to have users in mind during design and validate a new design from users' perspective. In this paper, we describe a user study carried out to validate a newly developed haptically enabled virtual training system. By taking consideration of the complexity of individual differences on human performance, adoption and acceptance of haptic and audio-visual I/O devices, we address how well users learn, perform, adapt to and perceive object assembly training. We also explore user experience and interaction with the system, and discuss how multisensory feedback affects user performance, perception and acceptance. At last, we discuss how to better design VEs that enhance users perception, their interaction and motor activity.

A haptic platform to enable fingertip grasping and manipulation

Haptic technologies allow human users to haptically interact with virtual environments. Haptics has been employed in many application domains including operator training, virtual exploration and teleoperation. Currently, most commercially available haptic devices focus on a single point of haptic interaction. While single-point haptics have been successfully employed in many applications, they remain limited to particular types of haptic interaction. Multi-point haptic devices are a logical progression and facilitate a far wider range of interactions including object grasping, multi-finger object manipulation and size discrimination. The ability to effectively achieve such interactions offers significant benefits for many applications including virtual training, telesurgery and telemanipulation. In such applications, the ability to use multi-point haptic interactions can provide far more effective user interaction as well improved perception of the virtual environment.

Inclusive practices in academia and beyond

This chapter explores a set of principles that underpin ensuring that the learning needs of all students are addressed in next generation learning spaces. With increasingly diverse higher education environments and populations, higher education needs to move from seeing student diversity as problematic and deficit-based, to welcoming, celebrating and recognising diversity for the contributions it makes to enhancing the experience and learning outcomes for all students. The principles of Universal Design for Learning (CAST, 2011) provide a framework for highquality university teaching and learning, as well as guidance on the multiple methods and means by which all students can be engaged and learn in ways that best suit their individual styles and needs. An inclusive approach is important pedagogically and applies to both the physical and virtual environments and spaces inhabited by students. When the design of physical environments does not incorporate universal design principles, the result is that some students can be locked out of participating in campus or university life or, for some, the energy required to participate can be substantial. With the digital education frontier expanding at an exponential rate, there is also a need to ensure that online and virtual environments are accessible for all. This chapter draws on the relevant research and the combined experience of the authors to explore an approach to inclusive practices in higher education next generation learning spaces and beyond. © 2014 by Emerald Group Publishing Limited.

Blended learning and art education: student participation in a collaborative, authentic assessment project

This paper reports on higher education student engagement with blended learning experiences incorporating located (on campus), cloud based (online e-learning ) and graphically built, socially networked 3D multi user virtual environments (MUVES). Immersion in this environment enabled collaboration between two groups of students enrolled in separate undergraduate art education and public relations units, to identify, develop and participate in an integrated, authentic assessment project. It is contended that immersive blended learning experiences support creative problem solving and encourages synchronous and asynchronous student participation in authentic problem solving and collaborative practice. Interacting with co-learners, students gain knowledge and skills through situated learning, defined as the application of knowledge, learned in one setting and transferred to another and where immersion in a virtual learning experience leads to higher level engagement on the transfer task in a real world setting. In this project, collaborative blended learning involved the creation of a collection of digital artworks by art education students using computer software located in a real world environment. These artworks were curated and exhibited by the students in a virtual gallery they designed and built on Deakin Arts Education island in Second Life. For public relations students, the virtual art exhibition was the focus of a virtual campaign, designed, researched and developed by them to promote the Deakin Virtual Art Gallery on Deakin island in Second Life. The final promotion for the Virtual Gallery was presented by the students at a symposium in both real world and virtual world environments.

Splitting centre : directing attention in trans-media dance performance

In theatrical vernacular, the term ‘splitting centre’ refers to two performers staged at an equal distance from a centre point and sharing the focus of the audience. This term encapsulates the notion that two people (or, in the case of trans-media dance, two or more performance entities) are dividing the attention of the audience, operating as equal collaborators in a performance context. The augmentation of live performance with 3D projected scenography and mobile devices offers a starting point for discussions on the potential for dramaturgy, choreographic process, and changing expectations for audience behaviour in the theatre. In 2014, Deakin Motion.Lab premiered The Crack Up, a trans-media dance work that incorporated live performance, 3D digital scenography, and The Crack Up App, an app for mobile devices that audience members were invited to interact with during the performance. This investigation into the potential of trans-media dance performance, (defined here as a live performance in which both the digital and biological elements are choreographed as artistic equals within the theatrical context) with the addition of a mobile device raises questions about how the makers of trans-media dance might direct the attention of their audiences when the work is performed simultaneously across multiple platforms.