Optometry training simulation with augmented reality and haptics

Optometry is an essential health care profession that has existed for many centuries and is still evolving. However, the training approaches for optometrists are not yet on par with the latest technological evolution. The traditional supervisor-student training mode could not provide good immersion and repeatability, while most existing vision-based computer-assisted simulations provide even worse immersion on screens. In this paper, we propose an effective system for optometry training simulation with two major components: augmented reality and haptics. These components are integrated with the actual slit lamp and are able to greatly enhance the immersion for typical optometry training tasks such as foreign body removal. Medical doctors are also involved in suggesting configurations and validating visual and haptic rendering results. Preliminary user studies show very positive feedbacks from optometry students.

Condensing a priori data for recognition based augmented reality

My research proposes novel methods to reduce the cardinality of a priori data used in recognition based augmented reality, whilst retaining distinctive and persistent features in the sets. This research will help reduce latency and increase accuracy in recognition based pose estimation systems, thus improving the user experience for augmented reality applications.

Immigration Museum application: augmented reality application with animated museum guide

Starting with the Immigration Museum, this app will support Multilingual Orientation Tours to each venue of Museum Victoria.

Bird flock

The work consists of a set of six, utilitarian wine glasses. Each glass features the engraving of a bird silhouette; the six engravings represent a flock of birds and also six key-poses in the animated flight-cycle of one bird. Beside the array of wine glasses sits a screen on which is displayed a looped animation of this same bird in flight. In a way, the six wine glasses represent the six animation cels of the displayed animation. Additionally, each engraved glass serves as a sight-trigger for a smart-phone or other mobile device. If one has the Aurasma app on their device and holds it up to ‘look at’ any one of the engravings, the animation of the flying bird will also be displayed on their device’s screen.

Augmented optometry training simulator with multi-point haptics

Training of optometrists is traditionally achieved under close supervision of peers and superiors. With the rapid advancement in technology, medical procedures are performed more efficiently and effectively, resulting in faster recovery times and less trauma to the patient. However, application of this technology has made it difficult to effectively demonstrate and teach these manual skills as the education is now a combination of not only the medical procedure but also the use of the technology. In this paper we propose to increase the training capabilities of optometry students through haptically-enabled single-point and multi-point training tools as well as augmented reality techniques. Haptics technology allows a human to touch and feel virtual computer models as though they are real. Through physical connection to the operator, haptic devices are considered to be personal robots that are capable of improving the human-computer interaction with a virtual environment. These devices have played an increasing role in developing expertise, reducing instances of medical error and reducing training costs. A haptically-enabled virtual training environment, integrated with an optometry slit lamp instrument can be used to teach cognitive and manual skills while the system tracks the performance of each individual. These interactions would ideally replicate every aspect of the real procedure, consequently preparing the trainee for every possible scenario, without risking the health of a real patient.

Computer-simulated environment for training : challenge of efficacy evaluation

Computer-assisted instruction has been around for decades. There has been much speculation about the benefits of computer-mediated learning. Numerous applications have been developed in different domains incorporated with emerging technologies. In recently years, advanced technologies, such as Augmented Reality (AR) and Virtual Reality (VR), have received much attention in their potential of creating interactive learning experience for the users. However, related literature and empirical studies indicated that learning effects in computer-simulated environments or Virtual Environments (VEs) are not systematically tested. Furthermore, the performance and learning in computer-simulated learning environment need to be evaluated through more rigorous methods. This paper suggests that 1) the efficacy of VEs is subject to a close examination, not only in terms of how VE-based training systems are easy of use, but also in terms of how effective learning is; 2) evaluation of learning in computer simulated learning environments is required to be reconsidered in terms of theoretical basis and evaluation methodologies that are relevant to the measurement of training effectiveness in computer-simulated virtual learning environment. This paper explains on how learning can be assessed in VEs through the lens of training evaluation.

3D sparse feature model using short baseline stereo and multiple view registration

This paper outlines a methodology to generate a distinctive object representation offline, using short-baseline stereo fundamentals to triangulate highly descriptive object features in multiple pairs of stereo images. A group of sparse 2.5D perspective views are built and the multiple views are then fused into a single sparse 3D model using a common 3D shape registration technique. Having prior knowledge, such as the proposed sparse feature model, is useful when detecting an object and estimating its pose for real-time systems like augmented reality.

Generation of 3D sparse feature models using multiple stereo views

Augmented Reality (AR) renders virtual information onto objects in the real world. This new user interface paradigm presents a seamless blend of the virtual and real, where the convergence of the two is difficult to discern. However, errors in the registration of the real and virtual worlds are common and often destroy the AR illusion. To achieve accurate and efficient registration, the pose of real objects must be resolved in a quick and precise manner.

Alison Bennett : shifting skin

Ten large exhibition prints with digital 3D augmented reality overlay

Shifting skin

AR[t] is published by the AR Lab. This article provides insight into the development of the exhibition 'Alison Bennett: Shifting Skin'