Integration of Real-Time Simulator, Motion Platform and Haptics

This thesis describes the process of the integration of a real-time simulator environment with a motion platform and a haptic device as a part of the Kvalive project. Several programs running on two computers were made to control the different devices of the environment. User tests were made to obtain information of needed improvements to make the simulator more realistic. Also new ideas for improving the simulator and directions of further research were obtained with the help of this research.

Novel virtual environment and real-time simulation based methods for improving life-cycle efficiency of non-road mobile machinery

Virtual environments and real-time simulators (VERS) are becoming more and more important tools in research and development (R&D) process of non-road mobile machinery (NRMM). The virtual prototyping techniques enable faster and more cost-efficient development of machines compared to use of real life prototypes. High energy efficiency has become an important topic in the world of NRMM because of environmental and economic demands. The objective of this thesis is to develop VERS based methods for research and development of NRMM. A process using VERS for assessing effects of human operators on the life-cycle efficiency of NRMM was developed. Human in the loop simulations are ran using an underground mining loader to study the developed process. The simulations were ran in the virtual environment of the Laboratory of Intelligent Machines of Lappeenranta University of Technology. A physically adequate real-time simulation model of NRMM was shown to be reliable and cost effective in testing of hardware components by the means of hardware-in-the-loop (HIL) simulations. A control interface connecting integrated electro-hydraulic energy converter (IEHEC) with virtual simulation model of log crane was developed. IEHEC consists of a hydraulic pump-motor and an integrated electrical permanent magnet synchronous motorgenerator. The results show that state of the art real-time NRMM simulators are capable to solve factors related to energy consumption and productivity of the NRMM. A significant variation between the test drivers is found. The results show that VERS can be used for assessing human effects on the life-cycle efficiency of NRMM. HIL simulation responses compared to that achieved with conventional simulation method demonstrate the advances and drawbacks of various possible interfaces between the simulator and hardware part of the system under study. Novel ideas for arranging the interface are successfully tested and compared with the more traditional one. The proposed process for assessing the effects of operators on the life-cycle efficiency will be applied for wider group of operators in the future. Driving styles of the operators can be analysed statistically from sufficient large result data. The statistical analysis can find the most life-cycle efficient driving style for the specific environment and machinery. The proposed control interface for HIL simulation need to be further studied. The robustness and the adaptation of the interface in different situations must be verified. The future work will also include studying the suitability of the IEHEC for different working machines using the proposed HIL simulation method.

Real-Time Simulation of Mobile and Industrial Machines Using the Multibody Simulation Approach

This thesis introduces a real-time simulation environment based on the multibody simulation approach. The environment consists of components that are used in conventional product development, including computer aided drawing, visualization, dynamic simulation and finite element software architecture, data transfer and haptics. These components are combined to perform as a coupled system on one platform. The environment is used to simulate mobile and industrial machines at different stages of a product life time. Consequently, the demands of the simulated scenarios vary. In this thesis, a real-time simulation environment based on the multibody approach is used to study a reel mechanism of a paper machine and a gantry crane. These case systems are used to demonstrate the usability of the real-time simulation environment for fault detection purposes and in the context of a training simulator. In order to describe the dynamical performance of a mobile or industrial machine, the nonlinear equations of motion must be defined. In this thesis, the dynamical behaviour of machines is modelled using the multibody simulation approach. A multibody system may consist of rigid and flexible bodies which are joined using kinematic joint constraints while force components are used to describe the actuators. The strength of multibody dynamics relies upon its ability to describe nonlinearities arising from wearing of the components, friction, large rotations or contact forces in a systematic manner. For this reason, the interfaces between subsystems such as mechanics, hydraulics and control systems of the mechatronic machine can be defined and analyzed in a straightforward manner.

Real-time PMU monitoring of an IEEE 5-Bus Network implemented on OPAL system

This thesis studies the state-of-the-art of phasor measurement units (PMUs) as well as their metrological requirements stated in the IEEE C37.118.1 and C37.118.2 Standards for guaranteeing correct measurement performances. Communication systems among PMUs and their possible applicability in the field of power quality (PQ) assessment are also investigated. This preliminary study is followed by an analysis of the working principle of real-time (RT) simulators and the importance of hardware-in-the-loop (HIL) implementation, examining the possible case studies specific for PMUs, including compliance tests which are one of the most important parts. The core of the thesis is focused on the implementation of a PMU model in the IEEE 5-bus network in Simulink and in the validation of the results using OPAL RT-4510 as a real-time simulator. An initial check allows one to get an idea about the goodness of the results in Simulink, comparing the PMU data with respect to the load-flow steady-state information. In this part, accuracy indices are also calculated for both voltage and current synchrophasors. The following part consists in the implementation of the same code in OPAL-RT 4510 simulator, after which an initial analysis is carried out in a qualitative way in order to get a sense of the goodness of the outcomes. Finally, the confirmation of the results is based on an examination of the attained voltage and current synchrophasors and accuracy indices coming from Simulink models and from OPAL system, using a Matlab script. This work also proposes suggestions for an upcoming operation of PMUs in a more complex system as the Digital Twin (DT) in order to improve the performances of the already-existing protection devices of the distribution system operator (DSO) for a future enhancement of power systems reliability.

Electrical Power Distribution System modeling with VHDL-AMS for the construction of a Real-Time Digital Simulator using FPGAS devices

This paper presents a distribution feeder simulation using VHDL-AMS, considering the standard IEEE 13 node test feeder admitted as an example. In an electronic spreadsheet all calculations are performed in order to develop the modeling in VHDL-AMS. The simulation results are compared in relation to the results from the well knowing MatLab/Simulink environment, in order to verify the feasibility of the VHDL-AMS modeling for a standard electrical distribution feeder, using the software SystemVision™. This paper aims to present the first major developments for a future Real-Time Digital Simulator applied to Electrical Power Distribution Systems. © 2012 IEEE.

Improvement of The Driver Simulator Control and Comparison Between Driver-Road-Vehicle Interaction in Real and Simulated Environment

The present doctoral thesis discusses the ways to improve the performance of driving simulator, provide objective measures for the road safety evaluation methodology based on driver’s behavior and response and investigates the drivers' adaptation to the driving assistant systems. The activities are divided into two macro areas; the driving simulation studies and on-road experiments. During the driving simulation experimentation, the classical motion cueing algorithm with logarithmic scale was implemented in the 2DOF motion cueing simulator and the motion cues were found desirable by the participants. In addition, it found out that motion stimuli could change the behaviour of the drivers in terms of depth/distance perception. During the on-road experimentations, The driver gaze behaviour was investigated to find the objective measures on the visibility of the road signs and reaction time of the drivers. The sensor infusion and the vehicle monitoring instruments were found useful for an objective assessment of the pavement condition and the drivers’ performance. In the last chapter of the thesis, the safety assessment during the use of level 1 automated driving “ACC” is discussed with the simulator and on-road experiment. The drivers’ visual behaviour was investigated in both studies with innovative classification method to find the epochs of the distraction of the drivers. The behavioural adaptation to ACC showed that drivers may divert their attention away from the driving task to engage in secondary, non-driving-related tasks.

Code offloading on real-time multimedia systems: a framework for handling code mobility and code offloading in a QoS aware environment

Actualmente, os smartphones e outros dispositivos móveis têm vindo a ser dotados com cada vez maior poder computacional, sendo capazes de executar um vasto conjunto de aplicações desde simples programas de para tirar notas até sofisticados programas de navegação. Porém, mesmo com a evolução do seu hardware, os actuais dispositivos móveis ainda não possuem as mesmas capacidades que os computadores de mesa ou portáteis. Uma possível solução para este problema é distribuir a aplicação, executando partes dela no dispositivo local e o resto em outros dispositivos ligados à rede. Adicionalmente, alguns tipos de aplicações como aplicações multimédia, jogos electrónicos ou aplicações de ambiente imersivos possuem requisitos em termos de Qualidade de Serviço, particularmente de tempo real. Ao longo desta tese é proposto um sistema de execução de código remota para sistemas distribuídos com restrições de tempo-real. A arquitectura proposta adapta-se a sistemas que necessitem de executar periodicamente e em paralelo mesmo conjunto de funções com garantias de tempo real, mesmo desconhecendo os tempos de execução das referidas funções. A plataforma proposta foi desenvolvida para sistemas móveis capazes de executar o Sistema Operativo Android.

A framework for realistic real-time walkthroughs in a VR distributed environment

Virtual and augmented reality (VR/AR) are increasingly being used in various business scenarios and are important driving forces in technology development. However the usage of these technologies in the home environment is restricted due to several factors including lack of low-cost (from the client point of view) highperformance solutions. In this paper we present a general client/server rendering architecture based on Real-Time concepts, including support for a wide range of client platforms and applications. The idea of focusing on the real-time behaviour of all components involved in distributed IP-based VR scenarios is new and has not been addressed before, except for simple sub-solutions. This is considered as “the most significant problem with the IP environment” [1]. Thus, the most important contribution of this research will be the holistic approach, in which networking, end-systems and rendering aspects are integrated into a cost-effective infrastructure for building distributed real-time VR applications on IP-based networks.

SPARTS: simulator for power aware and real-time systems

Real-time systems demand guaranteed and predictable run-time behaviour in order to ensure that no task has missed its deadline. Over the years we are witnessing an ever increasing demand for functionality enhancements in the embedded real-time systems. Along with the functionalities, the design itself grows more complex. Posed constraints, such as energy consumption, time, and space bounds, also require attention and proper handling. Additionally, efficient scheduling algorithms, as proven through analyses and simulations, often impose requirements that have significant run-time cost, specially in the context of multi-core systems. In order to further investigate the behaviour of such systems to quantify and compare these overheads involved, we have developed the SPARTS, a simulator of a generic embedded real- time device. The tasks in the simulator are described by externally visible parameters (e.g. minimum inter-arrival, sporadicity, WCET, BCET, etc.), rather than the code of the tasks. While our current implementation is primarily focused on our immediate needs in the area of power-aware scheduling, it is designed to be extensible to accommodate different task properties, scheduling algorithms and/or hardware models for the application in wide variety of simulations. The source code of the SPARTS is available for download at [1].

A framework for offloading real-time applications in a distributed environment

This work focuses on highly dynamic distributed systems with Quality of Service (QoS) constraints (most importantly real-time constraints). To that purpose, real-time applications may benefit from code offloading techniques, so that parts of the application can be offloaded and executed, as services, by neighbour nodes, which are willing to cooperate in such computations. These applications explicitly state their QoS requirements, which are translated into resource requirements, in order to evaluate the feasibility of accepting other applications in the system.

Real time manufacturing systems simulator

A área da simulação computacional teve um rápido crescimento desde o seu apareciment, sendo actualmente uma das ciências de gestão e de investigação operacional mais utilizadas. O seu princípio baseia-se na replicação da operação de processos ou sistemas ao longo de períodos de tempo, tornando-se assim uma metodologia indispensável para a resolução de variados problemas do mundo real, independentemente da sua complexidade. Das inúmeras áreas de aplicação, nos mais diversos campos, a que mais se destaca é a utilização em sistemas de produção, onde o leque de aplicações disponível é muito vasto. A sua aplicação tem vindo a ser utilizada para solucionar problemas em sistemas de produção, uma vez que permite às empresas ajustar e planear de uma maneira rápida, eficaz e ponderada as suas operações e os seus sistemas, permitindo assim uma rápida adaptação das mesmas às constantes mudanças das necessidades da economia global. As aplicações e packages de simulação têm seguindo as tendências tecnológicas pelo que é notório o recurso a tecnologias orientadas a objectos para o desenvolvimento das mesmas. Este estudo baseou-se, numa primeira fase, na recolha de informação de suporte aos conceitos de modelação e simulação, bem como a respectiva aplicação a sistemas de produção em tempo real. Posteriormente centralizou-se no desenvolvimento de um protótipo de uma aplicação de simulação de ambientes de fabrico em tempo real. O desenvolvimento desta ferramenta teve em vista eventuais fins pedagógicos e uma utilização a nível académico, sendo esta capaz de simular um modelo de um sistema de produção, estando também dotada de animação. Sem deixar de parte a possibilidade de integração de outros módulos ou, até mesmo, em outras plataformas, houve ainda a preocupação acrescida de que a sua implementação recorresse a metodologias de desenvolvimento orientadas a objectos.

Contextual Intelligent Load Management Considering Real-Time Pricing in a Smart Grid Environment

The use of demand response programs enables the adequate use of resources of small and medium players, bringing high benefits to the smart grid, and increasing its efficiency. One of the difficulties to proceed with this paradigm is the lack of intelligence in the management of small and medium size players. In order to make demand response programs a feasible solution, it is essential that small and medium players have an efficient energy management and a fair optimization mechanism to decrease the consumption without heavy loss of comfort, making it acceptable for the users. This paper addresses the application of real-time pricing in a house that uses an intelligent optimization module involving artificial neural networks.

The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment

This paper reports an experiment that investigated people"s body ownership of an avatar that was observed in a virtual mirror. Twenty subjects were recruited in a within-groups study where 10 first experienced a virtual character that synchronously reflected their upper-body movements as seen in a virtual mirror, and then an asynchronous condition where the mirror avatar displayed prerecorded actions, unrelated to those of the participant. The other 10 subjects experienced the conditions in the opposite order. In both conditions the participant could carry out actions that led to elevation above ground level, as seen from their first person perspective and correspondingly in the mirror. A rotating virtual fan eventually descended to 2m above the ground. The hypothesis was that synchronous mirror reflection would result in higher subjective sense of ownership. A questionnaire analysis showed that the body ownership illusion was significantly greater for thesynchronous than asynchronous condition. Additionally participants in the synchronous condition avoided collision with the descending fan significantly more often than those in the asynchronous condition. The results of this experiment are put into context within similar experiments on multisensory correlation and body ownership within cognitive neuroscience.

Real-time Image-based RGB-D Camera Motion Tracking and Environment Mapping

In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are particularly useful in conditions where the Lambertian assumption holds, and the 3D points have constant color despite viewing angle. The goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real-time applications. The developed image-based 3D pose and structure estimation methods are finally demonstrated in practise in indoor 3D reconstruction use, and in a live augmented reality application.

Teaching real-time with a scheduler simulator

In this paper we describe a scheduler simulator for real-time tasks, RTsim, that can be used as a tool to teach real-time scheduling algorithms. It simulates a variety of preprogrammed scheduling policies for single and multi-processor systems and simple algorithm variants introduced by its user. Using RTsim students can conduct experiments that will allow them to understand the effects of each policy given different load conditions and learn which policy is better for different workloads. We show how to use RTsim as a learning tool and the results achieved with its application on the Real-Time Systems course taught at the B.Sc. on Computer Science at Paulista State University - Unesp - at Rio Preto.