Real-Time Simulation of Mobile and Industrial Machines Using the Multibody Simulation Approach

This thesis introduces a real-time simulation environment based on the multibody simulation approach. The environment consists of components that are used in conventional product development, including computer aided drawing, visualization, dynamic simulation and finite element software architecture, data transfer and haptics. These components are combined to perform as a coupled system on one platform. The environment is used to simulate mobile and industrial machines at different stages of a product life time. Consequently, the demands of the simulated scenarios vary. In this thesis, a real-time simulation environment based on the multibody approach is used to study a reel mechanism of a paper machine and a gantry crane. These case systems are used to demonstrate the usability of the real-time simulation environment for fault detection purposes and in the context of a training simulator. In order to describe the dynamical performance of a mobile or industrial machine, the nonlinear equations of motion must be defined. In this thesis, the dynamical behaviour of machines is modelled using the multibody simulation approach. A multibody system may consist of rigid and flexible bodies which are joined using kinematic joint constraints while force components are used to describe the actuators. The strength of multibody dynamics relies upon its ability to describe nonlinearities arising from wearing of the components, friction, large rotations or contact forces in a systematic manner. For this reason, the interfaces between subsystems such as mechanics, hydraulics and control systems of the mechatronic machine can be defined and analyzed in a straightforward manner.

Finite-temperature T matrix in a real-time formalism

A generalized off-shell unitarity relation for the two-body scattering T matrix in a many-body medium at finite temperature is derived, through a consistent real-time perturbation expansion by means of Feynman diagrams. We comment on perturbation schemes at finite temperature in connection with an erroneous formulation of the Dyson equation in a paper recently published.

Real-time Musical Pair Improvisation on Mobile Devices

This thesis discusses the design and implementation of a real-time musical pair improvisation scenario for mobile devices. In the scenario transferring musical information over a network connection was required. The suitability of available wireless communication technologies was evaluated and communication was analyzed and designed on multiple layers of TCP/IP protocol stack. Also an application layer protocol was designed and implemented for the scenario. The implementation was integrated into a mobile musical software for children using available software components and libraries although the used platform lead to hardware and software constraints. Software limitations were taken into account in design. The results show that real-time musical improvisation can be implemented with wireless communication and mobile technology. The results also show that link layer had the most significant effect on real-time communication in the scenario.

A Soft Contact Collision Method For Real-Time Simulation of Triangularized Geometries in Multibody Dynamics

When modeling machines in their natural working environment collisions become a very important feature in terms of simulation accuracy. By expanding the simulation to include the operation environment, the need for a general collision model that is able to handle a wide variety of cases has become central in the development of simulation environments. With the addition of the operating environment the challenges for the collision modeling method also change. More simultaneous contacts with more objects occur in more complicated situations. This means that the real-time requirement becomes more difficult to meet. Common problems in current collision modeling methods include for example dependency on the geometry shape or mesh density, calculation need increasing exponentially in respect to the number of contacts, the lack of a proper friction model and failures due to certain configurations like closed kinematic loops. All these problems mean that the current modeling methods will fail in certain situations. A method that would not fail in any situation is not very realistic but improvements can be made over the current methods.

Real-time Image-based RGB-D Camera Motion Tracking and Environment Mapping

In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are particularly useful in conditions where the Lambertian assumption holds, and the 3D points have constant color despite viewing angle. The goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real-time applications. The developed image-based 3D pose and structure estimation methods are finally demonstrated in practise in indoor 3D reconstruction use, and in a live augmented reality application.

Comparison between dSPACE and NI systems based on real-time intelligent control of a teleoperated hydraulic servo system

The Laboratory of Intelligent Machine researches and develops energy-efficient power transmissions and automation for mobile construction machines and industrial processes. The laboratory's particular areas of expertise include mechatronic machine design using virtual technologies and simulators and demanding industrial robotics. The laboratory has collaborated extensively with industrial actors and it has participated in significant international research projects, particularly in the field of robotics. For years, dSPACE tools were the lonely hardware which was used in the lab to develop different control algorithms in real-time. dSPACE's hardware systems are in widespread use in the automotive industry and are also employed in drives, aerospace, and industrial automation. But new competitors are developing new sophisticated systems and their features convinced the laboratory to test new products. One of these competitors is National Instrument (NI). In order to get to know the specifications and capabilities of NI tools, an agreement was made to test a NI evolutionary system. This system is used to control a 1-D hydraulic slider. The objective of this research project is to develop a control scheme for the teleoperation of a hydraulically driven manipulator, and to implement a control algorithm between human and machine interaction, and machine and task environment interaction both on NI and dSPACE systems simultaneously and to compare the results.

Evaluation of stunned and infarcted canine myocardium by real time myocardial contrast echocardiography

Differentiation between stunned and infarcted myocardium in the setting of acute ischemia is challenging. Real time myocardial contrast echocardiography allows the simultaneous assessment of myocardial perfusion and function. In the present study we evaluated infarcted and stunned myocardium in an experimental model using real time myocardial contrast echocardiography. Sixteen dogs underwent 180 min of coronary occlusion followed by reperfusion (infarct model) and seven other dogs were submitted to 20 min of coronary occlusion followed by reperfusion (stunned model). Wall motion abnormality and perfusional myocardial defect areas were measured by planimetry. Risk and infarct areas were determined by tissue staining. In the infarct model, the wall motion abnormality area during coronary occlusion (5.52 ± 1.14 cm²) was larger than the perfusional myocardial defect area (3.71 ± 1.45 cm²; P < 0.001). Reperfusion resulted in maintenance of wall motion abnormality (5.45 ± 1.41 cm²; P = 0.43 versus occlusion) and reduction of perfusional myocardial defect (1.51 ± 1.29 cm²; P = 0.004 versus occlusion). Infarct size determined by contrast echocardiography correlated with tissue staining (r = 0.71; P = 0.002). In the stunned model, the wall motion abnormality area was 5.49 ± 0.68 cm² during occlusion and remained 5.1 ± 0.63 cm² after reperfusion (P = 0.07). Perfusional defect area was 2.43 ± 0.79 cm² during occlusion and was reduced to 0.2 ± 0.53 cm² after reperfusion (P = 0.04). 2,3,5-Triphenyl tetrazolium chloride staining confirmed the absence of necrotic myocardium in all dogs in the stunned model. Real time myocardial contrast echocardiography is a noninvasive technique capable of distinguishing between stunned and infarcted myocardium after acute ischemia.

Novel virtual environment and real-time simulation based methods for improving life-cycle efficiency of non-road mobile machinery

Virtual environments and real-time simulators (VERS) are becoming more and more important tools in research and development (R&D) process of non-road mobile machinery (NRMM). The virtual prototyping techniques enable faster and more cost-efficient development of machines compared to use of real life prototypes. High energy efficiency has become an important topic in the world of NRMM because of environmental and economic demands. The objective of this thesis is to develop VERS based methods for research and development of NRMM. A process using VERS for assessing effects of human operators on the life-cycle efficiency of NRMM was developed. Human in the loop simulations are ran using an underground mining loader to study the developed process. The simulations were ran in the virtual environment of the Laboratory of Intelligent Machines of Lappeenranta University of Technology. A physically adequate real-time simulation model of NRMM was shown to be reliable and cost effective in testing of hardware components by the means of hardware-in-the-loop (HIL) simulations. A control interface connecting integrated electro-hydraulic energy converter (IEHEC) with virtual simulation model of log crane was developed. IEHEC consists of a hydraulic pump-motor and an integrated electrical permanent magnet synchronous motorgenerator. The results show that state of the art real-time NRMM simulators are capable to solve factors related to energy consumption and productivity of the NRMM. A significant variation between the test drivers is found. The results show that VERS can be used for assessing human effects on the life-cycle efficiency of NRMM. HIL simulation responses compared to that achieved with conventional simulation method demonstrate the advances and drawbacks of various possible interfaces between the simulator and hardware part of the system under study. Novel ideas for arranging the interface are successfully tested and compared with the more traditional one. The proposed process for assessing the effects of operators on the life-cycle efficiency will be applied for wider group of operators in the future. Driving styles of the operators can be analysed statistically from sufficient large result data. The statistical analysis can find the most life-cycle efficient driving style for the specific environment and machinery. The proposed control interface for HIL simulation need to be further studied. The robustness and the adaptation of the interface in different situations must be verified. The future work will also include studying the suitability of the IEHEC for different working machines using the proposed HIL simulation method.

Bioeffects of albumin-encapsulated microbubbles and real-time myocardial contrast echocardiography in an experimental canine model

Myocardial contrast echocardiography has been used for assessing myocardial perfusion. Some concerns regarding its safety still remain, mainly regarding the induction of microvascular alterations. We sought to determine the bioeffects of microbubbles and real-time myocardial contrast echocardiography (RTMCE) in a closed-chest canine model. Eighteen mongrel dogs were randomly assigned to two groups. Nine were submitted to continuous intravenous infusion of perfluorocarbon-exposed sonicated dextrose albumin (PESDA) plus continuous imaging using power pulse inversion RTMCE for 180 min, associated with manually deflagrated high-mechanical index impulses. The control group consisted of 3 dogs submitted to continuous imaging using RTMCE without PESDA, 3 dogs received PESDA alone, and 3 dogs were sham-operated. Hemodynamics and cardiac rhythm were monitored continuously. Histological analysis was performed on cardiac and pulmonary tissues. No hemodynamic changes or cardiac arrhythmias were observed in any group. Normal left ventricular ejection fraction and myocardial perfusion were maintained throughout the protocol. Frequency of mild and focal microhemorrhage areas in myocardial and pulmonary tissue was similar in PESDA plus RTMCE and control groups. The percentages of positive microscopical fields in the myocardium were 0.4 and 0.7% (P = NS) in the PESDA plus RTMCE and control groups, respectively, and in the lungs they were 2.1 and 1.1%, respectively (P = NS). In this canine model, myocardial perfusion imaging obtained with PESDA and RTMCE was safe, with no alteration in cardiac rhythm or left ventricular function. Mild and focal myocardial and pulmonary microhemorrhages were observed in both groups, and may be attributed to surgical tissue manipulation.

Real-time simulation of multibody systems with applications for working mobile vehicles

This dissertation describes an approach for developing a real-time simulation for working mobile vehicles based on multibody modeling. The use of multibody modeling allows comprehensive description of the constrained motion of the mechanical systems involved and permits real-time solving of the equations of motion. By carefully selecting the multibody formulation method to be used, it is possible to increase the accuracy of the multibody model while at the same time solving equations of motion in real-time. In this study, a multibody procedure based on semi-recursive and augmented Lagrangian methods for real-time dynamic simulation application is studied in detail. In the semirecursive approach, a velocity transformation matrix is introduced to describe the dependent coordinates into relative (joint) coordinates, which reduces the size of the generalized coordinates. The augmented Lagrangian method is based on usage of global coordinates and, in that method, constraints are accounted using an iterative process. A multibody system can be modelled as either rigid or flexible bodies. When using flexible bodies, the system can be described using a floating frame of reference formulation. In this method, the deformation mode needed can be obtained from the finite element model. As the finite element model typically involves large number of degrees of freedom, reduced number of deformation modes can be obtained by employing model order reduction method such as Guyan reduction, Craig-Bampton method and Krylov subspace as shown in this study The constrained motion of the working mobile vehicles is actuated by the force from the hydraulic actuator. In this study, the hydraulic system is modeled using lumped fluid theory, in which the hydraulic circuit is divided into volumes. In this approach, the pressure wave propagation in the hoses and pipes is neglected. The contact modeling is divided into two stages: contact detection and contact response. Contact detection determines when and where the contact occurs, and contact response provides the force acting at the collision point. The friction between tire and ground is modelled using the LuGre friction model, which describes the frictional force between two surfaces. Typically, the equations of motion are solved in the full matrices format, where the sparsity of the matrices is not considered. Increasing the number of bodies and constraint equations leads to the system matrices becoming large and sparse in structure. To increase the computational efficiency, a technique for solution of sparse matrices is proposed in this dissertation and its implementation demonstrated. To assess the computing efficiency, augmented Lagrangian and semi-recursive methods are implemented employing a sparse matrix technique. From the numerical example, the results show that the proposed approach is applicable and produced appropriate results within the real-time period.

Aikaerotteisesti fluoresenssia mittaavan reaaliaikaisen PCR-laitteen kehitys

Kvantitatiivinen reaaliaikainen polymeraasiketjureaktio (engl. polymerase chain reaction, PCR) on osoittautunut käyttäjäystävällisimmäksi menetelmäksi nukleiinihapposekvenssien kvantitoimisessa. Tätä menetelmää voidaan herkistää pienempien DNA-pitoisuuksien havaitsemiseen käyttämällä hyväksi aikaerotteista fluorometriaa (engl. time-resolved fluorometry, TRF) ja luminoivia lantanidileimoja, joiden fluoresenssin pitkän eliniän ansiosta emission mittaus voidaan suorittaa vasta hetki virittävän valopulssin jälkeen, jolloin lyhytikäinen taustasäteily ehtii sammua. Tuloksena saadaan korkea signaali-taustasuhde. Tämän diplomityön tarkoituksena oli rakentaa TRF:än pystyvä reaaliaikainen PCR-laite, sillä tällaista laitetta ei ole markkinoilla tarjolla. Laite rakennettiin kehittämällä lämpökierrätin ja yhdistämällä se valmiiseen TRF:än kykenevään mittapäähän. Mittapään ja lämpökierrättimen hallitsemiseksi kehitettiin myös tietokoneohjelma. Valon tuottamiseksi ja mittaamiseksi haluttiin käyttää edullisia komponentteja, joten työssä käytettiin valmiin mittapään optiikkaa, jossa viritys tapahtuu hohtodiodilla (engl. light-emitting diode, LED) ja lantanidileiman emission mittaus fotodiodilla (engl. photodiode, PD) tai valomonistinputkella (engl. photomultiplier tube, PMT). Myös mittapään suorituskykyä tutkittiin. Työtä varten kehitettiin lämpökierrätin, joka koostui Peltier-elementillä lämmitettävästä PCR-putkitelineestä ja lämpökannesta. Mittalaitteen suorituskyvyn tutkimiseen käytettiin kelaattikomplementaatioon perustuvaa PCR-tuotteen havaitsemismenetelmää. Kelaattikomplementaatio perustuu kahteen erilliseen oligonukleotidimolekyyliin, joista toiseen on sidottu lantanidi-ioni ja toiseen valoa absorboiva ligandirakenne, jotka yhdessä muodostavat fluoresoivan kokonaisuuden. Kehitetyn lämpökierrättimen todettiin olevan tarpeeksi tarkka sekä tehokas ja sen lämmitys- ja jäähdytysnopeuden maksimeiksi saatiin 2,6 °C/sekunti. Detektorina käytetyn PD:n ei todettu olevan tarpeeksi herkkä emission havainnoimiseksi ja se korvattiin laitteessa PMT:llä. Käytetyllä PCR-määrityksellä kynnyssykleiksi (engl. threshold cycle, Ct) sekä kehitetylle että referenssilaitteelle saatiin 28,4 käyttämällä samaa 100 000 kopion DNA:n aloitusmäärää. Työssä osoitettiin, että on mahdollista kehittää edullisia komponentteja käyttävä, TRF:än pystyvä, reaaliaikainen PCR-laite, joka kykenee vastaavaan Ct-arvoon kuin vertailulaite. PD:n herkkyys ei kuitenkaan riittänyt. Tulokset olivat lupaavia, sillä LED- ja PD-teknologiat kehittyvät ja markkinoille on tullut myös muita komponentteja, joiden avulla on tulevaisuudessa mahdollista kehittää vielä herkempi laite.

Real-time adaptive control of ultra-fast laser scribing process with spectrometer online monitoring

There exist several researches and applications about laser welding monitoring and parameter control but not a single one have been created for controlling of laser scribing processes. Laser scribing is considered to be very fast and accurate process and thus it would be necessary to develop accurate turning and monitoring system for such a process. This research focuses on finding out whether it would be possible to develop real-time adaptive control for ultra-fast laser scribing processes utilizing spectrometer online monitoring. The thesis accurately presents how control code for laser parameter tuning is developed using National Instrument's LabVIEW and how spectrometer is being utilized in online monitoring. Results are based on behavior of the control code and accuracy of the spectrometer monitoring when scribing different steel materials. Finally control code success is being evaluated and possible development ideas for future are presented.

Real-time monitoring of laser scribing process utilizing high-speed camera

Currently, laser scribing is growing material processing method in the industry. Benefits of laser scribing technology are studied for example for improving an efficiency of solar cells. Due high-quality requirement of the fast scribing process, it is important to monitor the process in real time for detecting possible defects during the process. However, there is a lack of studies of laser scribing real time monitoring. Commonly used monitoring methods developed for other laser processes such a laser welding, are sufficient slow and existed applications cannot be implemented in fast laser scribing monitoring. The aim of this thesis is to find a method for laser scribing monitoring with a high-speed camera and evaluate reliability and performance of the developed monitoring system with experiments. The laser used in experiments is an IPG ytterbium pulsed fiber laser with 20 W maximum average power and Scan head optics used in the laser is Scanlab’s Hurryscan 14 II with an f100 tele-centric lens. The camera was connected to laser scanner using camera adapter to follow the laser process. A powerful fully programmable industrial computer was chosen for executing image processing and analysis. Algorithms for defect analysis, which are based on particle analysis, were developed using LabVIEW system design software. The performance of the algorithms was analyzed by analyzing a non-moving image from the scribing line with resolution 960x20 pixel. As a result, the maximum analysis speed was 560 frames per second. Reliability of the algorithm was evaluated by imaging scribing path with a variable number of defects 2000 mm/s when the laser was turned off and image analysis speed was 430 frames per second. The experiment was successful and as a result, the algorithms detected all defects from the scribing path. The final monitoring experiment was performed during a laser process. However, it was challenging to get active laser illumination work with the laser scanner due physical dimensions of the laser lens and the scanner. For reliable error detection, the illumination system is needed to be replaced.

Real-Time Automatic Object Classification and Tracking using Genetic Programming and NVIDIA R CUDA TM

Genetic Programming (GP) is a widely used methodology for solving various computational problems. GP's problem solving ability is usually hindered by its long execution times. In this thesis, GP is applied toward real-time computer vision. In particular, object classification and tracking using a parallel GP system is discussed. First, a study of suitable GP languages for object classification is presented. Two main GP approaches for visual pattern classification, namely the block-classifiers and the pixel-classifiers, were studied. Results showed that the pixel-classifiers generally performed better. Using these results, a suitable language was selected for the real-time implementation. Synthetic video data was used in the experiments. The goal of the experiments was to evolve a unique classifier for each texture pattern that existed in the video. The experiments revealed that the system was capable of correctly tracking the textures in the video. The performance of the system was on-par with real-time requirements.