Real-time Image-based RGB-D Camera Motion Tracking and Environment Mapping

In this work, image based estimation methods, also known as direct methods, are studied which avoid feature extraction and matching completely. Cost functions use raw pixels as measurements and the goal is to produce precise 3D pose and structure estimates. The cost functions presented minimize the sensor error, because measurements are not transformed or modified. In photometric camera pose estimation, 3D rotation and translation parameters are estimated by minimizing a sequence of image based cost functions, which are non-linear due to perspective projection and lens distortion. In image based structure refinement, on the other hand, 3D structure is refined using a number of additional views and an image based cost metric. Image based estimation methods are particularly useful in conditions where the Lambertian assumption holds, and the 3D points have constant color despite viewing angle. The goal is to improve image based estimation methods, and to produce computationally efficient methods which can be accomodated into real-time applications. The developed image-based 3D pose and structure estimation methods are finally demonstrated in practise in indoor 3D reconstruction use, and in a live augmented reality application.

Probabilistic position and orientation estimation of rigid objects using an RGB-D sensor

The recent emergence of low-cost RGB-D sensors has brought new opportunities for robotics by providing affordable devices that can provide synchronized images with both color and depth information. In this thesis, recent work on pose estimation utilizing RGBD sensors is reviewed. Also, a pose recognition system for rigid objects using RGB-D data is implemented. The implementation uses half-edge primitives extracted from the RGB-D images for pose estimation. The system is based on the probabilistic object representation framework by Detry et al., which utilizes Nonparametric Belief Propagation for pose inference. Experiments are performed on household objects to evaluate the performance and robustness of the system.

Depth sensors in augmented reality solutions. Literature review

The emergence of depth sensors has made it possible to track – not only monocular cues – but also the actual depth values of the environment. This is especially useful in augmented reality solutions, where the position and orientation (pose) of the observer need to be accurately determined. This allows virtual objects to be installed to the view of the user through, for example, a screen of a tablet or augmented reality glasses (e.g. Google glass, etc.). Although the early 3D sensors have been physically quite large, the size of these sensors is decreasing, and possibly – eventually – a 3D sensor could be embedded – for example – to augmented reality glasses. The wider subject area considered in this review is 3D SLAM methods, which take advantage of the 3D information available by modern RGB-D sensors, such as Microsoft Kinect. Thus the review for SLAM (Simultaneous Localization and Mapping) and 3D tracking in augmented reality is a timely subject. We also try to find out the limitations and possibilities of different tracking methods, and how they should be improved, in order to allow efficient integration of the methods to the augmented reality solutions of the future.

Evaluation of Mobile Telephone Text Message Reminders for People with Antipsychotic Medication

Nonadherence to treatment is a worldwide problem among people with severe mental disorders. Patient treatment adherence may be supported with simple reminding methods e.g. text message reminders. However, there is limited evidence of its benefits. Intervention evaluation is essential in mHealth research. Therefore, this evaluative study was conducted. This study aimed to evaluate text message reminder use in encouraging patients’treatment adherence among people with antipsychotic medication. The data were collected between September 2011 and December 2013. First, a systematic literature review revealed that text message reminders were widely used in healthcare. However, its impacts were conflicting. Second, a sub-sample (n = 562) analysis showed that patients preferred humorous text message reminders and preferred to receive them in the morning, at the beginning of the week. Age, gender and marital status seemed to have different effects on the preferred amount and timing of the selected reminders. Third, a cross-sectional survey revealed that people with antipsychotic medication (n = 408) expressed overall satisfaction towards the reminder system. Finally, the evaluative design showed that patient recruitment for a randomized controlled trial concerning people with antipsychotic medication was challenging due to low rates of eligible participants. Follow-up drop-out rates varied depending on the data collection method. Participants’ demographic characteristics were associated with the risk of dropping out from the trial. This study suggests that text messages are a potential reminder system in healthcare services among people with antipsychotic medication. More research is needed to gain a comprehensive picture of the impacts and effectiveness of text message reminders.

Coarse Global Visual Localization of a Mobile Robot

Localization, which is the ability of a mobile robot to estimate its position within its environment, is a key capability for autonomous operation of any mobile robot. This thesis presents a system for indoor coarse and global localization of a mobile robot based on visual information. The system is based on image matching and uses SIFT features as natural landmarks. Features extracted from training images arestored in a database for use in localization later. During localization an image of the scene is captured using the on-board camera of the robot, features are extracted from the image and the best match is searched from the database. Feature matching is done using the k-d tree algorithm. Experimental results showed that localization accuracy increases with the number of training features used in the training database, while, on the other hand, increasing number of features tended to have a negative impact on the computational time. For some parts of the environment the error rate was relatively high due to a strong correlation of features taken from those places across the environment.

Performance analysis of software run-time behaviour using 3D-visualisation

Monimutkaisen tietokonejärjestelmän suorituskykyoptimointi edellyttää järjestelmän ajonaikaisen käyttäytymisen ymmärtämistä. Ohjelmiston koon ja monimutkaisuuden kasvun myötä suorituskykyoptimointi tulee yhä tärkeämmäksi osaksi tuotekehitysprosessia. Tehokkaampien prosessorien käytön myötä myös energiankulutus ja lämmöntuotto ovat nousseet yhä suuremmiksi ongelmiksi, erityisesti pienissä, kannettavissa laitteissa. Lämpö- ja energiaongelmien rajoittamiseksi on kehitetty suorituskyvyn skaalausmenetelmiä, jotka edelleen lisäävät järjestelmän kompleksisuutta ja suorituskykyoptimoinnin tarvetta. Tässä työssä kehitettiin visualisointi- ja analysointityökalu ajonaikaisen käyttäytymisen ymmärtämisen helpottamiseksi. Lisäksi kehitettiin suorituskyvyn mitta, joka mahdollistaa erilaisten skaalausmenetelmien vertailun ja arvioimisen suoritusympäristöstä riippumatta, perustuen joko suoritustallenteen tai teoreettiseen analyysiin. Työkalu esittää ajonaikaisesti kerätyn tallenteen helposti ymmärrettävällä tavalla. Se näyttää mm. prosessit, prosessorikuorman, skaalausmenetelmien toiminnan sekä energiankulutuksen kolmiulotteista grafiikkaa käyttäen. Työkalu tuottaa myös käyttäjän valitsemasta osasta suorituskuvaa numeerista tietoa, joka sisältää useita oleellisia suorituskykyarvoja ja tilastotietoa. Työkalun sovellettavuutta tarkasteltiin todellisesta laitteesta saatua suoritustallennetta sekä suorituskyvyn skaalauksen simulointia analysoimalla. Skaalausmekanismin parametrien vaikutus simuloidun laitteen suorituskykyyn analysoitiin.

Virtual technology and haptic interface solutions for design and control of mobile working machines

Commercially available haptic interfaces are usable for many purposes. However, as generic devices they are not the most suitable for the control of heavy duty mobile working machines like mining machines, container handling equipment and excavators. Alternative mechanical constructions for a haptic controller are presented and analysed. A virtual reality environment (VRE) was built to test the proposed haptic controller mechanisms. Verification of an electric motor emulating a hydraulic pump in the electro-hydraulic system of a mobile working machine is carried out. A real-time simulator using multi-body-dynamics based software with hardware-in-loop (HIL) setup was used for the tests. Recommendations for further development of a haptic controller and emulator electric motor are given.

Natiivin Android-pelin käyttöliittymän skaalautuminen ja suorituskyvyn optimointi

Tässä työssä esiteltiin Android laitteisto- ja sovellusalustana sekä kuvattiin, kuinka Android-pelisovelluksen käyttöliittymä voidaan pitää yhtenäisenä eri näyttölaitteilla skaalauskertoimien ja ankkuroinnin avulla. Toisena osiona työtä käsiteltiin yksinkertaisia tapoja, joilla pelisovelluksien suorituskykyä voidaan parantaa. Näistä tarkempiin mittauksiin valittiin matalatarkkuuksinen piirtopuskuri ja näkymättömissä olevien kappaleiden piilotus. Mittauksissa valitut menetelmät vaikuttivat demosovelluksen suorituskykyyn huomattavasti. Tässä työssä rajauduttiin Android-ohjelmointiin Java-kielellä ilman ulkoisia kirjastoja, jolloin työn tuloksia voi helposti hyödyntää mahdollisimman monessa eri käyttökohteessa.

Novel virtual environment and real-time simulation based methods for improving life-cycle efficiency of non-road mobile machinery

Virtual environments and real-time simulators (VERS) are becoming more and more important tools in research and development (R&D) process of non-road mobile machinery (NRMM). The virtual prototyping techniques enable faster and more cost-efficient development of machines compared to use of real life prototypes. High energy efficiency has become an important topic in the world of NRMM because of environmental and economic demands. The objective of this thesis is to develop VERS based methods for research and development of NRMM. A process using VERS for assessing effects of human operators on the life-cycle efficiency of NRMM was developed. Human in the loop simulations are ran using an underground mining loader to study the developed process. The simulations were ran in the virtual environment of the Laboratory of Intelligent Machines of Lappeenranta University of Technology. A physically adequate real-time simulation model of NRMM was shown to be reliable and cost effective in testing of hardware components by the means of hardware-in-the-loop (HIL) simulations. A control interface connecting integrated electro-hydraulic energy converter (IEHEC) with virtual simulation model of log crane was developed. IEHEC consists of a hydraulic pump-motor and an integrated electrical permanent magnet synchronous motorgenerator. The results show that state of the art real-time NRMM simulators are capable to solve factors related to energy consumption and productivity of the NRMM. A significant variation between the test drivers is found. The results show that VERS can be used for assessing human effects on the life-cycle efficiency of NRMM. HIL simulation responses compared to that achieved with conventional simulation method demonstrate the advances and drawbacks of various possible interfaces between the simulator and hardware part of the system under study. Novel ideas for arranging the interface are successfully tested and compared with the more traditional one. The proposed process for assessing the effects of operators on the life-cycle efficiency will be applied for wider group of operators in the future. Driving styles of the operators can be analysed statistically from sufficient large result data. The statistical analysis can find the most life-cycle efficient driving style for the specific environment and machinery. The proposed control interface for HIL simulation need to be further studied. The robustness and the adaptation of the interface in different situations must be verified. The future work will also include studying the suitability of the IEHEC for different working machines using the proposed HIL simulation method.

Energy Consumption of Applications on Mobile Phones

Battery consumption in mobile applications development is a very important aspect and has to be considered by all the developers in their applications. This study will present an analysis of different relevant concepts and parameters that may have impact on energy consumption of Windows Phone applications. This operating system was chosen because there is limited research even though there are related studies for Android an iOS operating systems. Furthermore, another reason is the increasing number of Windows Phone users. The objective of this research is to categorise the energy consumption parameters (e.g. use of one thread or several thread for the same output). The result for each group of experiment will be analyzed and a rule will be derived. The set of derived rules will serve as a guide for developers who intend to develop energy efficient Windows Phone applications. For each experiment, one application is created for each concept and the results are presented in two ways: a table and a chart. The table presents the duration of the experiment, the battery consumed by the experiment, the expected battery lifetime and the energy consumption, while the charts display the energy distribution based on the main threads: UI thread, application thread and network thread.

3D Mesh Simplification. A survey of algorithms and CAD model simplification tests

Simplification of highly detailed CAD models is an important step when CAD models are visualized or by other means utilized in augmented reality applications. Without simplification, CAD models may cause severe processing and storage is- sues especially in mobile devices. In addition, simplified models may have other advantages like better visual clarity or improved reliability when used for visual pose tracking. The geometry of CAD models is invariably presented in form of a 3D mesh. In this paper, we survey mesh simplification algorithms in general and focus especially to algorithms that can be used to simplify CAD models. We test some commonly known algorithms with real world CAD data and characterize some new CAD related simplification algorithms that have not been surveyed in previous mesh simplification reviews.

Development of 3D super-resolution tomographic STED microscopy and its application to studies on bone resorption

In this doctoral thesis, a tomographic STED microscopy technique for 3D super-resolution imaging was developed and utilized to observebone remodeling processes. To improve upon existing methods, wehave used a tomographic approach using a commercially available stimulated emission depletion (STED) microscope. A certain region of interest (ROI) was observed at two oblique angles: one at a standard inverted configuration from below (bottom view) and another from the side (side view) via a micro-mirror positioned close to the ROI. The two viewing angles were reconstructed into a final tomogram. The technique, named as tomographic STED microscopy, was able to achieve an axial resolution of approximately 70 nm on microtubule structures in a fixed biological specimen. High resolution imaging of osteoclasts (OCs) that are actively resorbing bone was achieved by creating an optically transparent coating on a microscope coverglass that imitates a fractured bone surface. 2D super-resolution STED microscopy on the bone layer showed approximately 60 nm of lateral resolution on a resorption associated organelle allowing these structures to be imaged with super-resolution microscopy for the first time. The developed tomographic STED microscopy technique was further applied to study resorption mechanisms of OCs cultured on the bone coating. The technique revealed actin cytoskeleton with specific structures, comet-tails, some of which were facing upwards and some others were facing downwards. This, in our opinion, indicated that during bone resorption, an involvement of the actin cytoskeleton in vesicular exocytosis and endocytosis is present. The application of tomographic STED microscopy in bone biology demonstrated that 3D super-resolution techniques can provide new insights into biological 3D nano-structures that are beyond the diffraction-limit when the optical constraints of super-resolution imaging are carefully taken into account.

Design of traction transmission and suspension systems for an omni-directional mobile robot

This project aims to design and manufacture a mobile robot with two Universal Robot UR10 mainly used indoors. In order to obtain omni-directional maneuverability, the mobile robot is constructed with Mecanum wheels. The Mecanum wheel can move in any direction with a series of rollers attached to itself. These rollers are angled at 45º about the hub’s circumference. This type of wheels can be used in both driving and steering with their any-direction property. This paper is focused on the design of traction system and suspension system, and the velocity control of Mecanum wheels in the close-loop control system. The mechanical design includes selection of bearing housing, couplers which are act as connection between shafts, motor parts, and other needed components. The 3D design software SolidWorks is utilized to assemble all the components in order to get correct tolerance. The driving shaft is designed based on assembled structure via the software as well. The design of suspension system is to compensate the assembly error of Mecanum wheels to guarantee the stability of the robot. The control system of motor drivers is realized through the Robot Operating System (ROS) on Ubuntu Linux. The purpose of inverse kinematics is to obtain the relationship among the movements of all Mecanum wheels. Via programming and interacting with the computer, the robot could move with required speed and direction.

Evaluation of Machine Learning Classifiers for Mobile Network Intrusion Detection Systems

Mobile malwares are increasing with the growing number of Mobile users. Mobile malwares can perform several operations which lead to cybersecurity threats such as, stealing financial or personal information, installing malicious applications, sending premium SMS, creating backdoors, keylogging and crypto-ransomware attacks. Knowing the fact that there are many illegitimate Applications available on the App stores, most of the mobile users remain careless about the security of their Mobile devices and become the potential victim of these threats. Previous studies have shown that not every antivirus is capable of detecting all the threats; due to the fact that Mobile malwares use advance techniques to avoid detection. A Network-based IDS at the operator side will bring an extra layer of security to the subscribers and can detect many advanced threats by analyzing their traffic patterns. Machine Learning(ML) will provide the ability to these systems to detect unknown threats for which signatures are not yet known. This research is focused on the evaluation of Machine Learning classifiers in Network-based Intrusion detection systems for Mobile Networks. In this study, different techniques of Network-based intrusion detection with their advantages, disadvantages and state of the art in Hybrid solutions are discussed. Finally, a ML based NIDS is proposed which will work as a subsystem, to Network-based IDS deployed by Mobile Operators, that can help in detecting unknown threats and reducing false positives. In this research, several ML classifiers were implemented and evaluated. This study is focused on Android-based malwares, as Android is the most popular OS among users, hence most targeted by cyber criminals. Supervised ML algorithms based classifiers were built using the dataset which contained the labeled instances of relevant features. These features were extracted from the traffic generated by samples of several malware families and benign applications. These classifiers were able to detect malicious traffic patterns with the TPR upto 99.6% during Cross-validation test. Also, several experiments were conducted to detect unknown malware traffic and to detect false positives. These classifiers were able to detect unknown threats with the Accuracy of 97.5%. These classifiers could be integrated with current NIDS', which use signatures, statistical or knowledge-based techniques to detect malicious traffic. Technique to integrate the output from ML classifier with traditional NIDS is discussed and proposed for future work.

Adherence to Treatment in Psychotic Disorders - Development of user-centered mobile health intervention

The aim of this study was to explore adherence to treatment among people with psychotic disorders through the development of user-centered mobile technology (mHealth) intervention. More specifically, this study investigates treatment adherence as well as mHealth intervention and the factors related to its possible usability. The data were collected from 2010 to 2013. First, patients’ and professionals’ perceptions of adherence management and restrictive factors of adherence were described (n = 61). Second, objectives and methods of the intervention were defined based on focus group interviews and previously used methods. Third, views of patients and professionals about barriers and requirements of the intervention were described (n = 61). Fourth, mHealth intervention was evaluated based on a literature review (n = 2) and patients preferences regarding the intervention (n = 562). Adherence management required support in everyday activities, social networks and maintaining a positive outlook. The factors restricting adherence were related to illness, behavior and the environment. The objective of the intervention was to support the intention to follow the treatment guidelines and recommendations with mHealth technology. The barriers and requirements for the use of the mHealth were related to technology, organizational issues and the users themselves. During the course of the intervention, 33 (6%) out of 562 participants wanted to edit the content, timing or amount of the mHealth tool, and 23 (4%) quit the intervention or study before its conclusion. According to the review, mHealth interventions were ineffective in promoting adherence. Prior to the intervention, participants perceived that adherence could be supported, and the use of mHealth as a part of treatment was seen as an acceptable and efficient method for doing so. In conclusion, the use of mHealth may be feasible among people with psychotic disorders. However, clear evidence for its effectiveness in regards to adherence is still currently inconclusive.