Deep convolutional neural networks for semantic video object segmentation

In this thesis, we propose to infer pixel-level labelling in video by utilising only object category information, exploiting the intrinsic structure of video data. Our motivation is the observation that image-level labels are much more easily to be acquired than pixel-level labels, and it is natural to find a link between the image level recognition and pixel level classification in video data, which would transfer learned recognition models from one domain to the other one. To this end, this thesis proposes two domain adaptation approaches to adapt the deep convolutional neural network (CNN) image recognition model trained from labelled image data to the target domain exploiting both semantic evidence learned from CNN, and the intrinsic structures of unlabelled video data. Our proposed approaches explicitly model and compensate for the domain adaptation from the source domain to the target domain which in turn underpins a robust semantic object segmentation method for natural videos. We demonstrate the superior performance of our methods by presenting extensive evaluations on challenging datasets comparing with the state-of-the-art methods.

Pretraining Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks (CNN) have become the state-of-the-art methods on many large scale visual recognition tasks. For a lot of practical applications, CNN architectures have a restrictive requirement: A huge amount of labeled data are needed for training. The idea of generative pretraining is to obtain initial weights of the network by training the network in a completely unsupervised way and then fine-tune the weights for the task at hand using supervised learning. In this thesis, a general introduction to Deep Neural Networks and algorithms are given and these methods are applied to classification tasks of handwritten digits and natural images for developing unsupervised feature learning. The goal of this thesis is to find out if the effect of pretraining is damped by recent practical advances in optimization and regularization of CNN. The experimental results show that pretraining is still a substantial regularizer, however, not a necessary step in training Convolutional Neural Networks with rectified activations. On handwritten digits, the proposed pretraining model achieved a classification accuracy comparable to the state-of-the-art methods.

Modelling non-industrial private forest landowners' strategic decision making by using logistic regression and neural networks: case of predicting the choice of forest taxation basis

Summary

The Connectivity of Musical Aptitude and Foreign Language Learning Skills: Neural and Behavioural Evidence

Given the structural and acoustical similarities between speech and music, and possible overlapping cerebral structures in speech and music processing, a possible relationship between musical aptitude and linguistic abilities, especially in terms of second language pronunciation skills, was investigated. Moreover, the laterality effect of the mother tongue was examined with both adults and children by means of dichotic listening scores. Finally, two event-related potential studies sought to reveal whether children with advanced second language pronunciation skills and higher general musical aptitude differed from children with less-advanced pronunciation skills and less musical aptitude in accuracy when preattentively processing mistuned triads and music / speech sound durations. The results showed a significant relationship between musical aptitude, English language pronunciation skills, chord discrimination ability, and sound-change-evoked brain activation in response to musical stimuli (durational differences and triad contrasts). Regular music practice may also have a modulatory effect on the brain’s linguistic organization and cause altered hemispheric functioning in those who have regularly practised music for years. Based on the present results, it is proposed that language skills, both in production and discrimination, are interconnected with perceptual musical skills.

Use of neural networks in robot positioning of large flexible redundant manipulators

Deflection compensation of flexible boom structures in robot positioning is usually done using tables containing the magnitude of the deflection with inverse kinematics solutions of a rigid structure. The number of table values increases greatly if the working area of the boom is large and the required positioning accuracy is high. The inverse kinematics problems are very nonlinear, and if the structure is redundant, in some cases it cannot be solved in a closed form. If the structural flexibility of the manipulator arms is taken into account, the problem is almost impossible to solve using analytical methods. Neural networks offer a possibility to approximate any linear or nonlinear function. This study presents four different methods of using neural networks in the static deflection compensation and inverse kinematics solution of a flexible hydraulically driven manipulator. The training information required for training neural networks is obtained by employing a simulation model that includes elasticity characteristics. The functionality of the presented methods is tested based on the simulated and measured results of positioning accuracy. The simulated positioning accuracy is tested in 25 separate coordinate points. For each point, the positioning is tested with five different mass loads. The mean positioning error of a manipulator decreased from 31.9 mm to 4.1 mm in the test points. This accuracy enables the use of flexible manipulators in the positioning of larger objects. The measured positioning accuracy is tested in 9 separate points using three different mass loads. The mean positioning error decreased from 10.6 mm to 4.7 mm and the maximum error from 27.5 mm to 11.0 mm.

Forecasting of wind speeds and directions with artificial neural networks

In this master’s thesis, wind speeds and directions were modeled with the aim of developing suitable models for hourly, daily, weekly and monthly forecasting. Artificial Neural Networks implemented in MATLAB software were used to perform the forecasts. Three main types of artificial neural network were built, namely: Feed forward neural networks, Jordan Elman neural networks and Cascade forward neural networks. Four sub models of each of these neural networks were also built, corresponding to the four forecast horizons, for both wind speeds and directions. A single neural network topology was used for each of the forecast horizons, regardless of the model type. All the models were then trained with real data of wind speeds and directions collected over a period of two years in the municipal region of Puumala in Finland. Only 70% of the data was used for training, validation and testing of the models, while the second last 15% of the data was presented to the trained models for verification. The model outputs were then compared to the last 15% of the original data, by measuring the mean square errors and sum square errors between them. Based on the results, the feed forward networks returned the lowest generalization errors for hourly, weekly and monthly forecasts of wind speeds; Jordan Elman networks returned the lowest errors when used for forecasting of daily wind speeds. Cascade forward networks gave the lowest errors when used for forecasting daily, weekly and monthly wind directions; Jordan Elman networks returned the lowest errors when used for hourly forecasting. The errors were relatively low during training of the models, but shot up upon simulation with new inputs. In addition, a combination of hyperbolic tangent transfer functions for both hidden and output layers returned better results compared to other combinations of transfer functions. In general, wind speeds were more predictable as compared to wind directions, opening up opportunities for further research into building better models for wind direction forecasting.

Inter organizational learning within and by innovation networks

The thesis deals with the phenomenon of learning between organizations in innovation networks that develop new products, services or processes. Inter organizational learning is studied especially at the level of the network. The role of the network can be seen as twofold: either the network is a context for inter organizational learning, if the learner is something else than the network (organization, group, individual), or the network itself is the learner. Innovations are regarded as a primary source of competitiveness and renewal in organizations. Networking has become increasingly common particularly because of the possibility to extend the resource base of the organization through partnerships and to concentrate on core competencies. Especially in innovation activities, networks provide the possibility to answer the complex needs of the customers faster and to share the costs and risks of the development work. Networked innovation activities are often organized in practice as distributed virtual teams, either within one organization or as cross organizational co operation. The role of technology is considered in the research mainly as an enabling tool for collaboration and learning. Learning has been recognized as one important collaborative process in networks or as a motivation for networking. It is even more important in the innovation context as an enabler of renewal, since the essence of the innovation process is creating new knowledge, processes, products and services. The thesis aims at providing enhanced understanding about the inter organizational learning phenomenon in and by innovation networks, especially concentrating on the network level. The perspectives used in the research are the theoretical viewpoints and concepts, challenges, and solutions for learning. The methods used in the study are literature reviews and empirical research carried out with semi structured interviews analyzed with qualitative content analysis. The empirical research concentrates on two different areas, firstly on the theoretical approaches to learning that are relevant to innovation networks, secondly on learning in virtual innovation teams. As a result, the research identifies insights and implications for learning in innovation networks from several viewpoints on organizational learning. Using multiple perspectives allows drawing a many sided picture of the learning phenomenon that is valuable because of the versatility and complexity of situations and challenges of learning in the context of innovation and networks. The research results also show some of the challenges of learning and possible solutions for supporting especially network level learning.

Deep generative models for facial keypoints detection

A new area of machine learning research called deep learning, has moved machine learning closer to one of its original goals: artificial intelligence and general learning algorithm. The key idea is to pretrain models in completely unsupervised way and finally they can be fine-tuned for the task at hand using supervised learning. In this thesis, a general introduction to deep learning models and algorithms are given and these methods are applied to facial keypoints detection. The task is to predict the positions of 15 keypoints on grayscale face images. Each predicted keypoint is specified by an (x,y) real-valued pair in the space of pixel indices. In experiments, we pretrained deep belief networks (DBN) and finally performed a discriminative fine-tuning. We varied the depth and size of an architecture. We tested both deterministic and sampled hidden activations and the effect of additional unlabeled data on pretraining. The experimental results show that our model provides better results than publicly available benchmarks for the dataset.

Päätöspuun käyttö tutkamaalien initialisoinnissa

Suomen ilmatilaa valvotaan reaaliaikaisesti, pääasiassa ilmavalvontatutkilla. Ilmatilassa on lentokoneiden lisäksi paljon muitakin kohteita, jotka tutka havaitsee. Tutka lähettää nämä tiedot edelleen ilmavalvontajärjestelmään. Ilmavalvontajärjestelmä käsittelee tiedot, sekä lähettää ne edelleen esitysjärjestelmään. Esitysjärjestelmässä tiedot esitetään synteettisinä merkkeinä, seurantoina joista käytetään nimitystä träkki. Näiden tietojen puitteissa sekä oman ammattitaitonsa perusteella ihmiset tekevät päätöksiä. Tämän työn tarkoituksena on tutkia tutkan havaintoja träkkien initialisointipisteessä siten, että voitaisiin määritellä tyypillinen rakenne sille mikä on oikea ja mikä väärä tai huono träkki. Tämän lisäksi tulisi ennustaa, mitkä Irakeista eivät aiheudu ilma- aluksista. Saadut tulokset voivat helpottaa työtä havaintojen tulkinnassa - jokainen lintuparvi ei ole ehdokas seurannaksi. Havaintojen luokittelu voidaan tehdä joko neurolaskennalla tai päätöspuulla. Neurolaskenta tehdään neuroverkoilla, jotka koostuvat neuroneista. Päätöspuu- luokittelijat ovat oppivia tietorakenteita kuten neuroverkotkin. Yleisin päätöpuu on binääripuu. Tämän työn tavoitteena on opettaa päätöspuuluokittelija havaintojen avulla siten, että se pystyy luokittelemaan väärät havainnot oikeista. Neurolaskennan mahdollisuuksia tässä työssä ei käsitellä kuin teoreettisesti. Työn tuloksena voi todeta, että päätöspuuluokittelijat ovat erittäin kykeneviä erottamaan oikeat havainnot vääristä. Vaikka tulokset olivat rohkaiseva, lisää tutkimusta tarvitaan määrittelemään luotettavammin tekijät, jotka parhaiten suorittavat luokittelun.

Application of Neural Network Methods in the Prediction of Stock Markets

The purpose of the research is to define practical profit which can be achieved using neural network methods as a prediction instrument. The thesis investigates the ability of neural networks to forecast future events. This capability is checked on the example of price prediction during intraday trading on stock market. The executed experiments show predictions of average 1, 2, 5 and 10 minutes’ prices based on data of one day and made by two different types of forecasting systems. These systems are based on the recurrent neural networks and back propagation neural nets. The precision of the predictions is controlled by the absolute error and the error of market direction. The economical effectiveness is estimated by a special trading system. In conclusion, the best structures of neural nets are tested with data of 31 days’ interval. The best results of the average percent of profit from one transaction (buying + selling) are 0.06668654, 0.188299453, 0.349854787 and 0.453178626, they were achieved for prediction periods 1, 2, 5 and 10 minutes. The investigation can be interesting for the investors who have access to a fast information channel with a possibility of every-minute data refreshment.