Forecasting of wind speeds and directions with artificial neural networks

In this master’s thesis, wind speeds and directions were modeled with the aim of developing suitable models for hourly, daily, weekly and monthly forecasting. Artificial Neural Networks implemented in MATLAB software were used to perform the forecasts. Three main types of artificial neural network were built, namely: Feed forward neural networks, Jordan Elman neural networks and Cascade forward neural networks. Four sub models of each of these neural networks were also built, corresponding to the four forecast horizons, for both wind speeds and directions. A single neural network topology was used for each of the forecast horizons, regardless of the model type. All the models were then trained with real data of wind speeds and directions collected over a period of two years in the municipal region of Puumala in Finland. Only 70% of the data was used for training, validation and testing of the models, while the second last 15% of the data was presented to the trained models for verification. The model outputs were then compared to the last 15% of the original data, by measuring the mean square errors and sum square errors between them. Based on the results, the feed forward networks returned the lowest generalization errors for hourly, weekly and monthly forecasts of wind speeds; Jordan Elman networks returned the lowest errors when used for forecasting of daily wind speeds. Cascade forward networks gave the lowest errors when used for forecasting daily, weekly and monthly wind directions; Jordan Elman networks returned the lowest errors when used for hourly forecasting. The errors were relatively low during training of the models, but shot up upon simulation with new inputs. In addition, a combination of hyperbolic tangent transfer functions for both hidden and output layers returned better results compared to other combinations of transfer functions. In general, wind speeds were more predictable as compared to wind directions, opening up opportunities for further research into building better models for wind direction forecasting.

Efficient Optimization Algorithms for Nonlinear Data Analysis

Identification of low-dimensional structures and main sources of variation from multivariate data are fundamental tasks in data analysis. Many methods aimed at these tasks involve solution of an optimization problem. Thus, the objective of this thesis is to develop computationally efficient and theoretically justified methods for solving such problems. Most of the thesis is based on a statistical model, where ridges of the density estimated from the data are considered as relevant features. Finding ridges, that are generalized maxima, necessitates development of advanced optimization methods. An efficient and convergent trust region Newton method for projecting a point onto a ridge of the underlying density is developed for this purpose. The method is utilized in a differential equation-based approach for tracing ridges and computing projection coordinates along them. The density estimation is done nonparametrically by using Gaussian kernels. This allows application of ridge-based methods with only mild assumptions on the underlying structure of the data. The statistical model and the ridge finding methods are adapted to two different applications. The first one is extraction of curvilinear structures from noisy data mixed with background clutter. The second one is a novel nonlinear generalization of principal component analysis (PCA) and its extension to time series data. The methods have a wide range of potential applications, where most of the earlier approaches are inadequate. Examples include identification of faults from seismic data and identification of filaments from cosmological data. Applicability of the nonlinear PCA to climate analysis and reconstruction of periodic patterns from noisy time series data are also demonstrated. Other contributions of the thesis include development of an efficient semidefinite optimization method for embedding graphs into the Euclidean space. The method produces structure-preserving embeddings that maximize interpoint distances. It is primarily developed for dimensionality reduction, but has also potential applications in graph theory and various areas of physics, chemistry and engineering. Asymptotic behaviour of ridges and maxima of Gaussian kernel densities is also investigated when the kernel bandwidth approaches infinity. The results are applied to the nonlinear PCA and to finding significant maxima of such densities, which is a typical problem in visual object tracking.

Detection of renal dysfunction based on serum creatinine levels in a Brazilian community: the Bambuí Health and Ageing Study

There are few population-based studies of renal dysfunction and none conducted in developing countries. In the present study the prevalence and predictors of elevated serum creatinine levels (SCr > or = 1.3 mg/dl for men and 1.1 mg/dl for women) were determined among Brazilian adults (18-59 years) and older adults (>60 years). Participants included all older adults (N = 1742) and a probabilistic sample of adults (N = 818) from Bambuí town, MG, Southeast Brazil. Predictors were investigated using multiple logistic regression. Mean SCr levels were 0.77 ± 0.15 mg/dl for adults, 1.02 ± 0.39 mg/dl for older men, and 0.81 ± 0.17 mg/dl for older women. Because there were only 4 cases (0.48%) with elevated SCr levels among adults, the analysis of elevated SCr levels was restricted to older adults. The overall prevalence of elevated SCr levels among the elderly was 5.09% (76/1494). The prevalence of hypercreatinemia increased significantly with age (chi² = 26.17, P = 0.000), being higher for older men (8.19%) than for older women (5.29%, chi² = 5.00, P = 0.02). Elevated SCr levels were associated with age 70-79 years (odds ratio [OR] = 2.25, 95% confidence interval [CI]: 1.15-4.42), hypertension (OR = 3.04, 95% CI: 1.34-6.92), use of antihypertensive drugs (OR = 2.46, 95% CI: 1.26-4.82), chest pain (OR = 3.37, 95% CI: 1.31-8.74), and claudication (OR = 3.43, 95% CI: 1.30-9.09) among men, and with age >80 years (OR = 4.88, 95% CI: 2.24-10.65), use of antihypertensive drugs (OR = 4.06, 95% CI: 1.67-9.86), physical inactivity (OR = 2.11, 95% CI: 1.11-4.02) and myocardial infarction (OR = 3.89, 95% CI: 1.58-9.62) among women. The prevalence of renal dysfunction observed was much lower than that reported in other population-based studies, but predictors were similar. New investigations are needed to confirm the variability in prevalence and associated factors of renal dysfunction among populations.

Parameter identification in time series models

Time series analysis can be categorized into three different approaches: classical, Box-Jenkins, and State space. Classical approach makes a basement for the analysis and Box-Jenkins approach is an improvement of the classical approach and deals with stationary time series. State space approach allows time variant factors and covers up a broader area of time series analysis. This thesis focuses on parameter identifiablity of different parameter estimation methods such as LSQ, Yule-Walker, MLE which are used in the above time series analysis approaches. Also the Kalman filter method and smoothing techniques are integrated with the state space approach and MLE method to estimate parameters allowing them to change over time. Parameter estimation is carried out by repeating estimation and integrating with MCMC and inspect how well different estimation methods can identify the optimal model parameters. Identification is performed in probabilistic and general senses and compare the results in order to study and represent identifiability more informative way.

Use of alcohol among the inhabitants of the 107 largest cities in Brazil - 2001

Alcohol is part of the history of humanity, seemingly as a result of countless factors including the easy production of alcoholic beverages in practically all regions of the world. The authors studied aspects of the use of and the dependence on alcohol in Brazil, through a household survey conducted by Centro Brasileiro de Informações sobre Drogas Psicotrópicas (CEBRID). A total of 8,589 interviews were held in 107 of the largest cities in Brazil, all of them with more than 200 thousand inhabitants. The study was planned to gather information within the household environment about a stratified probabilistic sample obtained in three selection phases: 1) the censitaire sectors for each municipality, 2) a systematic randomized sampling, and 3) drafting a respondent by lot in each household to provide information. Approximately 11.2% of the subjects were concerned with their own consumption of alcohol. The signs/symptoms of the syndrome of dependence evident in a greater percentage were the desire to stop or reduce the use of alcohol and to stop or reduce resorting to alcoholic beverages more often than desired, as reported by 14.5 and 9.4% of the respondents, respectively. The regions in Brazil with the highest percentage of dependents were the North (16.3%) and the Northeast (19.9%). According to the estimates obtained in the survey, 5.2% of the teenagers were concerned about the use of alcohol. The estimates obtained in this survey reveal a need to implant specific preventive programs for the problem of alcohol, especially for the very young.

Operant serial feature-positive conditional discrimination with composite features and different topography responses

In a serial feature-positive conditional discrimination procedure the properties of a target stimulus A are defined by the presence or not of a feature stimulus X preceding it. In the present experiment, composite features preceded targets associated with two different topography operant responses (right and left bar pressing); matching and non-matching-to-sample arrangements were also used. Five water-deprived Wistar rats were trained in 6 different trials: X-R®Ar and X-L®Al, in which X and A were same modality visual stimuli and the reinforcement was contingent to pressing either the right (r) or left (l) bar that had the light on during the feature (matching-to-sample); Y-R®Bl and Y-L®Br, in which Y and B were same modality auditory stimuli and the reinforcement was contingent to pressing the bar that had the light off during the feature (non-matching-to-sample); A- and B- alone. After 100 training sessions, the animals were submitted to transfer tests with the targets used plus a new one (auditory click). Average percentages of stimuli with a response were measured. Acquisition occurred completely only for Y-L®Br+; however, complex associations were established along training. Transfer was not complete during the tests since concurrent effects of extinction and response generalization also occurred. Results suggest the use of both simple conditioning and configurational strategies, favoring the most recent theories of conditional discrimination learning. The implications of the use of complex arrangements for discussing these theories are considered.

Synergistic control of forearm based on accelerometer data and artificial neural networks

In the present study, we modeled a reaching task as a two-link mechanism. The upper arm and forearm motion trajectories during vertical arm movements were estimated from the measured angular accelerations with dual-axis accelerometers. A data set of reaching synergies from able-bodied individuals was used to train a radial basis function artificial neural network with upper arm/forearm tangential angular accelerations. The trained radial basis function artificial neural network for the specific movements predicted forearm motion from new upper arm trajectories with high correlation (mean, 0.9149-0.941). For all other movements, prediction was low (range, 0.0316-0.8302). Results suggest that the proposed algorithm is successful in generalization over similar motions and subjects. Such networks may be used as a high-level controller that could predict forearm kinematics from voluntary movements of the upper arm. This methodology is suitable for restoring the upper limb functions of individuals with motor disabilities of the forearm, but not of the upper arm. The developed control paradigm is applicable to upper-limb orthotic systems employing functional electrical stimulation. The proposed approach is of great significance particularly for humans with spinal cord injuries in a free-living environment. The implication of a measurement system with dual-axis accelerometers, developed for this study, is further seen in the evaluation of movement during the course of rehabilitation. For this purpose, training-related changes in synergies apparent from movement kinematics during rehabilitation would characterize the extent and the course of recovery. As such, a simple system using this methodology is of particular importance for stroke patients. The results underlie the important issue of upper-limb coordination.

A Bayesian Approach for Forecasting Heat Load in a District Heating System

The growing population in cities increases the energy demand and affects the environment by increasing carbon emissions. Information and communications technology solutions which enable energy optimization are needed to address this growing energy demand in cities and to reduce carbon emissions. District heating systems optimize the energy production by reusing waste energy with combined heat and power plants. Forecasting the heat load demand in residential buildings assists in optimizing energy production and consumption in a district heating system. However, the presence of a large number of factors such as weather forecast, district heating operational parameters and user behavioural parameters, make heat load forecasting a challenging task. This thesis proposes a probabilistic machine learning model using a Naive Bayes classifier, to forecast the hourly heat load demand for three residential buildings in the city of Skellefteå, Sweden over a period of winter and spring seasons. The district heating data collected from the sensors equipped at the residential buildings in Skellefteå, is utilized to build the Bayesian network to forecast the heat load demand for horizons of 1, 2, 3, 6 and 24 hours. The proposed model is validated by using four cases to study the influence of various parameters on the heat load forecast by carrying out trace driven analysis in Weka and GeNIe. Results show that current heat load consumption and outdoor temperature forecast are the two parameters with most influence on the heat load forecast. The proposed model achieves average accuracies of 81.23 % and 76.74 % for a forecast horizon of 1 hour in the three buildings for winter and spring seasons respectively. The model also achieves an average accuracy of 77.97 % for three buildings across both seasons for the forecast horizon of 1 hour by utilizing only 10 % of the training data. The results indicate that even a simple model like Naive Bayes classifier can forecast the heat load demand by utilizing less training data.

Scalable Feature Selection Applications for Genome-Wide Association Studies of Complex Diseases

Personalized medicine will revolutionize our capabilities to combat disease. Working toward this goal, a fundamental task is the deciphering of geneticvariants that are predictive of complex diseases. Modern studies, in the formof genome-wide association studies (GWAS) have afforded researchers with the opportunity to reveal new genotype-phenotype relationships through the extensive scanning of genetic variants. These studies typically contain over half a million genetic features for thousands of individuals. Examining this with methods other than univariate statistics is a challenging task requiring advanced algorithms that are scalable to the genome-wide level. In the future, next-generation sequencing studies (NGS) will contain an even larger number of common and rare variants. Machine learning-based feature selection algorithms have been shown to have the ability to effectively create predictive models for various genotype-phenotype relationships. This work explores the problem of selecting genetic variant subsets that are the most predictive of complex disease phenotypes through various feature selection methodologies, including filter, wrapper and embedded algorithms. The examined machine learning algorithms were demonstrated to not only be effective at predicting the disease phenotypes, but also doing so efficiently through the use of computational shortcuts. While much of the work was able to be run on high-end desktops, some work was further extended so that it could be implemented on parallel computers helping to assure that they will also scale to the NGS data sets. Further, these studies analyzed the relationships between various feature selection methods and demonstrated the need for careful testing when selecting an algorithm. It was shown that there is no universally optimal algorithm for variant selection in GWAS, but rather methodologies need to be selected based on the desired outcome, such as the number of features to be included in the prediction model. It was also demonstrated that without proper model validation, for example using nested cross-validation, the models can result in overly-optimistic prediction accuracies and decreased generalization ability. It is through the implementation and application of machine learning methods that one can extract predictive genotype–phenotype relationships and biological insights from genetic data sets.

Radioaktiivisen päästön ennustaminen valmiustilanteessa

Ydinvoimaloissa käytetään toiminnallisia syvyyssuuntaisia puolustustasoja ydinturvallisuuden varmistamiseksi. Puolustuksen viidennessä ja viimeisessä tasossa pyritään lieventämään vakavan onnettomuuden ympäristövaikutuksia ja väestöön kohdistuvaa säteilyaltistusta. Suojelutoimien onnistumisen kannalta on tärkeää pystyä arvioimaan etukäteen radioaktiivisen päästön suuruus ja ajankohta mahdollisimman tarkasti. Tässä diplomityössä on esitelty radioaktiivisen päästön suuruuteen ja ajankohtaan vaikuttavat ilmiöt sekä niihin liittyvät merkittävät epävarmuudet. Ydinvoimalaitosten turvallisuusjärjestelmien osalta tarkastelun kohteena ovat suomalaiset käynnissä olevat reaktorit Olkiluoto 1 & 2 sekä Loviisa 1 & 2. Kaikissa Suomen laitoksissa on käytössä vakavan onnettomuuden hallintaan soveltuvia järjestelmiä ja toimintoja. Työssä etsittiin tietoa eri maiden radioaktiivisen päästön ennustamiseen käytettävistä ohjelmista. Eri mailla on eri toimintaperiaatteilla ja laajuuksilla toimivia ohjelmia. Osassa työkaluja käytetään ennalta laskettuja tuloksia ja osassa onnettomuustilanteet lasketaan onnettomuuden aikana. Lisäksi lähivuosina Euroopassa on tavoitteena kehittää yhteistyömaille yhteisiä valmiuskäyttöön soveltuvia ohjelmia. Työssä kehitettiin uusi valmiustyökalu Säteilyturvakeskuksen käyttöön Microsoft Excelin VBAohjelmoinnin avulla. Valmiustyökalu hyödyntää etukäteen laskettujen todennäköisyyspohjaisten analyysien onnettomuussekvenssejä. Tällöin valmiustilanteessa laitoksen tilanteen kehittymistä on mahdollista arvioida suojarakennuksen toimintakyvyn perusteella. Valmiustyökalu pyrittiin kehittämään mahdollisimman helppokäyttöiseksi ja helposti päivitettäväksi.

Generative Part-Based Gabor Object Detector

Object detection is a fundamental task of computer vision that is utilized as a core part in a number of industrial and scientific applications, for example, in robotics, where objects need to be correctly detected and localized prior to being grasped and manipulated. Existing object detectors vary in (i) the amount of supervision they need for training, (ii) the type of a learning method adopted (generative or discriminative) and (iii) the amount of spatial information used in the object model (model-free, using no spatial information in the object model, or model-based, with the explicit spatial model of an object). Although some existing methods report good performance in the detection of certain objects, the results tend to be application specific and no universal method has been found that clearly outperforms all others in all areas. This work proposes a novel generative part-based object detector. The generative learning procedure of the developed method allows learning from positive examples only. The detector is based on finding semantically meaningful parts of the object (i.e. a part detector) that can provide additional information to object location, for example, pose. The object class model, i.e. the appearance of the object parts and their spatial variance, constellation, is explicitly modelled in a fully probabilistic manner. The appearance is based on bio-inspired complex-valued Gabor features that are transformed to part probabilities by an unsupervised Gaussian Mixture Model (GMM). The proposed novel randomized GMM enables learning from only a few training examples. The probabilistic spatial model of the part configurations is constructed with a mixture of 2D Gaussians. The appearance of the parts of the object is learned in an object canonical space that removes geometric variations from the part appearance model. Robustness to pose variations is achieved by object pose quantization, which is more efficient than previously used scale and orientation shifts in the Gabor feature space. Performance of the resulting generative object detector is characterized by high recall with low precision, i.e. the generative detector produces large number of false positive detections. Thus a discriminative classifier is used to prune false positive candidate detections produced by the generative detector improving its precision while keeping high recall. Using only a small number of positive examples, the developed object detector performs comparably to state-of-the-art discriminative methods.

Formal development of resilient distributed systems

Resilience is the property of a system to remain trustworthy despite changes. Changes of a different nature, whether due to failures of system components or varying operational conditions, significantly increase the complexity of system development. Therefore, advanced development technologies are required to build robust and flexible system architectures capable of adapting to such changes. Moreover, powerful quantitative techniques are needed to assess the impact of these changes on various system characteristics. Architectural flexibility is achieved by embedding into the system design the mechanisms for identifying changes and reacting on them. Hence a resilient system should have both advanced monitoring and error detection capabilities to recognise changes as well as sophisticated reconfiguration mechanisms to adapt to them. The aim of such reconfiguration is to ensure that the system stays operational, i.e., remains capable of achieving its goals. Design, verification and assessment of the system reconfiguration mechanisms is a challenging and error prone engineering task. In this thesis, we propose and validate a formal framework for development and assessment of resilient systems. Such a framework provides us with the means to specify and verify complex component interactions, model their cooperative behaviour in achieving system goals, and analyse the chosen reconfiguration strategies. Due to the variety of properties to be analysed, such a framework should have an integrated nature. To ensure the system functional correctness, it should rely on formal modelling and verification, while, to assess the impact of changes on such properties as performance and reliability, it should be combined with quantitative analysis. To ensure scalability of the proposed framework, we choose Event-B as the basis for reasoning about functional correctness. Event-B is a statebased formal approach that promotes the correct-by-construction development paradigm and formal verification by theorem proving. Event-B has a mature industrial-strength tool support { the Rodin platform. Proof-based verification as well as the reliance on abstraction and decomposition adopted in Event-B provides the designers with a powerful support for the development of complex systems. Moreover, the top-down system development by refinement allows the developers to explicitly express and verify critical system-level properties. Besides ensuring functional correctness, to achieve resilience we also need to analyse a number of non-functional characteristics, such as reliability and performance. Therefore, in this thesis we also demonstrate how formal development in Event-B can be combined with quantitative analysis. Namely, we experiment with integration of such techniques as probabilistic model checking in PRISM and discrete-event simulation in SimPy with formal development in Event-B. Such an integration allows us to assess how changes and di erent recon guration strategies a ect the overall system resilience. The approach proposed in this thesis is validated by a number of case studies from such areas as robotics, space, healthcare and cloud domain.

Saline distribution during multicomponent salting in pre-cooked quail eggs

The relationship of NaCl with problems of arterial hypertension has led to a reduction in the levels of this salt in food production. KCl has been used as a partial substitute for NaCl since it cannot be completely substituted without affecting the acceptability of the end product. In this study, the diffusion that occurs during quail egg salting in static and stirred brine was simulated. The mathematical model used was based on a generalization of the Fick's 2nd law, and the COMSOL Multiphysics software was used to simulate the diffusion in the NaCl-KCl-water system. The deviations in the simulated data and experimental data were 2.50% for NaCl and 6.98% for KCl in static brine, while in the stirred brine they were 3.48% for NaCl and 4.72% for KCl. The simulation results presented good agreement with the experimental values and validated the predictive capacity of the model.

Analysis of applicability of Peleg model to the cooking-infusion of mackerel (Scomber japonicus) slices

Mass transfer kinetics in osmotic dehydration is usually modeled by Fick's law, empirical models and probabilistic models. The aim of this study was to determine the applicability of Peleg model to investigate the mass transfer during osmotic dehydration of mackerel (Scomber japonicus) slices at different temperatures. Osmotic dehydration was performed on mackerel slices by cooking-infusion in solutions with glycerol and salt (a w = 0.64) at different temperatures: 50, 70, and 90 ºC. Peleg rate constant (K1) (h(g/gdm)-1) varied with temperature variation from 0.761 to 0.396 for water loss, from 5.260 to 2.947 for salt gain, and from 0.854 to 0.566 for glycerol intake. In all cases, it followed the Arrhenius relationship (R²>0.86). The Ea (kJ / mol) values obtained were 16.14; 14.21, and 10.12 for water, salt, and glycerol, respectively. The statistical parameters that qualify the goodness of fit (R²>0.91 and RMSE<0.086) indicate promising applicability of Peleg model.

Artificial neural networks (ANN): prediction of sensory measurements from instrumental data

The objective of this study was to predict by means of Artificial Neural Network (ANN), multilayer perceptrons, the texture attributes of light cheesecurds perceived by trained judges based on instrumental texture measurements. Inputs to the network were the instrumental texture measurements of light cheesecurd (imitative and fundamental parameters). Output variables were the sensory attributes consistency and spreadability. Nine light cheesecurd formulations composed of different combinations of fat and water were evaluated. The measurements obtained by the instrumental and sensory analyses of these formulations constituted the data set used for training and validation of the network. Network training was performed using a back-propagation algorithm. The network architecture selected was composed of 8-3-9-2 neurons in its layers, which quickly and accurately predicted the sensory texture attributes studied, showing a high correlation between the predicted and experimental values for the validation data set and excellent generalization ability, with a validation RMSE of 0.0506.