Probabilistic Models for Scalable Knowledge Graph Construction

In the past decade, systems that extract information from millions of Internet documents have become commonplace. Knowledge graphs -- structured knowledge bases that describe entities, their attributes and the relationships between them -- are a powerful tool for understanding and organizing this vast amount of information. However, a significant obstacle to knowledge graph construction is the unreliability of the extracted information, due to noise and ambiguity in the underlying data or errors made by the extraction system and the complexity of reasoning about the dependencies between these noisy extractions. My dissertation addresses these challenges by exploiting the interdependencies between facts to improve the quality of the knowledge graph in a scalable framework. I introduce a new approach called knowledge graph identification (KGI), which resolves the entities, attributes and relationships in the knowledge graph by incorporating uncertain extractions from multiple sources, entity co-references, and ontological constraints. I define a probability distribution over possible knowledge graphs and infer the most probable knowledge graph using a combination of probabilistic and logical reasoning. Such probabilistic models are frequently dismissed due to scalability concerns, but my implementation of KGI maintains tractable performance on large problems through the use of hinge-loss Markov random fields, which have a convex inference objective. This allows the inference of large knowledge graphs using 4M facts and 20M ground constraints in 2 hours. To further scale the solution, I develop a distributed approach to the KGI problem which runs in parallel across multiple machines, reducing inference time by 90%. Finally, I extend my model to the streaming setting, where a knowledge graph is continuously updated by incorporating newly extracted facts. I devise a general approach for approximately updating inference in convex probabilistic models, and quantify the approximation error by defining and bounding inference regret for online models. Together, my work retains the attractive features of probabilistic models while providing the scalability necessary for large-scale knowledge graph construction. These models have been applied on a number of real-world knowledge graph projects, including the NELL project at Carnegie Mellon and the Google Knowledge Graph.

Impact of a community-based exercise programme on physical fitness in middle-aged and older patients with type 2 diabetes

Objectives: Physical fitness is related to all-cause mortality, quality of life and risk of falls in patients with type 2 diabetes. This study aimed to analyse the impact of a long-term community-based combined exercise program (aerobic + resistance + agility/balance + flexibility) developed with minimum and low-cost material resources on physical fitness in middle-aged and older patients with type 2 diabetes. Methods: This was a non-experimental pre-post evaluation study. Participants (N = 43; 62.92 ± 5.92 years old) were engaged in a community-based supervised exercise programme (consisting of combined aerobic, resistance, agility/balance and flexibility exercises; three sessions per week; 70 min per session) of 9 months' duration. Aerobic fitness (6-Minute Walk Test), muscle strength (30-Second Chair Stand Test), agility/balance (Timed Up and Go Test) and flexibility (Chair Sit and Reach Test) were assessed before (baseline) and after the exercise intervention. Results: Significant improvements in the performance of the 6-Minute Walk Test (Δ = 8.20%, p < 0.001), 30-Second Chair Stand Test (Δ = 28.84%, p < 0.001), Timed Up and Go Test (Δ = 14.31%, p < 0.001), and Chair Sit and Reach Test (Δ = 102.90%, p < 0.001) were identified between baseline and end-exercise intervention time points. Conclusions: A long-term community-based combined exercise programme, developed with low-cost exercise strategies, produced significant benefits in physical fitness in middle-aged and older patients with type 2 diabetes. This supervised group exercise programme significantly improved aerobic fitness, muscle strength, agility/balance and flexibility, assessed with field tests in community settings.

Leveraging Multiple Features for Image Retrieval and Matching

The goal of image retrieval and matching is to find and locate object instances in images from a large-scale image database. While visual features are abundant, how to combine them to improve performance by individual features remains a challenging task. In this work, we focus on leveraging multiple features for accurate and efficient image retrieval and matching. We first propose two graph-based approaches to rerank initially retrieved images for generic image retrieval. In the graph, vertices are images while edges are similarities between image pairs. Our first approach employs a mixture Markov model based on a random walk model on multiple graphs to fuse graphs. We introduce a probabilistic model to compute the importance of each feature for graph fusion under a naive Bayesian formulation, which requires statistics of similarities from a manually labeled dataset containing irrelevant images. To reduce human labeling, we further propose a fully unsupervised reranking algorithm based on a submodular objective function that can be efficiently optimized by greedy algorithm. By maximizing an information gain term over the graph, our submodular function favors a subset of database images that are similar to query images and resemble each other. The function also exploits the rank relationships of images from multiple ranked lists obtained by different features. We then study a more well-defined application, person re-identification, where the database contains labeled images of human bodies captured by multiple cameras. Re-identifications from multiple cameras are regarded as related tasks to exploit shared information. We apply a novel multi-task learning algorithm using both low level features and attributes. A low rank attribute embedding is joint learned within the multi-task learning formulation to embed original binary attributes to a continuous attribute space, where incorrect and incomplete attributes are rectified and recovered. To locate objects in images, we design an object detector based on object proposals and deep convolutional neural networks (CNN) in view of the emergence of deep networks. We improve a Fast RCNN framework and investigate two new strategies to detect objects accurately and efficiently: scale-dependent pooling (SDP) and cascaded rejection classifiers (CRC). The SDP improves detection accuracy by exploiting appropriate convolutional features depending on the scale of input object proposals. The CRC effectively utilizes convolutional features and greatly eliminates negative proposals in a cascaded manner, while maintaining a high recall for true objects. The two strategies together improve the detection accuracy and reduce the computational cost.

Run-time profiling of Algol 68-R programs using DIDYMUS and SCAMP

Information concerning the run-time behaviour of programs ("program profiling") can be of the greatest assistance in improving program efficiency. Two software devices have been developed for use on ICL 1900 Series machines to provide such information. DIDYMUS is probabilistic in approach and uses multi- tasking facilities to sample the instruction addresses used by a program at run time. It will work regardless of the source language of the program and matches the detected addresses against a loader map to produce a histogram. SCAMP is restricted to profiling Algol 68-R programs, but provides deterministic information concerning those language constructs that are monitored. Procedure calls to appropriate counting routines are inserted into the source text in a pre-pass prior to compilation. The profile information is printed out at the end of the program run. It has been found that these two approaches complement each other very effectively.

On L2 grammar and processing: the case of oblique relative clauses and the Null-Prep phenomenon

This dissertation investigates the acquisition of oblique relative clauses in L2 Spanish by English and Moroccan Arabic speakers in order to understand the role of previous linguistic knowledge and its interaction with Universal Grammar on the one hand, and the relationship between grammatical knowledge and its use in real-time, on the other hand. Three types of tasks were employed: an oral production task, an on-line self-paced grammaticality judgment task, and an on-line self-paced reading comprehension task. Results indicated that the acquisition of oblique relative clauses in Spanish is a problematic area for second language learners of intermediate proficiency in the language, regardless of their native language. In particular, this study has showed that, even when the learners’ native language shares the main properties of the L2, i.e., fronting of the obligatory preposition (Pied-Piping), there is still room for divergence, especially in production and timed grammatical intuitions. On the other hand, reaction time data have shown that L2 learners can and do converge at the level of sentence processing, showing exactly the same real-time effects for oblique relative clauses that native speakers had. Processing results demonstrated that native and non-native speakers alike are able to apply universal processing principles such as the Minimal Chain Principle (De Vincenzi, 1991) even when the L2 learners still have incomplete grammatical representations, a result that contradicts some of the predictions of the Shallow Structure Hypothesis (Clahsen & Felser, 2006). Results further suggest that the L2 processing and comprehension domains may be able to access some type of information that it is not yet available to other grammatical modules, probably because transfer of certain L1 properties occurs asymmetrically across linguistic domains. In addition, this study also explored the Null-Prep phenomenon in L2 Spanish, and proposed that Null-Prep is an interlanguage stage, fully available and accounted within UG, which intermediate L2 as well as first language learners go through in the development of pied-piping oblique relative clauses. It is hypothesized that this intermediate stage is the result of optionality of the obligatory preposition in the derivation, when it is not crucial for the meaning of the sentence, and when the DP is going to be in an A-bar position, so it can get default case. This optionality can be predicted by the Bottleneck Hypothesis (Slabakova, 2009c) if we consider that these prepositions are some sort of functional morphology. This study contributes to the field of SLA and L2 processing in various ways. First, it demonstrates that the grammatical representations may be dissociated from grammatical processing in the sense that L2 learners, unlike native speakers, can present unexpected asymmetries such as a convergent processing but divergent grammatical intuitions or production. This conclusion is only possible under the assumption of a modular language system. Finally, it contributes to the general debate of generative SLA since in argues for a fully UG-constrained interlanguage grammar.

Increased hospital-based physical rehabilitation and information provision after intensive care unit discharge: the RECOVER randomized clinical trial.

Importance: critical illness results in disability and reduced health-related quality of life (HRQOL), but the optimum timing and components of rehabilitation are uncertain. Objective: to evaluate the effect of increasing physical and nutritional rehabilitation plus information delivered during the post–intensive care unit (ICU) acute hospital stay by dedicated rehabilitation assistants on subsequent mobility, HRQOL, and prevalent disabilities. Design, Setting, and Participants: a parallel group, randomized clinical trial with blinded outcome assessment at 2 hospitals in Edinburgh, Scotland, of 240 patients discharged from the ICU between December 1, 2010, and January 31, 2013, who required at least 48 hours of mechanical ventilation. Analysis for the primary outcome and other 3-month outcomes was performed between June and August 2013; for the 6- and 12-month outcomes and the health economic evaluation, between March and April 2014. Interventions: during the post-ICU hospital stay, both groups received physiotherapy and dietetic, occupational, and speech/language therapy, but patients in the intervention group received rehabilitation that typically increased the frequency of mobility and exercise therapies 2- to 3-fold, increased dietetic assessment and treatment, used individualized goal setting, and provided greater illness-specific information. Intervention group therapy was coordinated and delivered by a dedicated rehabilitation practitioner. Main Outcomes and Measures: the Rivermead Mobility Index (RMI) (range 0-15) at 3 months; higher scores indicate greater mobility. Secondary outcomes included HRQOL, psychological outcomes, self-reported symptoms, patient experience, and cost-effectiveness during a 12-month follow-up (completed in February 2014). Results: median RMI at randomization was 3 (interquartile range [IQR], 1-6) and at 3 months was 13 (IQR, 10-14) for the intervention and usual care groups (mean difference, −0.2 [95% CI, −1.3 to 0.9; P = .71]). The HRQOL scores were unchanged by the intervention (mean difference in the Physical Component Summary score, −0.1 [95% CI, −3.3 to 3.1; P = .96]; and in the Mental Component Summary score, 0.2 [95% CI, −3.4 to 3.8; P = .91]). No differences were found for self-reported symptoms of fatigue, pain, appetite, joint stiffness, or breathlessness. Levels of anxiety, depression, and posttraumatic stress were similar, as were hand grip strength and the timed Up & Go test. No differences were found at the 6- or 12-month follow-up for any outcome measures. However, patients in the intervention group reported greater satisfaction with physiotherapy, nutritional support, coordination of care, and information provision. Conclusions and Relevance: post-ICU hospital-based rehabilitation, including increased physical and nutritional therapy plus information provision, did not improve physical recovery or HRQOL, but improved patient satisfaction with many aspects of recovery.

Probabilistic design optimization of horizontal axis wind turbine rotors

Considerable interest in renewable energy has increased in recent years due to the concerns raised over the environmental impact of conventional energy sources and their price volatility. In particular, wind power has enjoyed a dramatic global growth in installed capacity over the past few decades. Nowadays, the advancement of wind turbine industry represents a challenge for several engineering areas, including materials science, computer science, aerodynamics, analytical design and analysis methods, testing and monitoring, and power electronics. In particular, the technological improvement of wind turbines is currently tied to the use of advanced design methodologies, allowing the designers to develop new and more efficient design concepts. Integrating mathematical optimization techniques into the multidisciplinary design of wind turbines constitutes a promising way to enhance the profitability of these devices. In the literature, wind turbine design optimization is typically performed deterministically. Deterministic optimizations do not consider any degree of randomness affecting the inputs of the system under consideration, and result, therefore, in an unique set of outputs. However, given the stochastic nature of the wind and the uncertainties associated, for instance, with wind turbine operating conditions or geometric tolerances, deterministically optimized designs may be inefficient. Therefore, one of the ways to further improve the design of modern wind turbines is to take into account the aforementioned sources of uncertainty in the optimization process, achieving robust configurations with minimal performance sensitivity to factors causing variability. The research work presented in this thesis deals with the development of a novel integrated multidisciplinary design framework for the robust aeroservoelastic design optimization of multi-megawatt horizontal axis wind turbine (HAWT) rotors, accounting for the stochastic variability related to the input variables. The design system is based on a multidisciplinary analysis module integrating several simulations tools needed to characterize the aeroservoelastic behavior of wind turbines, and determine their economical performance by means of the levelized cost of energy (LCOE). The reported design framework is portable and modular in that any of its analysis modules can be replaced with counterparts of user-selected fidelity. The presented technology is applied to the design of a 5-MW HAWT rotor to be used at sites of wind power density class from 3 to 7, where the mean wind speed at 50 m above the ground ranges from 6.4 to 11.9 m/s. Assuming the mean wind speed to vary stochastically in such range, the rotor design is optimized by minimizing the mean and standard deviation of the LCOE. Airfoil shapes, spanwise distributions of blade chord and twist, internal structural layup and rotor speed are optimized concurrently, subject to an extensive set of structural and aeroelastic constraints. The effectiveness of the multidisciplinary and robust design framework is demonstrated by showing that the probabilistically designed turbine achieves more favorable probabilistic performance than those of the initial baseline turbine and a turbine designed deterministically.

Spectral Frame Analysis and Learning through Graph Structure

This dissertation investigates the connection between spectral analysis and frame theory. When considering the spectral properties of a frame, we present a few novel results relating to the spectral decomposition. We first show that scalable frames have the property that the inner product of the scaling coefficients and the eigenvectors must equal the inverse eigenvalues. From this, we prove a similar result when an approximate scaling is obtained. We then focus on the optimization problems inherent to the scalable frames by first showing that there is an equivalence between scaling a frame and optimization problems with a non-restrictive objective function. Various objective functions are considered, and an analysis of the solution type is presented. For linear objectives, we can encourage sparse scalings, and with barrier objective functions, we force dense solutions. We further consider frames in high dimensions, and derive various solution techniques. From here, we restrict ourselves to various frame classes, to add more specificity to the results. Using frames generated from distributions allows for the placement of probabilistic bounds on scalability. For discrete distributions (Bernoulli and Rademacher), we bound the probability of encountering an ONB, and for continuous symmetric distributions (Uniform and Gaussian), we show that symmetry is retained in the transformed domain. We also prove several hyperplane-separation results. With the theory developed, we discuss graph applications of the scalability framework. We make a connection with graph conditioning, and show the in-feasibility of the problem in the general case. After a modification, we show that any complete graph can be conditioned. We then present a modification of standard PCA (robust PCA) developed by Cand\`es, and give some background into Electron Energy-Loss Spectroscopy (EELS). We design a novel scheme for the processing of EELS through robust PCA and least-squares regression, and test this scheme on biological samples. Finally, we take the idea of robust PCA and apply the technique of kernel PCA to perform robust manifold learning. We derive the problem and present an algorithm for its solution. There is also discussion of the differences with RPCA that make theoretical guarantees difficult.

Combined exposure to multiple mycotoxins in food: are we ready for risk assessment and risk management?

It is nowadays recognized that the risk of human co-exposure to multiple mycotoxins is real. In the last years, a number of studies have approached the issue of co-exposure and the best way to develop a more precise and realistic assessment. Likewise, the growing concern about the combined effects of mycotoxins and their potential impact on human health has been reflected by the increasing number of toxicological studies on the combined toxicity of these compounds. Nevertheless, risk assessment of these toxins, still follows the conventional paradigm of single exposure and single effects, incorporating only the possibility of additivity but not taking into account the complex dynamics associated to interactions between different mycotoxins or between mycotoxins and other food contaminants. Considering that risk assessment is intimately related to the establishment of regulatory guidelines, once the risk assessment is completed, an effort to reduce or manage the risk should be followed to protect public health. Risk assessment of combined human exposure to multiple mycotoxins thus poses several challenges to scientists, risk assessors and risk managers and opens new avenues for research. This presentation aims to give an overview of the different challenges posed by the likelihood of human co-exposure to mycotoxins and the possibility of interactive effects occurring after absorption, towards knowledge generation to support a more accurate human risk assessment and risk management. For this purpose, a physiologically-based framework that includes knowledge on the bioaccessibility, toxicokinetics and toxicodynamics of multiple toxins is proposed. Regarding exposure assessment, the need of harmonized food consumption data, availability of multianalyte methods for mycotoxin quantification, management of left-censored data and use of probabilistic models will be highlight, in order to develop a more precise and realistic exposure assessment. On the other hand, the application of predictive mathematical models to estimate mycotoxins’ combined effects from in vitro toxicity studies will be also discussed. Results from a recent Portuguese project aimed at exploring the toxic effects of mixtures of mycotoxins in infant foods and their potential health impact will be presented as a case study, illustrating the different aspects of risk assessment highlighted in this presentation. Further studies on hazard and exposure assessment of multiple mycotoxins, using harmonized approaches and methodologies, will be crucial towards an improvement in data quality and contributing to holistic risk assessment and risk management strategies for multiple mycotoxins in foodstuffs.

Bayesian Optimisation Algorithm for Nurse Scheduling, Scalable Optimization via Probabilistic Modeling

Our research has shown that schedules can be built mimicking a human scheduler by using a set of rules that involve domain knowledge. This chapter presents a Bayesian Optimization Algorithm (BOA) for the nurse scheduling problem that chooses such suitable scheduling rules from a set for each nurse’s assignment. Based on the idea of using probabilistic models, the BOA builds a Bayesian network for the set of promising solutions and samples these networks to generate new candidate solutions. Computational results from 52 real data instances demonstrate the success of this approach. It is also suggested that the learning mechanism in the proposed algorithm may be suitable for other scheduling problems.

Probabilistic verification of satellite systems for mission critical applications

In this thesis, we present a quantitative approach using probabilistic verification techniques for the analysis of reliability, availability, maintainability, and safety (RAMS) properties of satellite systems. The subject of our research is satellites used in mission critical industrial applications. A strong case for using probabilistic model checking to support RAMS analysis of satellite systems is made by our verification results. This study is intended to build a foundation to help reliability engineers with a basic background in model checking to apply probabilistic model checking to small satellite systems. We make two major contributions. One of these is the approach of RAMS analysis to satellite systems. In the past, RAMS analysis has been extensively applied to the field of electrical and electronics engineering. It allows system designers and reliability engineers to predict the likelihood of failures from the indication of historical or current operational data. There is a high potential for the application of RAMS analysis in the field of space science and engineering. However, there is a lack of standardisation and suitable procedures for the correct study of RAMS characteristics for satellite systems. This thesis considers the promising application of RAMS analysis to the case of satellite design, use, and maintenance, focusing on its system segments. Data collection and verification procedures are discussed, and a number of considerations are also presented on how to predict the probability of failure. Our second contribution is leveraging the power of probabilistic model checking to analyse satellite systems. We present techniques for analysing satellite systems that differ from the more common quantitative approaches based on traditional simulation and testing. These techniques have not been applied in this context before. We present the use of probabilistic techniques via a suite of detailed examples, together with their analysis. Our presentation is done in an incremental manner: in terms of complexity of application domains and system models, and a detailed PRISM model of each scenario. We also provide results from practical work together with a discussion about future improvements.

A case study of the English language teachers’ attitudes toward use of information and communication technology (ICT) in Finland

This qualitative study was aimed at investigating foreign language teachers’ attitudes toward use of information and communication technology (ICT) in their instruction. The insight was gained through the reported experience of ICT implementation by teachers, in what way and for which purpose they refer to use of technology, what kind of support and training they are provided with, and what beliefs they express about the influence of ICT implementation. This case study took place in one of the training schools in Finland. Five teachers participated in semi-structured interviews through a face-to-face approach. The findings demonstrated positive attitudes of teachers toward integration of ICT. The teachers shared their opinions about positive influence that ICT implementation has on both teaching and learning processes. However, they also pointed out the negative sides of ICT use: distraction of the students from usage of technology and technical problems causing frustration to the teachers. In addition, the responses revealed that the teachers are provided with adequate training aimed at enhancing their qualification which is provided with well-timed technology support and colleagues’ collaboration facilitating an efficient and smooth pace of the teaching process. According to the teachers’ opinions ICT integration in education appeared to have changed the role of the teacher. Due to different alterations in the field of ICT development teachers are required to upgrade their skills. The paper concludes with the limitations of the study and the recommendations for conducting further research.

Learning Binary Code Representations for Effective and Efficient Image Retrieval

The size of online image datasets is constantly increasing. Considering an image dataset with millions of images, image retrieval becomes a seemingly intractable problem for exhaustive similarity search algorithms. Hashing methods, which encodes high-dimensional descriptors into compact binary strings, have become very popular because of their high efficiency in search and storage capacity. In the first part, we propose a multimodal retrieval method based on latent feature models. The procedure consists of a nonparametric Bayesian framework for learning underlying semantically meaningful abstract features in a multimodal dataset, a probabilistic retrieval model that allows cross-modal queries and an extension model for relevance feedback. In the second part, we focus on supervised hashing with kernels. We describe a flexible hashing procedure that treats binary codes and pairwise semantic similarity as latent and observed variables, respectively, in a probabilistic model based on Gaussian processes for binary classification. We present a scalable inference algorithm with the sparse pseudo-input Gaussian process (SPGP) model and distributed computing. In the last part, we define an incremental hashing strategy for dynamic databases where new images are added to the databases frequently. The method is based on a two-stage classification framework using binary and multi-class SVMs. The proposed method also enforces balance in binary codes by an imbalance penalty to obtain higher quality binary codes. We learn hash functions by an efficient algorithm where the NP-hard problem of finding optimal binary codes is solved via cyclic coordinate descent and SVMs are trained in a parallelized incremental manner. For modifications like adding images from an unseen class, we propose an incremental procedure for effective and efficient updates to the previous hash functions. Experiments on three large-scale image datasets demonstrate that the incremental strategy is capable of efficiently updating hash functions to the same retrieval performance as hashing from scratch.

A case study of the English language teachers’ attitudes toward use of information and communication technology (ICT) in Finland

This qualitative study was aimed at investigating foreign language teachers’ attitudes toward use of information and communication technology (ICT) in their instruction. The insight was gained through the reported experience of ICT implementation by teachers, in what way and for which purpose they refer to use of technology, what kind of support and training they are provided with, and what beliefs they express about the influence of ICT implementation. This case study took place in one of the training schools in Finland. Five teachers participated in semi-structured interviews through a face-to-face approach. The findings demonstrated positive attitudes of teachers toward integration of ICT. The teachers shared their opinions about positive influence that ICT implementation has on both teaching and learning processes. However, they also pointed out the negative sides of ICT use: distraction of the students from usage of technology and technical problems causing frustration to the teachers. In addition, the responses revealed that the teachers are provided with adequate training aimed at enhancing their qualification which is provided with well-timed technology support and colleagues’ collaboration facilitating an efficient and smooth pace of the teaching process. According to the teachers’ opinions ICT integration in education appeared to have changed the role of the teacher. Due to different alterations in the field of ICT development teachers are required to upgrade their skills. The paper concludes with the limitations of the study and the recommendations for conducting further research.

Falls and other geriatric syndromes in Blantyre, Malawi: a community survey of older adults

Background The prevalence of geriatric syndromes (falls, immobility, intellectual or memory impairment, and incontinence) is unknown in many resource-poor countries. With an aging population such knowledge is essential to develop national policies on the health and social needs of older people. The aim of this study was to provide a preliminary survey to explore the prevalence of falls and other geriatric syndromes and their association with known risk factors in people aged > 60 years in urban Blantyre, Malawi. Methods This was a cross-sectional, community survey of adults aged > 60 years. Subjects were recruited at home or in the waiting areas of chronic care clinics. They were interviewed to complete a questionnaire on ageassociated syndromes and comorbid problems. The Abbreviated Mental Test (AMT) and Timed Up and Go (TUG) tests were carried out. Results Ninety-eight subjects were studied; 41% reported falling in the past 12 months, 33% of whom (13% of all subjects) were recurrent fallers. Twenty-five percent reported urine incontinence, 66% self-reported memory difficulties, and 11% had an AMT score < 7. A history of falling was significantly associated with urine incontinence (p=0.01), selfreported memory problems (p=0.004) and AMT score < 7 (p=0.02). Conclusions Geriatric syndromes, including falls, appear to be prevalent in older people in Blantyre, Malawi. Falling is associated with cognitive impairment and urinary incontinence. There is an urgent need for more understanding of geriatric problems in this setting to develop national policies on health and social needs of older people. It is likely that many of the contributory factors to falls would be amenable to multifactorial interventions similar to those found to be effective in developed countries.