Simple approximate MAP inference for Dirichlet processes mixtures

The Dirichlet process mixture model (DPMM) is a ubiquitous, flexible Bayesian nonparametric statistical model. However, full probabilistic inference in this model is analytically intractable, so that computationally intensive techniques such as Gibbs sampling are required. As a result, DPMM-based methods, which have considerable potential, are restricted to applications in which computational resources and time for inference is plentiful. For example, they would not be practical for digital signal processing on embedded hardware, where computational resources are at a serious premium. Here, we develop a simplified yet statistically rigorous approximate maximum a-posteriori (MAP) inference algorithm for DPMMs. This algorithm is as simple as DP-means clustering, solves the MAP problem as well as Gibbs sampling, while requiring only a fraction of the computational effort. (For freely available code that implements the MAP-DP algorithm for Gaussian mixtures see http://www.maxlittle.net/.) Unlike related small variance asymptotics (SVA), our method is non-degenerate and so inherits the “rich get richer” property of the Dirichlet process. It also retains a non-degenerate closed-form likelihood which enables out-of-sample calculations and the use of standard tools such as cross-validation. We illustrate the benefits of our algorithm on a range of examples and contrast it to variational, SVA and sampling approaches from both a computational complexity perspective as well as in terms of clustering performance. We demonstrate the wide applicabiity of our approach by presenting an approximate MAP inference method for the infinite hidden Markov model whose performance contrasts favorably with a recently proposed hybrid SVA approach. Similarly, we show how our algorithm can applied to a semiparametric mixed-effects regression model where the random effects distribution is modelled using an infinite mixture model, as used in longitudinal progression modelling in population health science. Finally, we propose directions for future research on approximate MAP inference in Bayesian nonparametrics.

Average-case analysis of power consumption in embedded systems

Power efficiency is one of the most important constraints in the design of embedded systems since such systems are generally driven by batteries with limited energy budget or restricted power supply. In every embedded system, there are one or more processor cores to run the software and interact with the other hardware components of the system. The power consumption of the processor core(s) has an important impact on the total power dissipated in the system. Hence, the processor power optimization is crucial in satisfying the power consumption constraints, and developing low-power embedded systems. A key aspect of research in processor power optimization and management is “power estimation”. Having a fast and accurate method for processor power estimation at design time helps the designer to explore a large space of design possibilities, to make the optimal choices for developing a power efficient processor. Likewise, understanding the processor power dissipation behaviour of a specific software/application is the key for choosing appropriate algorithms in order to write power efficient software. Simulation-based methods for measuring the processor power achieve very high accuracy, but are available only late in the design process, and are often quite slow. Therefore, the need has arisen for faster, higher-level power prediction methods that allow the system designer to explore many alternatives for developing powerefficient hardware and software. The aim of this thesis is to present fast and high-level power models for the prediction of processor power consumption. Power predictability in this work is achieved in two ways: first, using a design method to develop power predictable circuits; second, analysing the power of the functions in the code which repeat during execution, then building the power model based on average number of repetitions. In the first case, a design method called Asynchronous Charge Sharing Logic (ACSL) is used to implement the Arithmetic Logic Unit (ALU) for the 8051 microcontroller. The ACSL circuits are power predictable due to the independency of their power consumption to the input data. Based on this property, a fast prediction method is presented to estimate the power of ALU by analysing the software program, and extracting the number of ALU-related instructions. This method achieves less than 1% error in power estimation and more than 100 times speedup in comparison to conventional simulation-based methods. In the second case, an average-case processor energy model is developed for the Insertion sort algorithm based on the number of comparisons that take place in the execution of the algorithm. The average number of comparisons is calculated using a high level methodology called MOdular Quantitative Analysis (MOQA). The parameters of the energy model are measured for the LEON3 processor core, but the model is general and can be used for any processor. The model has been validated through the power measurement experiments, and offers high accuracy and orders of magnitude speedup over the simulation-based method.

Hilbert Space Valued Signal Plus Noise Models: Analysis of Structural Breaks under High Dimensionality and Temporal Dependence

This thesis is concerned with change point analysis for time series, i.e. with detection of structural breaks in time-ordered, random data. This long-standing research field regained popularity over the last few years and is still undergoing, as statistical analysis in general, a transformation to high-dimensional problems. We focus on the fundamental »change in the mean« problem and provide extensions of the classical non-parametric Darling-Erdős-type cumulative sum (CUSUM) testing and estimation theory within highdimensional Hilbert space settings. In the first part we contribute to (long run) principal component based testing methods for Hilbert space valued time series under a rather broad (abrupt, epidemic, gradual, multiple) change setting and under dependence. For the dependence structure we consider either traditional m-dependence assumptions or more recently developed m-approximability conditions which cover, e.g., MA, AR and ARCH models. We derive Gumbel and Brownian bridge type approximations of the distribution of the test statistic under the null hypothesis of no change and consistency conditions under the alternative. A new formulation of the test statistic using projections on subspaces allows us to simplify the standard proof techniques and to weaken common assumptions on the covariance structure. Furthermore, we propose to adjust the principal components by an implicit estimation of a (possible) change direction. This approach adds flexibility to projection based methods, weakens typical technical conditions and provides better consistency properties under the alternative. In the second part we contribute to estimation methods for common changes in the means of panels of Hilbert space valued time series. We analyze weighted CUSUM estimates within a recently proposed »high-dimensional low sample size (HDLSS)« framework, where the sample size is fixed but the number of panels increases. We derive sharp conditions on »pointwise asymptotic accuracy« or »uniform asymptotic accuracy« of those estimates in terms of the weighting function. Particularly, we prove that a covariance-based correction of Darling-Erdős-type CUSUM estimates is required to guarantee uniform asymptotic accuracy under moderate dependence conditions within panels and that these conditions are fulfilled, e.g., by any MA(1) time series. As a counterexample we show that for AR(1) time series, close to the non-stationary case, the dependence is too strong and uniform asymptotic accuracy cannot be ensured. Finally, we conduct simulations to demonstrate that our results are practically applicable and that our methodological suggestions are advantageous.

Integrated soil fertility management in eastern and western Africa: The role of knowledge and innovation systems, its adoption and impact

The rise in population growth, as well as nutrient mining, has contributed to low agricultural productivity in Sub-Saharan Africa (SSA). A plethora of technologies to boost agricultural production have been developed but the dissemination of these agricultural innovations and subsequent uptake by smallholder farmers has remained a challenge. Scientists and philanthropists have adopted the Integrated Soil Fertility Management (ISFM) paradigm as a means to promote sustainable intensification of African farming systems. This comparative study aimed: 1) To assess the efficacy of Agricultural Knowledge and Innovation Systems (AKIS) in East (Kenya) and West (Ghana) Africa in the communication and dissemination of ISFM (Study I); 2) To investigate how specifically soil quality, and more broadly socio-economic status and institutional factors, influence farmer adoption of ISFM (Study II); and 3) To assess the effect of ISFM on maize yield and total household income of smallholder farmers (Study III). To address these aims, a mixed methodology approach was employed for study I. AKIS actors were subjected to social network analysis methods and in-depth interviews. Structured questionnaires were administered to 285 farming households in Tamale and 300 households in Kakamega selected using a stratified random sampling approach. There was a positive relationship between complete ISFM awareness among farmers and weak knowledge ties to both formal and informal actors at both research locations. The Kakamega AKIS revealed a relationship between complete ISFM awareness among farmers and them having strong knowledge ties to formal actors implying that further integration of formal actors with farmers’ local knowledge is crucial for the agricultural development progress. The structured questionnaire was also utilized to answer the query pertaining to study II. Soil samples (0-20 cm depth) were drawn from 322 (Tamale, Ghana) and 459 (Kakamega, Kenya) maize plots and analysed non-destructively for various soil fertility indicators. Ordinal regression modeling was applied to assess the cumulative adoption of ISFM. According to model estimates, soil carbon seemed to preclude farmers from intensifying input use in Tamale, whereas in Kakamega it spurred complete adoption. This varied response by farmers to soil quality conditions is multifaceted. From the Tamale perspective, it is consistent with farmers’ tendency to judiciously allocate scarce resources. Viewed from the Kakamega perspective, it points to a need for farmers here to intensify agricultural production in order to foster food security. In Kakamega, farmers with more acidic soils were more likely to adopt ISFM. Other household and farm-level factors necessary for ISFM adoption included off-farm income, livestock ownership, farmer associations, and market inter-linkages. Finally, in study III a counterfactual model was used to calculate the difference in outcomes (yield and household income) of the treatment (ISFM adoption) in order to estimate causal effects of ISFM adoption. Adoption of ISFM contributed to a yield increase of 16% in both Tamale and Kakamega. The innovation affected total household income only in Tamale, where ISFM adopters had an income gain of 20%. This may be attributable to the different policy contexts under which the two sets of farmers operate. The main recommendations underscored the need to: (1) improve the functioning of AKIS, (2) enhance farmer access to hybrid maize seed and credit, (3) and conduct additional multi-locational studies as farmers operate under varying contexts.

Sustainability analysis in the mining sector: a case study on new recycling technologies for sulphidic mine residues valorisation

Research has demonstrated that mining activities can cause serious impacts on the environment, as well as the surrounding communities, mainly due to the unsafe storage of mine tailings. This research focuses on the sustainability assessment of new technologies for the recovery of metals from mine residues. The assessment consists in the evaluation of the environmental, economic, and social impacts through the Life Cycle based methods: Life Cycle Assessment (LCA), Life Cycle Costing (LCC), and Social Life Cycle Assessment (SLCA). The analyses are performed on the Mondo Minerals bioleaching project, which aim is to recover nickel and cobalt from the Sotkamo and Vuonos mine tailings. The LCA demonstrates that the project contributes to the avoided production of nickel and cobalt concentrates from new resources, hence reducing several environmental impacts. The LCC analysis shows that the company’s main costs are linked to the bioleaching process, caused by electricity consumption and the chemicals used. The SLCA analyses the impacts on three main stakeholder categories: workers, local community, and society. The results demonstrated that a fair salary (or the absence of it) impacts the workers the most, while the local community stakeholder category impacts are related to the access to material resources. The health and safety category is the most impacted category for the society stakeholder. The environmental and economic analyses demonstrate that the recovery of mine tailings may represents a good opportunity for mine companies both to reduce the environmental impacts linked to mine tailings and to increase the profitability. In particular, the project helps reduce the amounts of metals extracted from new resources and demonstrates that the use of the bioleaching technology for the extraction of metals can be economically profitable.

Machine learning tools for protein annotation: the cases of transmembrane β-barrel and myristoylated proteins

Biology is now a “Big Data Science” thanks to technological advancements allowing the characterization of the whole macromolecular content of a cell or a collection of cells. This opens interesting perspectives, but only a small portion of this data may be experimentally characterized. From this derives the demand of accurate and efficient computational tools for automatic annotation of biological molecules. This is even more true when dealing with membrane proteins, on which my research project is focused leading to the development of two machine learning-based methods: BetAware-Deep and SVMyr. BetAware-Deep is a tool for the detection and topology prediction of transmembrane beta-barrel proteins found in Gram-negative bacteria. These proteins are involved in many biological processes and primary candidates as drug targets. BetAware-Deep exploits the combination of a deep learning framework (bidirectional long short-term memory) and a probabilistic graphical model (grammatical-restrained hidden conditional random field). Moreover, it introduced a modified formulation of the hydrophobic moment, designed to include the evolutionary information. BetAware-Deep outperformed all the available methods in topology prediction and reported high scores in the detection task. Glycine myristoylation in Eukaryotes is the binding of a myristic acid on an N-terminal glycine. SVMyr is a fast method based on support vector machines designed to predict this modification in dataset of proteomic scale. It uses as input octapeptides and exploits computational scores derived from experimental examples and mean physicochemical features. SVMyr outperformed all the available methods for co-translational myristoylation prediction. In addition, it allows (as a unique feature) the prediction of post-translational myristoylation. Both the tools here described are designed having in mind best practices for the development of machine learning-based tools outlined by the bioinformatics community. Moreover, they are made available via user-friendly web servers. All this make them valuable tools for filling the gap between sequential and annotated data.

Simulation of photoinduced processes in organic chromophores: mapping photochemically relevant states for accurate dynamics

The simulation of ultrafast photoinduced processes is a fundamental step towards the understanding of the underlying molecular mechanism and interpretation/prediction of experimental data. Performing a computer simulation of a complex photoinduced process is only possible introducing some approximations but, in order to obtain reliable results, the need to reduce the complexity must balance with the accuracy of the model, which should include all the relevant degrees of freedom and a quantitatively correct description of the electronic states involved in the process. This work presents new computational protocols and strategies for the parameterisation of accurate models for photochemical/photophysical processes based on state-of-the-art multiconfigurational wavefunction-based methods. The required ingredients for a dynamics simulation include potential energy surfaces (PESs) as well as electronic state couplings, which must be mapped across the wide range of geometries visited during the wavepacket/trajectory propagation. The developed procedures allow to obtain solid and extended databases reducing as much as possible the computational cost, thanks to, e.g., specific tuning of the level of theory for different PES regions and/or direct calculation of only the needed components of vectorial quantities (like gradients or nonadiabatic couplings). The presented approaches were applied to three case studies (azobenzene, pyrene, visual rhodopsin), all requiring an accurate parameterisation but for different reasons. The resulting models and simulations allowed to elucidate the mechanism and time scale of the internal conversion, reproducing or even predicting new transient experiments. The general applicability of the developed protocols to systems with different peculiarities and the possibility to parameterise different types of dynamics on an equal footing (classical vs purely quantum) prove that the developed procedures are flexible enough to be tailored for each specific system, and pave the way for exact quantum dynamics with multiple degrees of freedom.

Variational and learning models for image and time series inverse problems

Inverse problems are at the core of many challenging applications. Variational and learning models provide estimated solutions of inverse problems as the outcome of specific reconstruction maps. In the variational approach, the result of the reconstruction map is the solution of a regularized minimization problem encoding information on the acquisition process and prior knowledge on the solution. In the learning approach, the reconstruction map is a parametric function whose parameters are identified by solving a minimization problem depending on a large set of data. In this thesis, we go beyond this apparent dichotomy between variational and learning models and we show they can be harmoniously merged in unified hybrid frameworks preserving their main advantages. We develop several highly efficient methods based on both these model-driven and data-driven strategies, for which we provide a detailed convergence analysis. The arising algorithms are applied to solve inverse problems involving images and time series. For each task, we show the proposed schemes improve the performances of many other existing methods in terms of both computational burden and quality of the solution. In the first part, we focus on gradient-based regularized variational models which are shown to be effective for segmentation purposes and thermal and medical image enhancement. We consider gradient sparsity-promoting regularized models for which we develop different strategies to estimate the regularization strength. Furthermore, we introduce a novel gradient-based Plug-and-Play convergent scheme considering a deep learning based denoiser trained on the gradient domain. In the second part, we address the tasks of natural image deblurring, image and video super resolution microscopy and positioning time series prediction, through deep learning based methods. We boost the performances of supervised, such as trained convolutional and recurrent networks, and unsupervised deep learning strategies, such as Deep Image Prior, by penalizing the losses with handcrafted regularization terms.

Contaminants of emerging concern in drinking water: assessment of the effectiveness of removal by drinking water treatments and possible risks to human health

Contaminants of emerging concern are increasingly detected in the water cycle, with endocrine-disrupting chemicals (EDCs) receiving attention due to their potential to cause adverse health effects even at low concentrations. Although the EU has recently introduced some EDCs into drinking water legislation, most drinking water treatment plants (DWTPs) are not designed to remove EDCs, making their detection and removal in DWTPs an important challenge. The aim of this doctoral project was to investigate hormones and phenolic compounds as suspected EDCs in drinking waters across the Romagna area (Italy). The main objectives were to assess the occurrence of considered contaminants in source and drinking water from three DWTPs, characterize the effectiveness of removal by different water treatment processes, and evaluate the potential biological impact on drinking water and human health. Specifically, a complementary approach of target chemical analysis and effect-based methods was adopted to explore drinking water quality, treatment efficacy, and biological potential. This study found that nonylphenol (NP) was prevalent in all samples, followed by BPA. Sporadic contamination of hormones was found only in source waters. Although the measured EDC concentrations in drinking water did not exceed threshold guideline values, the potential role of DWTPs as an additional source of EDC contamination should be considered. Significant increases in BPA and NP levels were observed during water treatment steps, which were also reflected in estrogenic and mutagenic responses in water samples after the ultrafiltration. This highlights the need to monitor water quality during various treatment processes to improve the efficiency of DWTPs. Biological assessments on finished water did not reveal any bioactivity, except for few treated water samples that exhibited estrogenic responses. Overall, the data emphasize the high quality of produced drinking water and the value of applying integrated chemical analysis and in vitro bioassays for water quality assessment.

Numerical and physical modelling of axially loaded piles for offshore wind turbines

The scope of the thesis is to broaden the knowledge about axially loaded pipe piles, that can play as foundations for offshore wind turbines based on jacket structures. The goal of the work was pursued by interpreting experimental data on large-scale model piles and by developing numerical tools for the prediction of their monotonic response to tensile and compressive loads to failure. The availability of experimental results on large scale model piles produced in two different campaigns at Fraunhofer IWES (Hannover, Germany) represented the reference for the whole work. Data from CPTs, blow counts during installation and load-displacement curves allowed to develop considerations on the experimental results and comparison with empirical methods from literature, such as CPT-based methods and Load Transfer methods. The understanding of soil-structure interaction mechanisms has been involved in the study in order to better assess the mechanical response of the sand with the scope to help in developing predictive tools of the experiments. A lack of information on the response of Rohsand 3152 when in contact with steel was highlighted, so the necessity of better assessing its response was fulfilled with a comprehensive campaign of interface shear test. It was found how the response of the sand to ultimate conditions evolve with the roughness of the steel, which is a precious information to take account of when attempting the prediction of a pile capacity. Parallel to this topic, the work has developed a numerical modelling procedure that was validated on the available large-scale model piles at IWES. The modelling strategy is intended to build a FE model whose mechanical properties of the sand come from an interpretation of commonly available geotechnical tests. The results of the FE model were compared with other predictive tools currently used in the engineering practice.

Applied genomics to monitor the genetic diversity and integrity of honey bee (Apis mellifera) populations and define sustainable strategies to benefit the beekeeping sector

Honey bees are considered keystone species in ecosystem, the effect of harmful pesticides for the honey bees, the action of extreme climatic waves and their consequence on honey bees health can cause the loss of many colonies which could contribute to the reduction of the effective population size and incentive the use of non-autochthonous queens to replace dead colonies. Over the last decades, the use of non-ligustica bee subspecies in Italy has increased and together with the mentioned phenomena exposed native honey bees to hybridization, laeding to a dramatic loss of genetic erosion and admixture. Healthy genetic diversity within honey bee populations is critical to provide tolerance and resistance to current and future threatening. Nowadays it is urgent to design strategies for the conservation of local subspecies and their valorisation on a productive scale. In this Thesis we applied genomics tool for the analysis of the genetic diversity and the genomic integrity of honey bee populations in Italy are described. In this work mtDNA based methods are presented using honey bee DNA or honey eDNA as source of information of the genetic diversity of A. mellifera at different level. Taken together, the results derived from these studies should enlarge the knowledge of the genetic diversity and integrity of the honey bee populations in Italy, filling the gap of information necessary to design efficient conservation programmes. Furthermore, the methods presented in these works will provide a tool for the honey authentication to sustain and valorise beekeeping products and sector against frauds.

Embedding Video Object Detection Capabilities on Low-Power Nano-Drones

Our objective for this thesis work was the deployment of a Neural Network based approach for video object detection on board a nano-drone. Furthermore, we have studied some possible extensions to exploit the temporal nature of videos to improve the detection capabilities of our algorithm. For our project, we have utilized the Mobilenetv2/v3SSDLite due to their limited computational and memory requirements. We have trained our networks on the IMAGENET VID 2015 dataset and to deploy it onto the nano-drone we have used the NNtool and Autotiler tools by GreenWaves. To exploit the temporal nature of video data we have tried different approaches: the introduction of an LSTM based convolutional layer in our architecture, the introduction of a Kalman filter based tracker as a postprocessing step to augment the results of our base architecture. We have obtain a total improvement in our performances of about 2.5 mAP with the Kalman filter based method(BYTE). Our detector run on a microcontroller class processor on board the nano-drone at 1.63 fps.

Modeling microbes: New methods for integrated metabolic and regulatory network reconstruction

PhD Thesis in Bioengineering

EEG-based functional brain networks: does the network size matter?

Functional connectivity in human brain can be represented as a network using electroencephalography (EEG) signals. These networks--whose nodes can vary from tens to hundreds--are characterized by neurobiologically meaningful graph theory metrics. This study investigates the degree to which various graph metrics depend upon the network size. To this end, EEGs from 32 normal subjects were recorded and functional networks of three different sizes were extracted. A state-space based method was used to calculate cross-correlation matrices between different brain regions. These correlation matrices were used to construct binary adjacency connectomes, which were assessed with regards to a number of graph metrics such as clustering coefficient, modularity, efficiency, economic efficiency, and assortativity. We showed that the estimates of these metrics significantly differ depending on the network size. Larger networks had higher efficiency, higher assortativity and lower modularity compared to those with smaller size and the same density. These findings indicate that the network size should be considered in any comparison of networks across studies.

Soil infiltration based on bp neural network and grey relational analysis

Soil infiltration is a key link of the natural water cycle process. Studies on soil permeability are conducive for water resources assessment and estimation, runoff regulation and management, soil erosion modeling, nonpoint and point source pollution of farmland, among other aspects. The unequal influence of rainfall duration, rainfall intensity, antecedent soil moisture, vegetation cover, vegetation type, and slope gradient on soil cumulative infiltration was studied under simulated rainfall and different underlying surfaces. We established a six factor-model of soil cumulative infiltration by the improved back propagation (BP)-based artificial neural network algorithm with a momentum term and self-adjusting learning rate. Compared to the multiple nonlinear regression method, the stability and accuracy of the improved BP algorithm was better. Based on the improved BP model, the sensitive index of these six factors on soil cumulative infiltration was investigated. Secondly, the grey relational analysis method was used to individually study grey correlations among these six factors and soil cumulative infiltration. The results of the two methods were very similar. Rainfall duration was the most influential factor, followed by vegetation cover, vegetation type, rainfall intensity and antecedent soil moisture. The effect of slope gradient on soil cumulative infiltration was not significant.