Statistical learning of random probability measures

The study of random probability measures is a lively research topic that has attracted interest from different fields in recent years. In this thesis, we consider random probability measures in the context of Bayesian nonparametrics, where the law of a random probability measure is used as prior distribution, and in the context of distributional data analysis, where the goal is to perform inference given avsample from the law of a random probability measure. The contributions contained in this thesis can be subdivided according to three different topics: (i) the use of almost surely discrete repulsive random measures (i.e., whose support points are well separated) for Bayesian model-based clustering, (ii) the proposal of new laws for collections of random probability measures for Bayesian density estimation of partially exchangeable data subdivided into different groups, and (iii) the study of principal component analysis and regression models for probability distributions seen as elements of the 2-Wasserstein space. Specifically, for point (i) above we propose an efficient Markov chain Monte Carlo algorithm for posterior inference, which sidesteps the need of split-merge reversible jump moves typically associated with poor performance, we propose a model for clustering high-dimensional data by introducing a novel class of anisotropic determinantal point processes, and study the distributional properties of the repulsive measures, shedding light on important theoretical results which enable more principled prior elicitation and more efficient posterior simulation algorithms. For point (ii) above, we consider several models suitable for clustering homogeneous populations, inducing spatial dependence across groups of data, extracting the characteristic traits common to all the data-groups, and propose a novel vector autoregressive model to study of growth curves of Singaporean kids. Finally, for point (iii), we propose a novel class of projected statistical methods for distributional data analysis for measures on the real line and on the unit-circle.

Learning representations for graph-structured socio-technical systems

The recent widespread use of social media platforms and web services has led to a vast amount of behavioral data that can be used to model socio-technical systems. A significant part of this data can be represented as graphs or networks, which have become the prevalent mathematical framework for studying the structure and the dynamics of complex interacting systems. However, analyzing and understanding these data presents new challenges due to their increasing complexity and diversity. For instance, the characterization of real-world networks includes the need of accounting for their temporal dimension, together with incorporating higher-order interactions beyond the traditional pairwise formalism. The ongoing growth of AI has led to the integration of traditional graph mining techniques with representation learning and low-dimensional embeddings of networks to address current challenges. These methods capture the underlying similarities and geometry of graph-shaped data, generating latent representations that enable the resolution of various tasks, such as link prediction, node classification, and graph clustering. As these techniques gain popularity, there is even a growing concern about their responsible use. In particular, there has been an increased emphasis on addressing the limitations of interpretability in graph representation learning. This thesis contributes to the advancement of knowledge in the field of graph representation learning and has potential applications in a wide range of complex systems domains. We initially focus on forecasting problems related to face-to-face contact networks with time-varying graph embeddings. Then, we study hyperedge prediction and reconstruction with simplicial complex embeddings. Finally, we analyze the problem of interpreting latent dimensions in node embeddings for graphs. The proposed models are extensively evaluated in multiple experimental settings and the results demonstrate their effectiveness and reliability, achieving state-of-the-art performances and providing valuable insights into the properties of the learned representations.

Developing a radiomic based machine learning model to identify patients with resected non-small-cell lung cancer at high risk of relapse

Background There is a wide variation of recurrence risk of Non-small-cell lung cancer (NSCLC) within the same Tumor Node Metastasis (TNM) stage, suggesting that other parameters are involved in determining this probability. Radiomics allows extraction of quantitative information from images that can be used for clinical purposes. The primary objective of this study is to develop a radiomic prognostic model that predicts a 3 year disease free-survival (DFS) of resected Early Stage (ES) NSCLC patients. Material and Methods 56 pre-surgery non contrast Computed Tomography (CT) scans were retrieved from the PACS of our institution and anonymized. Then they were automatically segmented with an open access deep learning pipeline and reviewed by an experienced radiologist to obtain 3D masks of the NSCLC. Images and masks underwent to resampling normalization and discretization. From the masks hundreds Radiomic Features (RF) were extracted using Py-Radiomics. Hence, RF were reduced to select the most representative features. The remaining RF were used in combination with Clinical parameters to build a DFS prediction model using Leave-one-out cross-validation (LOOCV) with Random Forest. Results and Conclusion A poor agreement between the radiologist and the automatic segmentation algorithm (DICE score of 0.37) was found. Therefore, another experienced radiologist manually segmented the lesions and only stable and reproducible RF were kept. 50 RF demonstrated a high correlation with the DFS but only one was confirmed when clinicopathological covariates were added: Busyness a Neighbouring Gray Tone Difference Matrix (HR 9.610). 16 clinical variables (which comprised TNM) were used to build the LOOCV model demonstrating a higher Area Under the Curve (AUC) when RF were included in the analysis (0.67 vs 0.60) but the difference was not statistically significant (p=0,5147).

Vocabulary Transfer and Knowledge Distillation for Language Model Compression

Natural Language Processing (NLP) has seen tremendous improvements over the last few years. Transformer architectures achieved impressive results in almost any NLP task, such as Text Classification, Machine Translation, and Language Generation. As time went by, transformers continued to improve thanks to larger corpora and bigger networks, reaching hundreds of billions of parameters. Training and deploying such large models has become prohibitively expensive, such that only big high tech companies can afford to train those models. Therefore, a lot of research has been dedicated to reducing a model’s size. In this thesis, we investigate the effects of Vocabulary Transfer and Knowledge Distillation for compressing large Language Models. The goal is to combine these two methodologies to further compress models without significant loss of performance. In particular, we designed different combination strategies and conducted a series of experiments on different vertical domains (medical, legal, news) and downstream tasks (Text Classification and Named Entity Recognition). Four different methods involving Vocabulary Transfer (VIPI) with and without a Masked Language Modelling (MLM) step and with and without Knowledge Distillation are compared against a baseline that assigns random vectors to new elements of the vocabulary. Results indicate that VIPI effectively transfers information of the original vocabulary and that MLM is beneficial. It is also noted that both vocabulary transfer and knowledge distillation are orthogonal to one another and may be applied jointly. The application of knowledge distillation first before subsequently applying vocabulary transfer is recommended. Finally, model performance due to vocabulary transfer does not always show a consistent trend as the vocabulary size is reduced. Hence, the choice of vocabulary size should be empirically selected by evaluation on the downstream task similar to hyperparameter tuning.

Meta-learning for informed model searches in Automated Machine Learning

Day by day, machine learning is changing our lives in ways we could not have imagined just 5 years ago. ML expertise is more and more requested and needed, though just a limited number of ML engineers are available on the job market, and their knowledge is always limited by an inherent characteristic of theirs: they are humans. This thesis explores the possibilities offered by meta-learning, a new field in ML that takes learning a level higher: models are trained on other models' training data, starting from features of the dataset they were trained on, inference times, obtained performances, to try to understand the relationship between a good model and the way it was obtained. The so-called metamodel was trained on data collected by OpenML, the largest ML metadata platform that's publicly available today. Datasets were analyzed to obtain meta-features that describe them, which were then tied to model performances in a regression task. The obtained metamodel predicts the expected performances of a given model type (e.g., a random forest) on a given ML task (e.g., classification on the UCI census dataset). This research was then integrated into a custom-made AutoML framework, to show how meta-learning is not an end in itself, but it can be used to further progress our ML research. Encoding ML engineering expertise in a model allows better, faster, and more impactful ML applications across the whole world, while reducing the cost that is inevitably tied to human engineers.

Machine Learning in Automatic Tabulation for Constraint Model Reformulation

Combinatorial decision and optimization problems belong to numerous applications, such as logistics and scheduling, and can be solved with various approaches. Boolean Satisfiability and Constraint Programming solvers are some of the most used ones and their performance is significantly influenced by the model chosen to represent a given problem. This has led to the study of model reformulation methods, one of which is tabulation, that consists in rewriting the expression of a constraint in terms of a table constraint. To apply it, one should identify which constraints can help and which can hinder the solving process. So far this has been performed by hand, for example in MiniZinc, or automatically with manually designed heuristics, in Savile Row. Though, it has been shown that the performances of these heuristics differ across problems and solvers, in some cases helping and in others hindering the solving procedure. However, recent works in the field of combinatorial optimization have shown that Machine Learning (ML) can be increasingly useful in the model reformulation steps. This thesis aims to design a ML approach to identify the instances for which Savile Row’s heuristics should be activated. Additionally, it is possible that the heuristics miss some good tabulation opportunities, so we perform an exploratory analysis for the creation of a ML classifier able to predict whether or not a constraint should be tabulated. The results reached towards the first goal show that a random forest classifier leads to an increase in the performances of 4 different solvers. The experimental results in the second task show that a ML approach could improve the performance of a solver for some problem classes.

Random Access Procedures in 5G IoT via NTN: System level performances

The Internet of Things (IoT) is a critical pillar in the digital transformation because it enables interaction with the physical world through remote sensing and actuation. Owing to the advancements in wireless technology, we now have the opportunity of using their features to the best of our abilities and improve over the current situation. Indeed, the Internet of Things market is expanding at an exponential rate, with devices such as alarms and detectors, smart metres, trackers, and wearables being used on a global scale for automotive and agriculture, environment monitoring, infrastructure surveillance and management, healthcare, energy and utilities, logistics, good tracking, and so on. The Third Generation Partnership Project (3GPP) acknowledged the importance of IoT by introducing new features to support it. In particular, in Rel.13, the 3GPP introduced the so-called IoT to support Low Power Wide Area Networks (LPWAN).As these devices will be distributed in areas where terrestrial networks are not feasible or commercially viable, satellite networks will play a complementary role due to their ability to provide global connectivity via their large footprint size and short service deployment time. In this context, the goal of this thesis is to investigate the viability of integrating IoT technology with satellite communication (SatCom) systems, with a focus on the Random Access(RA) Procedure. Indeed, the RA is the most critical procedure because it allows the UE to achieve uplink synchronisation, obtain the permanent ID, and obtain uplink transmission resources. The goal of this thesis is to evaluate preamble detection in the SatCom environment.

Local and nonlocal integro-differential reaction-diffusion models for protein dynamics in Alzheimer's disease

In this work, integro-differential reaction-diffusion models are presented for the description of the temporal and spatial evolution of the concentrations of Abeta and tau proteins involved in Alzheimer's disease. Initially, a local model is analysed: this is obtained by coupling with an interaction term two heterodimer models, modified by adding diffusion and Holling functional terms of the second type. We then move on to the presentation of three nonlocal models, which differ according to the type of the growth (exponential, logistic or Gompertzian) considered for healthy proteins. In these models integral terms are introduced to consider the interaction between proteins that are located at different spatial points possibly far apart. For each of the models introduced, the determination of equilibrium points with their stability and a study of the clearance inequalities are carried out. In addition, since the integrals introduced imply a spatial nonlocality in the models exhibited, some general features of nonlocal models are presented. Afterwards, with the aim of developing simulations, it is decided to transfer the nonlocal models to a brain graph called connectome. Therefore, after setting out the construction of such a graph, we move on to the description of Laplacian and convolution operations on a graph. Taking advantage of all these elements, we finally move on to the translation of the continuous models described above into discrete models on the connectome. To conclude, the results of some simulations concerning the discrete models just derived are presented.

Integrated Proteomics Identified Up-regulated Focal Adhesion-mediated Proteins In Human Squamous Cell Carcinoma In An Orthotopic Murine Model.

Understanding the molecular mechanisms of oral carcinogenesis will yield important advances in diagnostics, prognostics, effective treatment, and outcome of oral cancer. Hence, in this study we have investigated the proteomic and peptidomic profiles by combining an orthotopic murine model of oral squamous cell carcinoma (OSCC), mass spectrometry-based proteomics and biological network analysis. Our results indicated the up-regulation of proteins involved in actin cytoskeleton organization and cell-cell junction assembly events and their expression was validated in human OSCC tissues. In addition, the functional relevance of talin-1 in OSCC adhesion, migration and invasion was demonstrated. Taken together, this study identified specific processes deregulated in oral cancer and provided novel refined OSCC-targeting molecules.

Electronic And Structural Study Of Pt-modified Au Vicinal Surfaces: A Model System For Pt-au Catalysts.

Two single crystalline surfaces of Au vicinal to the (111) plane were modified with Pt and studied using scanning tunneling microscopy (STM) and X-ray photoemission spectroscopy (XPS) in ultra-high vacuum environment. The vicinal surfaces studied are Au(332) and Au(887) and different Pt coverage (θPt) were deposited on each surface. From STM images we determine that Pt deposits on both surfaces as nanoislands with heights ranging from 1 ML to 3 ML depending on θPt. On both surfaces the early growth of Pt ad-islands occurs at the lower part of the step edge, with Pt ad-atoms being incorporated into the steps in some cases. XPS results indicate that partial alloying of Pt occurs at the interface at room temperature and at all coverage, as suggested by the negative chemical shift of Pt 4f core line, indicating an upward shift of the d-band center of the alloyed Pt. Also, the existence of a segregated Pt phase especially at higher coverage is detected by XPS. Sample annealing indicates that the temperature rise promotes a further incorporation of Pt atoms into the Au substrate as supported by STM and XPS results. Additionally, the catalytic activity of different PtAu systems reported in the literature for some electrochemical reactions is discussed considering our findings.

Reduced Insulin Clearance And Lower Insulin-degrading Enzyme Expression In The Liver Might Contribute To The Thrifty Phenotype Of Protein-restricted Mice.

Nutrient restriction during the early stages of life usually leads to alterations in glucose homeostasis, mainly insulin secretion and sensitivity, increasing the risk of metabolic disorders in adulthood. Despite growing evidence regarding the importance of insulin clearance during glucose homeostasis in health and disease, no information exists about this process in malnourished animals. Thus, in the present study, we aimed to determine the effect of a nutrient-restricted diet on insulin clearance using a model in which 30-d-old C57BL/6 mice were exposed to a protein-restricted diet for 14 weeks. After this period, we evaluated many metabolic variables and extracted pancreatic islet, liver, gastrocnemius muscle (GCK) and white adipose tissue samples from the control (normal-protein diet) and restricted (low-protein diet, LP) mice. Insulin concentrations were determined using RIA and protein expression and phosphorylation by Western blot analysis. The LP mice exhibited lower body weight, glycaemia, and insulinaemia, increased glucose tolerance and altered insulin dynamics after the glucose challenge. The improved glucose tolerance could partially be explained by an increase in insulin sensitivity through the phosphorylation of the insulin receptor/protein kinase B and AMP-activated protein kinase/acetyl-CoA carboxylase in the liver, whereas the changes in insulin dynamics could be attributed to reduced insulin secretion coupled with reduced insulin clearance and lower insulin-degrading enzyme (IDE) expression in the liver and GCK. In summary, protein-restricted mice not only produce and secrete less insulin, but also remove and degrade less insulin. This phenomenon has the double benefit of sparing insulin while prolonging and potentiating its effects, probably due to the lower expression of IDE in the liver, possibly with long-term consequences.

The Soluble Guanylyl Cyclase Activator Bay 60-2770 Potently Relaxes The Pulmonary Artery On Congenital Diaphragmatic Hernia Rabbit Model.

Congenital diaphragmatic hernia (CDH) is associated with pulmonary hypertension which is often difficult to manage, and a significant cause of morbidity and mortality. In this study, we have used a rabbit model of CDH to evaluate the effects of BAY 60-2770 on the in vitro reactivity of left pulmonary artery. CDH was performed in New Zealand rabbit fetuses (n = 10 per group) and compared to controls. Measurements of body, total and left lung weights (BW, TLW, LLW) were done. Pulmonary artery rings were pre-contracted with phenylephrine (10 μM), after which cumulative concentration-response curves to glyceryl trinitrate (GTN; NO donor), tadalafil (PDE5 inhibitor) and BAY 60-2770 (sGC activator) were obtained as well as the levels of NO (NO3/NO2). LLW, TLW and LBR were decreased in CDH (p < 0.05). In left pulmonary artery, the potency (pEC50) for GTN was markedly lower in CDH (8.25 ± 0.02 versus 9.27 ± 0.03; p < 0.01). In contrast, the potency for BAY 60-2770 was markedly greater in CDH (11.7 ± 0.03 versus 10.5 ± 0.06; p < 0.01). The NO2/NO3 levels were 62 % higher in CDH (p < 0.05). BAY 60-2770 exhibits a greater potency to relax the pulmonary artery in CDH, indicating a potential use for pulmonary hypertension in this disease.

An Integrated Framework To Improve The Concept Of Resource Specialisation.

Resource specialisation, although a fundamental component of ecological theory, is employed in disparate ways. Most definitions derive from simple counts of resource species. We build on recent advances in ecophylogenetics and null model analysis to propose a concept of specialisation that comprises affinities among resources as well as their co-occurrence with consumers. In the distance-based specialisation index (DSI), specialisation is measured as relatedness (phylogenetic or otherwise) of resources, scaled by the null expectation of random use of locally available resources. Thus, specialists use significantly clustered sets of resources, whereas generalists use over-dispersed resources. Intermediate species are classed as indiscriminate consumers. The effectiveness of this approach was assessed with differentially restricted null models, applied to a data set of 168 herbivorous insect species and their hosts. Incorporation of plant relatedness and relative abundance greatly improved specialisation measures compared to taxon counts or simpler null models, which overestimate the fraction of specialists, a problem compounded by insufficient sampling effort. This framework disambiguates the concept of specialisation with an explicit measure applicable to any mode of affinity among resource classes, and is also linked to ecological and evolutionary processes. This will enable a more rigorous deployment of ecological specialisation in empirical and theoretical studies.

Bay 41-2272, A Soluble Guanylate Cyclase Stimulator, Relaxes Isolated Human Ureter In A Standardized In Vitro Model.

To characterize the relaxation induced by BAY 41-2272 in human ureteral segments. Ureter specimens (n = 17) from multiple organ human deceased donors (mean age 40 ± 3.2 years, male/female ratio 2:1) were used to characterize the relaxing response of BAY 41-2272. Immunohistochemical analysis for endothelial and neuronal nitric oxide synthase, guanylate cyclase stimulator (sGC) and type 5 phosphodiesterase was also performed. The potency values were determined as the negative log of the molar to produce 50% of the maximal relaxation in potassium chloride-precontracted specimens. The unpaired Student t test was used for the comparisons. Immunohistochemistry revealed the presence of endothelial nitric oxide synthase in vessel endothelia and neuronal nitric oxide synthase in urothelium and nerve structures. sGC was expressed in the smooth muscle and urothelium layer, and type 5 phosphodiesterase was present in the smooth muscle only. BAY 41-2272 (0.001-100 μM) relaxed the isolated ureter in a concentration dependent manner, with a potency and maximal relaxation value of 5.82 ± 0.14 and 84% ± 5%, respectively. The addition of nitric oxide synthase and sGC inhibitors reduced the maximal relaxation values by 21% and 45%, respectively. However, the presence of sildenafil (100 nM) significantly potentiated (6.47 ± 0.10, P <.05) this response. Neither glibenclamide or tetraethylammonium nor ureteral urothelium removal influenced the relaxation response by BAY 41-2272. BAY 41-2272 relaxes the human isolated ureter in a concentration-dependent manner, mainly by activating the sGC enzyme in smooth muscle cells rather than in the urothelium, although a cyclic guanosine monophosphate-independent mechanism might have a role. The potassium channels do not seem to be involved.

Atomic Charge Transfer-counter Polarization Effects Determine Infrared Ch Intensities Of Hydrocarbons: A Quantum Theory Of Atoms In Molecules Model.

Atomic charge transfer-counter polarization effects determine most of the infrared fundamental CH intensities of simple hydrocarbons, methane, ethylene, ethane, propyne, cyclopropane and allene. The quantum theory of atoms in molecules/charge-charge flux-dipole flux model predicted the values of 30 CH intensities ranging from 0 to 123 km mol(-1) with a root mean square (rms) error of only 4.2 km mol(-1) without including a specific equilibrium atomic charge term. Sums of the contributions from terms involving charge flux and/or dipole flux averaged 20.3 km mol(-1), about ten times larger than the average charge contribution of 2.0 km mol(-1). The only notable exceptions are the CH stretching and bending intensities of acetylene and two of the propyne vibrations for hydrogens bound to sp hybridized carbon atoms. Calculations were carried out at four quantum levels, MP2/6-311++G(3d,3p), MP2/cc-pVTZ, QCISD/6-311++G(3d,3p) and QCISD/cc-pVTZ. The results calculated at the QCISD level are the most accurate among the four with root mean square errors of 4.7 and 5.0 km mol(-1) for the 6-311++G(3d,3p) and cc-pVTZ basis sets. These values are close to the estimated aggregate experimental error of the hydrocarbon intensities, 4.0 km mol(-1). The atomic charge transfer-counter polarization effect is much larger than the charge effect for the results of all four quantum levels. Charge transfer-counter polarization effects are expected to also be important in vibrations of more polar molecules for which equilibrium charge contributions can be large.