Meaningful insights: explainability techniques for black-box models on tabular data

Artificial Intelligence (AI) and Machine Learning (ML) are novel data analysis techniques providing very accurate prediction results. They are widely adopted in a variety of industries to improve efficiency and decision-making, but they are also being used to develop intelligent systems. Their success grounds upon complex mathematical models, whose decisions and rationale are usually difficult to comprehend for human users to the point of being dubbed as black-boxes. This is particularly relevant in sensitive and highly regulated domains. To mitigate and possibly solve this issue, the Explainable AI (XAI) field became prominent in recent years. XAI consists of models and techniques to enable understanding of the intricated patterns discovered by black-box models. In this thesis, we consider model-agnostic XAI techniques, which can be applied to Tabular data, with a particular focus on the Credit Scoring domain. Special attention is dedicated to the LIME framework, for which we propose several modifications to the vanilla algorithm, in particular: a pair of complementary Stability Indices that accurately measure LIME stability, and the OptiLIME policy which helps the practitioner finding the proper balance among explanations' stability and reliability. We subsequently put forward GLEAMS a model-agnostic surrogate interpretable model which requires to be trained only once, while providing both Local and Global explanations of the black-box model. GLEAMS produces feature attributions and what-if scenarios, from both dataset and model perspective. Eventually, we argue that synthetic data are an emerging trend in AI, being more and more used to train complex models instead of original data. To be able to explain the outcomes of such models, we must guarantee that synthetic data are reliable enough to be able to translate their explanations to real-world individuals. To this end we propose DAISYnt, a suite of tests to measure synthetic tabular data quality and privacy.

Complexity of N-Myc transcriptional function in childhood neuroblastoma

Myc is a transcription factor that can activate transcription of several hundreds genes by direct binding to their promoters at specific DNA sequences (E-box). However, recent studies have also shown that it can exert its biological role by repressing transcription. Such studies collectively support a model in which c-Myc-mediated repression occurs through interactions with transcription factors bound to promoter DNA regions but not through direct recognition of typical E-box sequences. Here, we investigated whether N-Myc can also repress gene transcription, and how this is mechanistically achieved. We used human neuroblastoma cells as a model system in that N-MYC amplification/over-expression represents a key prognostic marker of this tumour. By means of transcription profile analyses we could identify at least 5 genes (TRKA, p75NTR, ABCC3, TG2, p21) that are specifically repressed by N-Myc. Through a dual-step-ChIP assay and genetic dissection of gene promoters, we found that N-Myc is physically associated with gene promoters in vivo, in proximity of the transcription start site. N-Myc association with promoters requires interaction with other proteins, such as Sp1 and Miz1 transcription factors. Furthermore, we found that N-Myc may repress gene expression by interfering directly with Sp1 and/or with Miz1 activity (i.e. TRKA, p75NTR, ABCC3, p21) or by recruiting Histone Deacetylase 1 (Hdac1) (i.e. TG2). In vitro analyses show that distinct N-Myc domains can interact with Sp1, Miz1 and Hdac1, supporting the idea that Myc may participate in distinct repression complexes by interacting specifically with diverse proteins. Finally, results show that N-Myc, through repressed genes, affects important cellular functions, such as apoptosis, growth, differentiation and motility. Overall, our results support a model in which N-Myc, like c-Myc, can repress gene transcription by direct interaction with Sp1 and/or Miz1, and provide further lines of evidence on the importance of transcriptional repression by Myc factors in tumour biology.

Architectures and design patterns for functional design of logic control and diagnostics in industrial automation

Recently in most of the industrial automation process an ever increasing degree of automation has been observed. This increasing is motivated by the higher requirement of systems with great performance in terms of quality of products/services generated, productivity, efficiency and low costs in the design, realization and maintenance. This trend in the growth of complex automation systems is rapidly spreading over automated manufacturing systems (AMS), where the integration of the mechanical and electronic technology, typical of the Mechatronics, is merging with other technologies such as Informatics and the communication networks. An AMS is a very complex system that can be thought constituted by a set of flexible working stations, one or more transportation systems. To understand how this machine are important in our society let considerate that every day most of us use bottles of water or soda, buy product in box like food or cigarets and so on. Another important consideration from its complexity derive from the fact that the the consortium of machine producers has estimated around 350 types of manufacturing machine. A large number of manufacturing machine industry are presented in Italy and notably packaging machine industry,in particular a great concentration of this kind of industry is located in Bologna area; for this reason the Bologna area is called “packaging valley”. Usually, the various parts of the AMS interact among them in a concurrent and asynchronous way, and coordinate the parts of the machine to obtain a desiderated overall behaviour is an hard task. Often, this is the case in large scale systems, organized in a modular and distributed manner. Even if the success of a modern AMS from a functional and behavioural point of view is still to attribute to the design choices operated in the definition of the mechanical structure and electrical electronic architecture, the system that governs the control of the plant is becoming crucial, because of the large number of duties associated to it. Apart from the activity inherent to the automation of themachine cycles, the supervisory system is called to perform other main functions such as: emulating the behaviour of traditional mechanical members thus allowing a drastic constructive simplification of the machine and a crucial functional flexibility; dynamically adapting the control strategies according to the different productive needs and to the different operational scenarios; obtaining a high quality of the final product through the verification of the correctness of the processing; addressing the operator devoted to themachine to promptly and carefully take the actions devoted to establish or restore the optimal operating conditions; managing in real time information on diagnostics, as a support of the maintenance operations of the machine. The kind of facilities that designers can directly find on themarket, in terms of software component libraries provides in fact an adequate support as regard the implementation of either top-level or bottom-level functionalities, typically pertaining to the domains of user-friendly HMIs, closed-loop regulation and motion control, fieldbus-based interconnection of remote smart devices. What is still lacking is a reference framework comprising a comprehensive set of highly reusable logic control components that, focussing on the cross-cutting functionalities characterizing the automation domain, may help the designers in the process of modelling and structuring their applications according to the specific needs. Historically, the design and verification process for complex automated industrial systems is performed in empirical way, without a clear distinction between functional and technological-implementation concepts and without a systematic method to organically deal with the complete system. Traditionally, in the field of analog and digital control design and verification through formal and simulation tools have been adopted since a long time ago, at least for multivariable and/or nonlinear controllers for complex time-driven dynamics as in the fields of vehicles, aircrafts, robots, electric drives and complex power electronics equipments. Moving to the field of logic control, typical for industrial manufacturing automation, the design and verification process is approached in a completely different way, usually very “unstructured”. No clear distinction between functions and implementations, between functional architectures and technological architectures and platforms is considered. Probably this difference is due to the different “dynamical framework”of logic control with respect to analog/digital control. As a matter of facts, in logic control discrete-events dynamics replace time-driven dynamics; hence most of the formal and mathematical tools of analog/digital control cannot be directly migrated to logic control to enlighten the distinction between functions and implementations. In addition, in the common view of application technicians, logic control design is strictly connected to the adopted implementation technology (relays in the past, software nowadays), leading again to a deep confusion among functional view and technological view. In Industrial automation software engineering, concepts as modularity, encapsulation, composability and reusability are strongly emphasized and profitably realized in the so-calledobject-oriented methodologies. Industrial automation is receiving lately this approach, as testified by some IEC standards IEC 611313, IEC 61499 which have been considered in commercial products only recently. On the other hand, in the scientific and technical literature many contributions have been already proposed to establish a suitable modelling framework for industrial automation. During last years it was possible to note a considerable growth in the exploitation of innovative concepts and technologies from ICT world in industrial automation systems. For what concerns the logic control design, Model Based Design (MBD) is being imported in industrial automation from software engineering field. Another key-point in industrial automated systems is the growth of requirements in terms of availability, reliability and safety for technological systems. In other words, the control system should not only deal with the nominal behaviour, but should also deal with other important duties, such as diagnosis and faults isolations, recovery and safety management. Indeed, together with high performance, in complex systems fault occurrences increase. This is a consequence of the fact that, as it typically occurs in reliable mechatronic systems, in complex systems such as AMS, together with reliable mechanical elements, an increasing number of electronic devices are also present, that are more vulnerable by their own nature. The diagnosis problem and the faults isolation in a generic dynamical system consists in the design of an elaboration unit that, appropriately processing the inputs and outputs of the dynamical system, is also capable of detecting incipient faults on the plant devices, reconfiguring the control system so as to guarantee satisfactory performance. The designer should be able to formally verify the product, certifying that, in its final implementation, it will perform itsrequired function guarantying the desired level of reliability and safety; the next step is that of preventing faults and eventually reconfiguring the control system so that faults are tolerated. On this topic an important improvement to formal verification of logic control, fault diagnosis and fault tolerant control results derive from Discrete Event Systems theory. The aimof this work is to define a design pattern and a control architecture to help the designer of control logic in industrial automated systems. The work starts with a brief discussion on main characteristics and description of industrial automated systems on Chapter 1. In Chapter 2 a survey on the state of the software engineering paradigm applied to industrial automation is discussed. Chapter 3 presentes a architecture for industrial automated systems based on the new concept of Generalized Actuator showing its benefits, while in Chapter 4 this architecture is refined using a novel entity, the Generalized Device in order to have a better reusability and modularity of the control logic. In Chapter 5 a new approach will be present based on Discrete Event Systems for the problemof software formal verification and an active fault tolerant control architecture using online diagnostic. Finally conclusive remarks and some ideas on new directions to explore are given. In Appendix A are briefly reported some concepts and results about Discrete Event Systems which should help the reader in understanding some crucial points in chapter 5; while in Appendix B an overview on the experimental testbed of the Laboratory of Automation of University of Bologna, is reported to validated the approach presented in chapter 3, chapter 4 and chapter 5. In Appendix C some components model used in chapter 5 for formal verification are reported.

InSAR deformation measurements of the earthquake cycle in transcurrent tectonic domains, analytical and analog modeling

I applied the SBAS-DInSAR method to the Mattinata Fault (MF) (Southern Italy) and to the Doruneh Fault System (DFS) (Central Iran). In the first case, I observed limited internal deformation and determined the right lateral kinematic pattern with a compressional pattern in the northern sector of the fault. Using the Okada model I inverted the observed velocities defining a right lateral strike slip solution for the MF. Even if it fits the data within the uncertainties, the modeled slip rate of 13-15 mm yr-1 seems too high with respect to the geological record. Concerning the Western termination of DFS, SAR data confirms the main left lateral transcurrent kinematics of this fault segment, but reveal a compressional component. My analytical model fits successfully the observed data and quantifies the slip in ~4 mm yr-1 and ~2.5 mm yr-1 of pure horizontal and vertical displacement respectively. The horizontal velocity is compatible with geological record. I applied classic SAR interferometry to the October–December 2008 Balochistan (Central Pakistan) seismic swarm; I discerned the different contributions of the three Mw > 5.7 earthquakes determining fault positions, lengths, widths, depths and slip distributions, constraining the other source parameters using different Global CMT solutions. A well constrained solution has been obtained for the 09/12/2008 aftershock, whereas I tested two possible fault solutions for the 28-29/10/08 mainshocks. It is not possible to favor one of the solutions without independent constraints derived from geological data. Finally I approached the study of the earthquake-cycle in transcurrent tectonic domains using analog modeling, with alimentary gelatins like crust analog material. I successfully joined the study of finite deformation with the earthquake cycle study and sudden dislocation. A lot of seismic cycles were reproduced in which a characteristic earthquake is recognizable in terms of displacement, coseismic velocity and recurrence time.

The role of MYCN-mediated transcriptional repression in neuronal physiopathology

MYC is a transcription factor that can activate transcription of several targets by direct binding to their promoters at specific DNA sequences (E-box). Recent findings have also shown that it can exert its biological role by repressing transcription of other set of genes. C-MYC can mediate repression on its target genes through interaction with factors bound to promoter regions but not through direct recognition of typical E-Boxes. In this thesis, we investigated whether MYCN can also repress gene transcription and how this is mechanistically achieved. Moreover, expression of TRKA, P75NTR and ABCC3 is attenuated in aggressive MYCN-amplified tumors, suggesting a causal link between elevated MYCN activity and transcriptional repression of these three genes. We found that MYCN is physically associated with gene promoters in vivo in proximity of the transcriptional start sites and this association requires interactions with SP1 and/or MIZ-1. Furthermore, we show that this interaction could interfere with SP1 and MIZ-1 activation functions by recruiting co-repressors such as DNMT3a or HDACs. Studies in vitro suggest that MYCN interacts through distinct domains with SP1, MIZ-1 and HDAC1 supporting the idea that MYCN may form different complexes by interacting with different proteins. Re-expression of endogenous TRKA and P75NTR with exposure to the TSA sensitizes neuroblastoma to NGF-mediated apoptosis, whereas ectopic expression of ABCC3 decreases cell motility without interfering with growth. Finally, using shRNA whole genome library, we dissected the P75NTR repression trying to identify novel factors inside and/or outside MYCN complex for future therapeutic approaches. Overall, our results support a model in which MYCN can repress gene transcription by direct interaction with SP1 and/or MIZ-1, and provide further lines of evidence on the importance of transcriptional repression induced by Myc in tumor biology.

OPA1 isoforms and protein domains in the rescue of mitochondrial dysfunctions

Mutations in OPA1 gene have been identified in the majority of patients with Dominant Optic Atrophy (DOA), a blinding disease, and the syndromic form DOA-plus. OPA1 protein is a mitochondrial GTPase involved in various mitochondrial functions, present in humans in eight isoforms, resulting from alternative splicing and proteolytic processing. In this study we have investigated the specific role of each isoform through expression in OPA-/- MEFs, by evaluating their ability to improve the defective mitochondrial phenotypes. All isoforms were able to rescue the energetic efficiency, mitochondrial DNA (mtDNA) content and cristae integrity, but only the presence of both long and short forms could recover the mitochondrial morphology. In order to identify the OPA1 protein domains crucial for its functions, we selected and modified the isoform 1, shown to be one of the most efficient in preserving mitochondrial phenotype, to express three specific OPA1 variants, namely: one with a different N-terminus portion, one unable to generate short form owing to deletion of S1 cleavage site and one with a defective GTPase domain. We demonstrated that the simultaneous presence of the N- and C-terminus of OPA1 was essential for the mtDNA maintenance; a cleavable isoform generating s-forms was necessary to completely rescue the energetic competence and the presence of the C-terminus was sufficient to partially recover the cristae ultrastructure. Lastly, several pathogenic OPA1 mutations were inserted in MEF clones and the biochemical features investigated, to correlate the defective phenotypes with the clinical severity of patients. Our results clearly indicate that this cell model reflects very well the clinical characteristics of the patients, and therefore can be proposed as an useful tool to shed light on the pathomechanism underlying DOA.

Learning Methods and Algorithms for Semantic Text Classification across Multiple Domains

Information is nowadays a key resource: machine learning and data mining techniques have been developed to extract high-level information from great amounts of data. As most data comes in form of unstructured text in natural languages, research on text mining is currently very active and dealing with practical problems. Among these, text categorization deals with the automatic organization of large quantities of documents in priorly defined taxonomies of topic categories, possibly arranged in large hierarchies. In commonly proposed machine learning approaches, classifiers are automatically trained from pre-labeled documents: they can perform very accurate classification, but often require a consistent training set and notable computational effort. Methods for cross-domain text categorization have been proposed, allowing to leverage a set of labeled documents of one domain to classify those of another one. Most methods use advanced statistical techniques, usually involving tuning of parameters. A first contribution presented here is a method based on nearest centroid classification, where profiles of categories are generated from the known domain and then iteratively adapted to the unknown one. Despite being conceptually simple and having easily tuned parameters, this method achieves state-of-the-art accuracy in most benchmark datasets with fast running times. A second, deeper contribution involves the design of a domain-independent model to distinguish the degree and type of relatedness between arbitrary documents and topics, inferred from the different types of semantic relationships between respective representative words, identified by specific search algorithms. The application of this model is tested on both flat and hierarchical text categorization, where it potentially allows the efficient addition of new categories during classification. Results show that classification accuracy still requires improvements, but models generated from one domain are shown to be effectively able to be reused in a different one.