Construction of Neural Network Classification Expert Systems Using Switching Theory Algorithms

A new family of neural network architectures is presented. This family of architectures solves the problem of constructing and training minimal neural network classification expert systems by using switching theory. The primary insight that leads to the use of switching theory is that the problem of minimizing the number of rules and the number of IF statements (antecedents) per rule in a neural network expert system can be recast into the problem of minimizing the number of digital gates and the number of connections between digital gates in a Very Large Scale Integrated (VLSI) circuit. The rules that the neural network generates to perform a task are readily extractable from the network's weights and topology. Analysis and simulations on the Mushroom database illustrate the system's performance.

Explorations in a meaning making system: Kerry as an entrepreneurial platform and the equine sector as a cluster

This Portfolio is about the changes that can be supported and achieved through transformational education that impacts on personal, professional and organisational levels. Having lived through an era of tremendous change over the second half of the twentieth century and into the twenty-first the author has a great drawing board to contemplate in the context of professional career experience as an engineer. The ability to engage in ‘subject-object’ separation is the means by which Kegan (1994, 2009) explains that transformation takes place and the Essays in this Portfolio aim to support and bring about such change. Exploration of aspects of ‘Kerry’ is the material selected to both challenge support change in the way of knowing from being subject to certain information and knowledge that to being able to consider it more objectively. The task of being able to distance judgement about the economy and economic development of Kerry was facilitated by various readings around of a number of key thinkers including Kegan, Drucker, Porter and Penrose. The central themes of Kerry or the potential for economic development are built into each Essay. Essay One focuses on reflections of Kerry life - on Kerry people within and without Kerry - and events as they affected understandings of how people related to and worked with one another. These reflections formed the basis for transformational goals identified which required a shift from an engineering mindset to encompass an economics-based view. In Essay Two knowledge of economic concepts is developed by exploring the writings of Drucker, Penrose, and Porter with pertinence to considering economic development generally, and for Kerry in particular in the form of an ‘entrepreneurial platform’. The concepts and theories were the basis of explorations presented in Essays Three and Four. Essay Three focuses on Kerry’s potential for economic development give its current economic profile and includes results from interviews with selected businesses. Essay Four is an exercise in the application of Porter’s ‘Cluster’ concept to the equine sector.

Combining and choosing case base maintenance algorithms

Case-Based Reasoning (CBR) uses past experiences to solve new problems. The quality of the past experiences, which are stored as cases in a case base, is a big factor in the performance of a CBR system. The system's competence may be improved by adding problems to the case base after they have been solved and their solutions verified to be correct. However, from time to time, the case base may have to be refined to reduce redundancy and to get rid of any noisy cases that may have been introduced. Many case base maintenance algorithms have been developed to delete noisy and redundant cases. However, different algorithms work well in different situations and it may be difficult for a knowledge engineer to know which one is the best to use for a particular case base. In this thesis, we investigate ways to combine algorithms to produce better deletion decisions than the decisions made by individual algorithms, and ways to choose which algorithm is best for a given case base at a given time. We analyse five of the most commonly-used maintenance algorithms in detail and show how the different algorithms perform better on different datasets. This motivates us to develop a new approach: maintenance by a committee of experts (MACE). MACE allows us to combine maintenance algorithms to produce a composite algorithm which exploits the merits of each of the algorithms that it contains. By combining different algorithms in different ways we can also define algorithms that have different trade-offs between accuracy and deletion. While MACE allows us to define an infinite number of new composite algorithms, we still face the problem of choosing which algorithm to use. To make this choice, we need to be able to identify properties of a case base that are predictive of which maintenance algorithm is best. We examine a number of measures of dataset complexity for this purpose. These provide a numerical way to describe a case base at a given time. We use the numerical description to develop a meta-case-based classification system. This system uses previous experience about which maintenance algorithm was best to use for other case bases to predict which algorithm to use for a new case base. Finally, we give the knowledge engineer more control over the deletion process by creating incremental versions of the maintenance algorithms. These incremental algorithms suggest one case at a time for deletion rather than a group of cases, which allows the knowledge engineer to decide whether or not each case in turn should be deleted or kept. We also develop incremental versions of the complexity measures, allowing us to create an incremental version of our meta-case-based classification system. Since the case base changes after each deletion, the best algorithm to use may also change. The incremental system allows us to choose which algorithm is the best to use at each point in the deletion process.

Artefact detection and removal algorithms for EEG diagnostic systems

The electroencephalogram (EEG) is a medical technology that is used in the monitoring of the brain and in the diagnosis of many neurological illnesses. Although coarse in its precision, the EEG is a non-invasive tool that requires minimal set-up times, and is suitably unobtrusive and mobile to allow continuous monitoring of the patient, either in clinical or domestic environments. Consequently, the EEG is the current tool-of-choice with which to continuously monitor the brain where temporal resolution, ease-of- use and mobility are important. Traditionally, EEG data are examined by a trained clinician who identifies neurological events of interest. However, recent advances in signal processing and machine learning techniques have allowed the automated detection of neurological events for many medical applications. In doing so, the burden of work on the clinician has been significantly reduced, improving the response time to illness, and allowing the relevant medical treatment to be administered within minutes rather than hours. However, as typical EEG signals are of the order of microvolts (μV ), contamination by signals arising from sources other than the brain is frequent. These extra-cerebral sources, known as artefacts, can significantly distort the EEG signal, making its interpretation difficult, and can dramatically disimprove automatic neurological event detection classification performance. This thesis therefore, contributes to the further improvement of auto- mated neurological event detection systems, by identifying some of the major obstacles in deploying these EEG systems in ambulatory and clinical environments so that the EEG technologies can emerge from the laboratory towards real-world settings, where they can have a real-impact on the lives of patients. In this context, the thesis tackles three major problems in EEG monitoring, namely: (i) the problem of head-movement artefacts in ambulatory EEG, (ii) the high numbers of false detections in state-of-the-art, automated, epileptiform activity detection systems and (iii) false detections in state-of-the-art, automated neonatal seizure detection systems. To accomplish this, the thesis employs a wide range of statistical, signal processing and machine learning techniques drawn from mathematics, engineering and computer science. The first body of work outlined in this thesis proposes a system to automatically detect head-movement artefacts in ambulatory EEG and utilises supervised machine learning classifiers to do so. The resulting head-movement artefact detection system is the first of its kind and offers accurate detection of head-movement artefacts in ambulatory EEG. Subsequently, addtional physiological signals, in the form of gyroscopes, are used to detect head-movements and in doing so, bring additional information to the head- movement artefact detection task. A framework for combining EEG and gyroscope signals is then developed, offering improved head-movement arte- fact detection. The artefact detection methods developed for ambulatory EEG are subsequently adapted for use in an automated epileptiform activity detection system. Information from support vector machines classifiers used to detect epileptiform activity is fused with information from artefact-specific detection classifiers in order to significantly reduce the number of false detections in the epileptiform activity detection system. By this means, epileptiform activity detection which compares favourably with other state-of-the-art systems is achieved. Finally, the problem of false detections in automated neonatal seizure detection is approached in an alternative manner; blind source separation techniques, complimented with information from additional physiological signals are used to remove respiration artefact from the EEG. In utilising these methods, some encouraging advances have been made in detecting and removing respiration artefacts from the neonatal EEG, and in doing so, the performance of the underlying diagnostic technology is improved, bringing its deployment in the real-world, clinical domain one step closer.

Profiling side-channel attacks on cryptographic algorithms

Traditionally, attacks on cryptographic algorithms looked for mathematical weaknesses in the underlying structure of a cipher. Side-channel attacks, however, look to extract secret key information based on the leakage from the device on which the cipher is implemented, be it smart-card, microprocessor, dedicated hardware or personal computer. Attacks based on the power consumption, electromagnetic emanations and execution time have all been practically demonstrated on a range of devices to reveal partial secret-key information from which the full key can be reconstructed. The focus of this thesis is power analysis, more specifically a class of attacks known as profiling attacks. These attacks assume a potential attacker has access to, or can control, an identical device to that which is under attack, which allows him to profile the power consumption of operations or data flow during encryption. This assumes a stronger adversary than traditional non-profiling attacks such as differential or correlation power analysis, however the ability to model a device allows templates to be used post-profiling to extract key information from many different target devices using the power consumption of very few encryptions. This allows an adversary to overcome protocols intended to prevent secret key recovery by restricting the number of available traces. In this thesis a detailed investigation of template attacks is conducted, along with how the selection of various attack parameters practically affect the efficiency of the secret key recovery, as well as examining the underlying assumption of profiling attacks in that the power consumption of one device can be used to extract secret keys from another. Trace only attacks, where the corresponding plaintext or ciphertext data is unavailable, are then investigated against both symmetric and asymmetric algorithms with the goal of key recovery from a single trace. This allows an adversary to bypass many of the currently proposed countermeasures, particularly in the asymmetric domain. An investigation into machine-learning methods for side-channel analysis as an alternative to template or stochastic methods is also conducted, with support vector machines, logistic regression and neural networks investigated from a side-channel viewpoint. Both binary and multi-class classification attack scenarios are examined in order to explore the relative strengths of each algorithm. Finally these machine-learning based alternatives are empirically compared with template attacks, with their respective merits examined with regards to attack efficiency.

Algorithms for indoor localization based on IEEE 802.15.4-2011 UWB and inertial sensors

In this thesis, extensive experiments are firstly conducted to characterize the performance of using the emerging IEEE 802.15.4-2011 ultra wideband (UWB) for indoor localization, and the results demonstrate the accuracy and precision of using time of arrival measurements for ranging applications. A multipath propagation controlling technique is synthesized which considers the relationship between transmit power, transmission range and signal-to-noise ratio. The methodology includes a novel bilateral transmitter output power control algorithm which is demonstrated to be able to stabilize the multipath channel, and enable sub 5cm instant ranging accuracy in line of sight conditions. A fully-coupled architecture is proposed for the localization system using a combination of IEEE 802.15.4-2011 UWB and inertial sensors. This architecture not only implements the position estimation of the object by fusing the UWB and inertial measurements, but enables the nodes in the localization network to mutually share positional and other useful information via the UWB channel. The hybrid system has been demonstrated to be capable of simultaneous local-positioning and remote-tracking of the mobile object. Three fusion algorithms for relative position estimation are proposed, including internal navigation system (INS), INS with UWB ranging correction, and orientation plus ranging. Experimental results show that the INS with UWB correction algorithm achieves an average position accuracy of 0.1883m, and gets 83% and 62% improvements on the accuracy of the INS (1.0994m) and the existing extended Kalman filter tracking algorithm (0.5m), respectively.

Expression signatures of TP53 mutations in serous ovarian cancers.

BACKGROUND: Mutations in the TP53 gene are extremely common and occur very early in the progression of serous ovarian cancers. Gene expression patterns that relate to mutational status may provide insight into the etiology and biology of the disease. METHODS: The TP53 coding region was sequenced in 89 frozen serous ovarian cancers, 40 early stage (I/II) and 49 advanced stage (III/IV). Affymetrix U133A expression data was used to define gene expression patterns by mutation, type of mutation, and cancer stage. RESULTS: Missense or chain terminating (null) mutations in TP53 were found in 59/89 (66%) ovarian cancers. Early stage cancers had a significantly higher rate of null mutations than late stage disease (38% vs. 8%, p < 0.03). In advanced stage cases, mutations were more prevalent in short term survivors than long term survivors (81% vs. 30%, p = 0.0004). Gene expression patterns had a robust ability to predict TP53 status within training data. By using early versus late stage disease for out of sample predictions, the signature derived from early stage cancers could accurately (86%) predict mutation status of late stage cancers. CONCLUSIONS: This represents the first attempt to define a genomic signature of TP53 mutation in ovarian cancer. Patterns of gene expression characteristic of TP53 mutation could be discerned and included several genes that are known p53 targets or have been described in the context of expression signatures of TP53 mutation in breast cancer.

Study protocol: the Adherence and Intensification of Medications (AIM) study--a cluster randomized controlled effectiveness study.

BACKGROUND: Many patients with diabetes have poor blood pressure (BP) control. Pharmacological therapy is the cornerstone of effective BP treatment, yet there are high rates both of poor medication adherence and failure to intensify medications. Successful medication management requires an effective partnership between providers who initiate and increase doses of effective medications and patients who adhere to the regimen. METHODS: In this cluster-randomized controlled effectiveness study, primary care teams within sites were randomized to a program led by a clinical pharmacist trained in motivational interviewing-based behavioral counseling approaches and authorized to make BP medication changes or to usual care. This study involved the collection of data during a 14-month intervention period in three Department of Veterans Affairs facilities and two Kaiser Permanente Northern California facilities. The clinical pharmacist was supported by clinical information systems that enabled proactive identification of, and outreach to, eligible patients identified on the basis of poor BP control and either medication refill gaps or lack of recent medication intensification. The primary outcome is the relative change in systolic blood pressure (SBP) measurements over time. Secondary outcomes are changes in Hemoglobin A1c, low-density lipoprotein cholesterol (LDL), medication adherence determined from pharmacy refill data, and medication intensification rates. DISCUSSION: Integration of the three intervention elements--proactive identification, adherence counseling and medication intensification--is essential to achieve optimal levels of control for high-risk patients. Testing the effectiveness of this intervention at the team level allows us to study the program as it would typically be implemented within a clinic setting, including how it integrates with other elements of care. TRIAL REGISTRATION: The ClinicalTrials.gov registration number is NCT00495794.

Genetic signatures in the envelope glycoproteins of HIV-1 that associate with broadly neutralizing antibodies.

A steady increase in knowledge of the molecular and antigenic structure of the gp120 and gp41 HIV-1 envelope glycoproteins (Env) is yielding important new insights for vaccine design, but it has been difficult to translate this information to an immunogen that elicits broadly neutralizing antibodies. To help bridge this gap, we used phylogenetically corrected statistical methods to identify amino acid signature patterns in Envs derived from people who have made potently neutralizing antibodies, with the hypothesis that these Envs may share common features that would be useful for incorporation in a vaccine immunogen. Before attempting this, essentially as a control, we explored the utility of our computational methods for defining signatures of complex neutralization phenotypes by analyzing Env sequences from 251 clonal viruses that were differentially sensitive to neutralization by the well-characterized gp120-specific monoclonal antibody, b12. We identified ten b12-neutralization signatures, including seven either in the b12-binding surface of gp120 or in the V2 region of gp120 that have been previously shown to impact b12 sensitivity. A simple algorithm based on the b12 signature pattern was predictive of b12 sensitivity/resistance in an additional blinded panel of 57 viruses. Upon obtaining these reassuring outcomes, we went on to apply these same computational methods to define signature patterns in Env from HIV-1 infected individuals who had potent, broadly neutralizing responses. We analyzed a checkerboard-style neutralization dataset with sera from 69 HIV-1-infected individuals tested against a panel of 25 different Envs. Distinct clusters of sera with high and low neutralization potencies were identified. Six signature positions in Env sequences obtained from the 69 samples were found to be strongly associated with either the high or low potency responses. Five sites were in the CD4-induced coreceptor binding site of gp120, suggesting an important role for this region in the elicitation of broadly neutralizing antibody responses against HIV-1.

Novel Computational Protein Design Algorithms with Applications to Cystic Fibrosis and HIV

Proteins are essential components of cells and are crucial for catalyzing reactions, signaling, recognition, motility, recycling, and structural stability. This diversity of function suggests that nature is only scratching the surface of protein functional space. Protein function is determined by structure, which in turn is determined predominantly by amino acid sequence. Protein design aims to explore protein sequence and conformational space to design novel proteins with new or improved function. The vast number of possible protein sequences makes exploring the space a challenging problem.

Computational structure-based protein design (CSPD) allows for the rational design of proteins. Because of the large search space, CSPD methods must balance search accuracy and modeling simplifications. We have developed algorithms that allow for the accurate and efficient search of protein conformational space. Specifically, we focus on algorithms that maintain provability, account for protein flexibility, and use ensemble-based rankings. We present several novel algorithms for incorporating improved flexibility into CSPD with continuous rotamers. We applied these algorithms to two biomedically important design problems. We designed peptide inhibitors of the cystic fibrosis agonist CAL that were able to restore function of the vital cystic fibrosis protein CFTR. We also designed improved HIV antibodies and nanobodies to combat HIV infections.

Night-time neuronal activation of Cluster N in a day- and night-migrating songbird.

Magnetic compass orientation in a night-migratory songbird requires that Cluster N, a cluster of forebrain regions, is functional. Cluster N, which receives input from the eyes via the thalamofugal pathway, shows high neuronal activity in night-migrants performing magnetic compass-guided behaviour at night, whereas no activation is observed during the day, and covering up the birds' eyes strongly reduces neuronal activation. These findings suggest that Cluster N processes light-dependent magnetic compass information in night-migrating songbirds. The aim of this study was to test if Cluster N is active during daytime migration. We used behavioural molecular mapping based on ZENK activation to investigate if Cluster N is active in the meadow pipit (Anthus pratensis), a day- and night-migratory species. We found that Cluster N of meadow pipits shows high neuronal activity under dim-light at night, but not under full room-light conditions during the day. These data suggest that, in day- and night-migratory meadow pipits, the light-dependent magnetic compass, which requires an active Cluster N, may only be used during night-time, whereas another magnetosensory mechanism and/or other reference system(s), like the sun or polarized light, may be used as primary orientation cues during the day.

Design of Scheduling Algorithms Using Game Theoretic Ideas

Scheduling a set of jobs over a collection of machines to optimize a certain quality-of-service measure is one of the most important research topics in both computer science theory and practice. In this thesis, we design algorithms that optimize {\em flow-time} (or delay) of jobs for scheduling problems that arise in a wide range of applications. We consider the classical model of unrelated machine scheduling and resolve several long standing open problems; we introduce new models that capture the novel algorithmic challenges in scheduling jobs in data centers or large clusters; we study the effect of selfish behavior in distributed and decentralized environments; we design algorithms that strive to balance the energy consumption and performance.

The technically interesting aspect of our work is the surprising connections we establish between approximation and online algorithms, economics, game theory, and queuing theory. It is the interplay of ideas from these different areas that lies at the heart of most of the algorithms presented in this thesis.

The main contributions of the thesis can be placed in one of the following categories.

1. Classical Unrelated Machine Scheduling: We give the first polygorithmic approximation algorithms for minimizing the average flow-time and minimizing the maximum flow-time in the offline setting. In the online and non-clairvoyant setting, we design the first non-clairvoyant algorithm for minimizing the weighted flow-time in the resource augmentation model. Our work introduces iterated rounding technique for the offline flow-time optimization, and gives the first framework to analyze non-clairvoyant algorithms for unrelated machines.

2. Polytope Scheduling Problem: To capture the multidimensional nature of the scheduling problems that arise in practice, we introduce Polytope Scheduling Problem (\psp). The \psp problem generalizes almost all classical scheduling models, and also captures hitherto unstudied scheduling problems such as routing multi-commodity flows, routing multicast (video-on-demand) trees, and multi-dimensional resource allocation. We design several competitive algorithms for the \psp problem and its variants for the objectives of minimizing the flow-time and completion time. Our work establishes many interesting connections between scheduling and market equilibrium concepts, fairness and non-clairvoyant scheduling, and queuing theoretic notion of stability and resource augmentation analysis.

3. Energy Efficient Scheduling: We give the first non-clairvoyant algorithm for minimizing the total flow-time + energy in the online and resource augmentation model for the most general setting of unrelated machines.

4. Selfish Scheduling: We study the effect of selfish behavior in scheduling and routing problems. We define a fairness index for scheduling policies called {\em bounded stretch}, and show that for the objective of minimizing the average (weighted) completion time, policies with small stretch lead to equilibrium outcomes with small price of anarchy. Our work gives the first linear/ convex programming duality based framework to bound the price of anarchy for general equilibrium concepts such as coarse correlated equilibrium.

Comparative analyses of seven algorithms for copy number variant identification from single nucleotide polymorphism arrays.

Determination of copy number variants (CNVs) inferred in genome wide single nucleotide polymorphism arrays has shown increasing utility in genetic variant disease associations. Several CNV detection methods are available, but differences in CNV call thresholds and characteristics exist. We evaluated the relative performance of seven methods: circular binary segmentation, CNVFinder, cnvPartition, gain and loss of DNA, Nexus algorithms, PennCNV and QuantiSNP. Tested data included real and simulated Illumina HumHap 550 data from the Singapore cohort study of the risk factors for Myopia (SCORM) and simulated data from Affymetrix 6.0 and platform-independent distributions. The normalized singleton ratio (NSR) is proposed as a metric for parameter optimization before enacting full analysis. We used 10 SCORM samples for optimizing parameter settings for each method and then evaluated method performance at optimal parameters using 100 SCORM samples. The statistical power, false positive rates, and receiver operating characteristic (ROC) curve residuals were evaluated by simulation studies. Optimal parameters, as determined by NSR and ROC curve residuals, were consistent across datasets. QuantiSNP outperformed other methods based on ROC curve residuals over most datasets. Nexus Rank and SNPRank have low specificity and high power. Nexus Rank calls oversized CNVs. PennCNV detects one of the fewest numbers of CNVs.

Chemically tagging the Hyades stream: does it partly originate from the Hyades cluster?

The Hyades stream has long been thought to be a dispersed vestige of the Hyades cluster. However, recent analyses of the parallax distribution, of the mass function, and of the action-space distribution of stream stars have shown it to be rather composed of orbits trapped at a resonance of a density disturbance. This resonant scenario should leave a clearly different signature in the element abundances of stream stars than the dispersed cluster scenario, since the Hyades cluster is chemically homogeneous. Here, we study the metallicity as well as the element abundances of Li, Na, Mg, Fe, Zr, Ba, La, Ce, Nd and Eu for a random sample of stars belonging to the Hyades stream, and compare them with those of stars from the Hyades cluster. From this analysis: (i) we independently confirm that the Hyades stream cannot be solely composed of stars originating in the Hyades cluster; (ii) we show thatsomestars (namely 2/21) from the Hyades stream nevertheless have abundances compatible with an origin in the cluster; (iii) we emphasize that the use of Li as a chemical tag of the cluster origin of main-sequence stars is very efficient in the range 5500K ≤Teff≤ 6200K, since the Li sequence in the Hyades cluster is very tight, while at the same time spanning a large abundance range; (iv) we show that, while this evaporated population has a metallicity excess of ~0.2 dex with respect to the local thin-disc population, identical to that of the Hyades cluster, the remainder of the Hyades stream population has still a metallicity excess of ~0.06-0.15 dex, consistent with an origin in the inner Galaxy and (v) we show that the Hyades stream can be interpreted as an inner 4:1 resonance of the spiral pattern: this then also reproduces an orbital family compatible with the Sirius stream, and places the origin of the Hyades stream up to 1kpc inwards from the solar radius, which might explain the observed metallicity excess of the stream population. © 2011 The Authors Monthly Notices of the Royal Astronomical Society © 2011 RAS.

A New Approach to Media Cluster Research: Bringing the Human Factor back into Scope

This paper is part of a collaborative project being undertaken by the three leading universities of Brussels, VUB, ULB and USL-B supported by Innnoviris. The project called Media Clusters Brussels - MCB - started in October 2014 with the goal to analyze the development of a Media Park around the two public broadcasters at the site of Reyers in Brussels being host of a media cluster in the capital city. Not only policymakers but also many authors recognized in the last decade that the media industry is characterized from a geographical point of view by a heavy concentration to a limited number of large cities, where media clusters have emerged (Karlsson & Picard, 2011). The common assumption about media clusters is that locating inside a regional agglomeration of related actors brings advantages for these firms. Especially, the interrelations and interactions between the actors on a social level matter for the shape and efficiency of the agglomerations (Picard, 2008). However, even though the importance of the actors and their interrelations has been a common assumption, many authors solely focus on the macro-economical aspects of the clusters. Within this paper, we propose to realize a socio-economical analysis of media clusters to make informed decisions in the development and so, bring the social (human) factor back into scope. Therefore, this article focuses on the development of a novel valuable framework, the so-called 7P framework with a multilevel and interdisciplinary approach, which includes three aspects, which have been identified as emerging success-factors of media clusters: partnerships, (media) professionals and positive spillovers.