The evolution of learning systems:to Bayes or not to be

Bayesian algorithms pose a limit to the performance learning algorithms can achieve. Natural selection should guide the evolution of information processing systems towards those limits. What can we learn from this evolution and what properties do the intermediate stages have? While this question is too general to permit any answer, progress can be made by restricting the class of information processing systems under study. We present analytical and numerical results for the evolution of on-line algorithms for learning from examples for neural network classifiers, which might include or not a hidden layer. The analytical results are obtained by solving a variational problem to determine the learning algorithm that leads to maximum generalization ability. Simulations using evolutionary programming, for programs that implement learning algorithms, confirm and expand the results. The principal result is not just that the evolution is towards a Bayesian limit. Indeed it is essentially reached. In addition we find that evolution is driven by the discovery of useful structures or combinations of variables and operators. In different runs the temporal order of the discovery of such combinations is unique. The main result is that combinations that signal the surprise brought by an example arise always before combinations that serve to gauge the performance of the learning algorithm. This latter structures can be used to implement annealing schedules. The temporal ordering can be understood analytically as well by doing the functional optimization in restricted functional spaces. We also show that there is data suggesting that the appearance of these traits also follows the same temporal ordering in biological systems. © 2006 American Institute of Physics.

Attributed graph kernels using the Jensen-Tsallis q-differences

We propose a family of attributed graph kernels based on mutual information measures, i.e., the Jensen-Tsallis (JT) q-differences (for q  ∈ [1,2]) between probability distributions over the graphs. To this end, we first assign a probability to each vertex of the graph through a continuous-time quantum walk (CTQW). We then adopt the tree-index approach [1] to strengthen the original vertex labels, and we show how the CTQW can induce a probability distribution over these strengthened labels. We show that our JT kernel (for q  = 1) overcomes the shortcoming of discarding non-isomorphic substructures arising in the R-convolution kernels. Moreover, we prove that the proposed JT kernels generalize the Jensen-Shannon graph kernel [2] (for q = 1) and the classical subtree kernel [3] (for q = 2), respectively. Experimental evaluations demonstrate the effectiveness and efficiency of the JT kernels.

Entropy Based Approach to Finding Interacting Genes Responsible for Complex Human Disease

2000 Mathematics Subject Classification: 62P10, 92D10, 92D30, 94A17, 62L10.

Characterization of Schrödinger Processes with Unbounded Potentials

2000 Mathematics Subject Classification: 49L20, 60J60, 93E20

Regularity analysis of nocturnal oximetry recordings to assist in the diagnosis of sleep apnoea syndrome

The relationship between sleep apnoea–hypopnoea syndrome (SAHS) severity and the regularity of nocturnal oxygen saturation (SaO2) recordings was analysed. Three different methods were proposed to quantify regularity: approximate entropy (AEn), sample entropy (SEn) and kernel entropy (KEn). A total of 240 subjects suspected of suffering from SAHS took part in the study. They were randomly divided into a training set (96 subjects) and a test set (144 subjects) for the adjustment and assessment of the proposed methods, respectively. According to the measurements provided by AEn, SEn and KEn, higher irregularity of oximetry signals is associated with SAHS-positive patients. Receiver operating characteristic (ROC) and Pearson correlation analyses showed that KEn was the most reliable predictor of SAHS. It provided an area under the ROC curve of 0.91 in two-class classification of subjects as SAHS-negative or SAHS-positive. Moreover, KEn measurements from oximetry data exhibited a linear dependence on the apnoea–hypopnoea index, as shown by a correlation coefficient of 0.87. Therefore, these measurements could be used for the development of simplified diagnostic techniques in order to reduce the demand for polysomnographies. Furthermore, KEn represents a convincing alternative to AEn and SEn for the diagnostic analysis of noisy biomedical signals.

Early warnings of heart rate deterioration

Hospitals can experience difficulty in detecting and responding to early signs of patient deterioration leading to late intensive care referrals, excess mortality and morbidity, and increased hospital costs. Our study aims to explore potential indicators of physiological deterioration by the analysis of vital-signs. The dataset used comprises heart rate (HR) measurements from MIMIC II waveform database, taken from six patients admitted to the Intensive Care Unit (ICU) and diagnosed with severe sepsis. Different indicators were considered: 1) generic early warning indicators used in ecosystems analysis (autocorrelation at-1-lag (ACF1), standard deviation (SD), skewness, kurtosis and heteroskedasticity) and 2) entropy analysis (kernel entropy and multi scale entropy). Our preliminary findings suggest that when a critical transition is approaching, the equilibrium state changes what is visible in the ACF1 and SD values, but also by the analysis of the entropy. Entropy allows to characterize the complexity of the time series during the hospital stay and can be used as an indicator of regime shifts in a patient’s condition. One of the main problems is its dependency of the scale used. Our results demonstrate that different entropy scales should be used depending of the level of entropy verified.

Hybrid sampling on mutual information entropy-based clustering ensembles for optimizations

In this paper, we focus on the design of bivariate EDAs for discrete optimization problems and propose a new approach named HSMIEC. While the current EDAs require much time in the statistical learning process as the relationships among the variables are too complicated, we employ the Selfish gene theory (SG) in this approach, as well as a Mutual Information and Entropy based Cluster (MIEC) model is also set to optimize the probability distribution of the virtual population. This model uses a hybrid sampling method by considering both the clustering accuracy and clustering diversity and an incremental learning and resample scheme is also set to optimize the parameters of the correlations of the variables. Compared with several benchmark problems, our experimental results demonstrate that HSMIEC often performs better than some other EDAs, such as BMDA, COMIT, MIMIC and ECGA. © 2009 Elsevier B.V. All rights reserved.

A Kullback-Leibler relatív entrópia függvény alkalmazása páros összehasonlítás mátrix egy prioritásvektora meghatározására (Applying the Kullback-Leibler relative entropy function for determining priorities for the pairwise comparison matrix)

A dolgozatban a döntéselméletben fontos szerepet játszó páros összehasonlítás mátrix prioritásvektorának meghatározására új megközelítést alkalmazunk. Az A páros összehasonlítás mátrix és a prioritásvektor által definiált B konzisztens mátrix közötti eltérést a Kullback-Leibler relatív entrópia-függvény segítségével mérjük. Ezen eltérés minimalizálása teljesen kitöltött mátrix esetében konvex programozási feladathoz vezet, nem teljesen kitöltött mátrix esetében pedig egy fixpont problémához. Az eltérésfüggvényt minimalizáló prioritásvektor egyben azzal a tulajdonsággal is rendelkezik, hogy az A mátrix elemeinek összege és a B mátrix elemeinek összege közötti különbség éppen az eltérésfüggvény minimumának az n-szerese, ahol n a feladat mérete. Így az eltérésfüggvény minimumának értéke két szempontból is lehet alkalmas az A mátrix inkonzisztenciájának a mérésére. _____ In this paper we apply a new approach for determining a priority vector for the pairwise comparison matrix which plays an important role in Decision Theory. The divergence between the pairwise comparison matrix A and the consistent matrix B defined by the priority vector is measured with the help of the Kullback-Leibler relative entropy function. The minimization of this divergence leads to a convex program in case of a complete matrix, leads to a fixed-point problem in case of an incomplete matrix. The priority vector minimizing the divergence also has the property that the difference of the sums of elements of the matrix A and the matrix B is n times the minimum of the divergence function where n is the dimension of the problem. Thus we developed two reasons for considering the value of the minimum of the divergence as a measure of inconsistency of the matrix A.

Landscape genetics of Phaedranassa herb. (Amaryllidaceae) in Ecuador

Speciation can be understood as a continuum occurring at different levels, from population to species. The recent molecular revolution in population genetics has opened a pathway towards understanding species evolution. At the same time, speciation patterns can be better explained by incorporating a geographic context, through the use of geographic information systems (GIS). Phaedranassa (Amaryllidaceae) is a genus restricted to one of the world’s most biodiverse hotspots, the Northern Andes. I studied seven Phaedranassa species from Ecuador. Six of these species are endemic to the country. The topographic complexity of the Andes, which creates local microhabitats ranging from moist slopes to dry valleys, might explain the patterns of Phaedranassa species differentiation. With a Bayesian individual assignment approach, I assessed the genetic structure of the genus throughout Ecuador using twelve microsatellite loci. I also used bioclimatic variables and species geographic coordinates under a Maximum Entropy algorithm to generate distribution models of the species. My results show that Phaedranassa species are genetically well-differentiated. Furthermore, with the exception of two species, all Phaedranassa showed non-overlapping distributions. Phaedranassa viridiflora and P. glauciflora were the only species in which the model predicted a broad species distribution, but genetic evidence indicates that these findings are likely an artifact of species delimitation issues. Both genetic differentiation and nonoverlapping geographic distribution suggest that allopatric divergence could be the general model of genetic differentiation. Evidence of sympatric speciation was found in two geographically and genetically distinct groups of P. viridiflora. Additionally, I report the first register of natural hybridization for the genus. The findings of this research show that the genetic differentiation of species in an intricate landscape as the Andes does not necessarily show a unique trend. Although allopatric speciation is the most common form of speciation, I found evidence of sympatric speciation and hybridization. These results show that the processes of speciation in the Andes have followed several pathways. The mixture of these processes contributes to the high biodiversity of the region.

Exergoeconomic analysis of solar organic rankine cycle for geothermal air conditioned net zero energy buildings

This study is an attempt at achieving Net Zero Energy Building (NZEB) using a solar Organic Rankine Cycle (ORC) based on exergetic and economic measures. The working fluid, working conditions of the cycle, cycle configuration, and solar collector type are considered the optimization parameters for the solar ORC system. In the first section, a procedure is developed to compare ORC working fluids based on their molecular components, temperature-entropy diagram and fluid effects on the thermal efficiency, net power generated, vapor expansion ratio, and exergy efficiency of the Rankine cycle. Fluids with the best cycle performance are recognized in two different temperature levels within two different categories of fluids: refrigerants and non-refrigerants. Important factors that could lead to irreversibility reduction of the solar ORC are also investigated in this study. In the next section, the system requirements needed to maintain the electricity demand of a geothermal air-conditioned commercial building located in Pensacola of Florida is considered as the criteria to select the optimal components and optimal working condition of the system. The solar collector loop, building, and geothermal air conditioning system are modeled using TRNSYS. Available electricity bills of the building and the 3-week monitoring data on the performance of the geothermal system are employed to calibrate the simulation. The simulation is repeated for Miami and Houston in order to evaluate the effect of the different solar radiations on the system requirements. The final section discusses the exergoeconomic analysis of the ORC system with the optimum performance. Exergoeconomics rests on the philosophy that exergy is the only rational basis for assigning monetary costs to a system’s interactions with its surroundings and to the sources of thermodynamic inefficiencies within it. Exergoeconomic analysis of the optimal ORC system shows that the ratio Rex of the annual exergy loss to the capital cost can be considered a key parameter in optimizing a solar ORC system from the thermodynamic and economic point of view. It also shows that there is a systematic correlation between the exergy loss and capital cost for the investigated solar ORC system.

A study of branching fluid networks for enhancing the performance of thermal -fluid devices

Compact thermal-fluid systems are found in many industries from aerospace to microelectronics where a combination of small size, light weight, and high surface area to volume ratio fluid networks are necessary. These devices are typically designed with fluid networks consisting of many small parallel channels that effectively pack a large amount of heat transfer surface area in a very small volume but do so at the cost of increased pumping power requirements. ^ To offset this cost the use of a branching fluid network for the distribution of coolant within a heat sink is investigated. The goal of the branch design technique is to minimize the entropy generation associated with the combination of viscous dissipation and convection heat transfer experienced by the coolant in the heat sink while maintaining compact high heat transfer surface area to volume ratios. ^ The derivation of Murray's Law, originally developed to predict the geometry of physiological transport systems, is extended to heat sink designs which minimze entropy generation. Two heat sink designs at different scales are built, and tested experimentally and analytically. The first uses this new derivation of Murray's Law. The second uses a combination of Murray's Law and Constructal Theory. The results of the experiments were used to verify the analytical and numerical models. These models were then used to compare the performance of the heat sink with other compact high performance heat sink designs. The results showed that the techniques used to design branching fluid networks significantly improves the performance of active heat sinks. The design experience gained was then used to develop a set of geometric relations which optimize the heat transfer to pumping power ratio of a single cooling channel element. Each element can be connected together using a set of derived geometric guidelines which govern branch diameters and angles. The methodology can be used to design branching fluid networks which can fit any geometry. ^

Specific binding of the mammalian high mobility group protein AT-hook 2 to the minor groove of AT -rich DNAs: Thermodynamic and specificity studies

The mammalian high mobility group protein AT-hook 2 (HMGA2) is a small transcriptional factor involved in cell development and oncogenesis. It contains three "AT-hook" DNA binding domains, which specifically recognize the minor groove of AT-rich DNA sequences. It also has an acidic C-terminal motif. Previous studies showed that HMGA2 mediates all its biological effects through interactions with AT-rich DNA sequences in the promoter regions. In this dissertation, I used a variety of biochemical and biophysical methods to examine the physical properties of HMGA2 and to further investigate HMGA2's interactions with AT-rich DNA sequences. The following are three avenues perused in this study: (1) due to the asymmetrical charge distribution of HMGA2, I have developed a rapid procedure to purify HMGA2 in the milligram range. Preparation of large amounts of HMGA2 makes biophysical studies possible; (2) Since HMGA2 binds to different AT-rich sequences in the promoter regions, I used a combination of isothermal titration calorimetry (ITC) and DNA UV melting experiment to characterize interactions of HMGA2 with poly(dA-dT) 2 and poly(dA)poly(dT). My results demonstrated that (i) each HMGA2 molecule binds to 15 AT bp; (ii) HMGA2 binds to both AT DNAs with very high affinity. However, the binding reaction of HMGA2 to poly(dA-dT) 2 is enthalpy-driven and the binding reaction of HMGA2 with poly(dA)poly(dT) is entropy-driven; (iii) the binding reactions are strongly depended on salt concentrations; (3) Previous studies showed that HMGA2 may have sequence specificity. In this study, I used a PCR-based SELEX procedure to examine the DNA binding specificity of HMGA2. Two consensus sequences for HMGA2 have been identified: 5'-ATATTCGCGAWWATT-3' and 5'-ATATTGCGCAWWATT-3', where W represents A or T. These consensus sequences have a unique feature: the first five base pairs are AT-rich, the middle four to five base pairs are GC-rich, and the last five to six base pairs are AT-rich. All three segments are critical for high affinity binding. Replacing either one of the AT-rich sequences to a non-AT-rich sequence causes at least 100-fold decrease in the binding affinity. Intriguingly, if the GC-segment is substituted by an AT-rich segment, the binding affinity of HMGA2 is reduced approximately 5-fold. Identification of the consensus sequences for HMGA2 represents an important step towards finding its binding sites within the genome.

Joint-specific changes in locomotor complexity in the absence of muscle atrophy following incomplete spinal cord injury

ackground Following incomplete spinal cord injury (iSCI), descending drive is impaired, possibly leading to a decrease in the complexity of gait. To test the hypothesis that iSCI impairs gait coordination and decreases locomotor complexity, we collected 3D joint angle kinematics and muscle parameters of rats with a sham or an incomplete spinal cord injury. Methods 12 adult, female, Long-Evans rats, 6 sham and 6 mild-moderate T8 iSCI, were tested 4 weeks following injury. The Basso Beattie Bresnahan locomotor score was used to verify injury severity. Animals had reflective markers placed on the bony prominences of their limb joints and were filmed in 3D while walking on a treadmill. Joint angles and segment motion were analyzed quantitatively, and complexity of joint angle trajectory and overall gait were calculated using permutation entropy and principal component analysis, respectively. Following treadmill testing, the animals were euthanized and hindlimb muscles removed. Excised muscles were tested for mass, density, fiber length, pennation angle, and relaxed sarcomere length. Results Muscle parameters were similar between groups with no evidence of muscle atrophy. The animals showed overextension of the ankle, which was compensated for by a decreased range of motion at the knee. Left-right coordination was altered, leading to left and right knee movements that are entirely out of phase, with one joint moving while the other is stationary. Movement patterns remained symmetric. Permutation entropy measures indicated changes in complexity on a joint specific basis, with the largest changes at the ankle. No significant difference was seen using principal component analysis. Rats were able to achieve stable weight bearing locomotion at reasonable speeds on the treadmill despite these deficiencies. Conclusions Decrease in supraspinal control following iSCI causes a loss of complexity of ankle kinematics. This loss can be entirely due to loss of supraspinal control in the absence of muscle atrophy and may be quantified using permutation entropy. Joint-specific differences in kinematic complexity may be attributed to different sources of motor control. This work indicates the importance of the ankle for rehabilitation interventions following spinal cord injury.

Foundations of quantitative information flow: Channels, cascades, and the information order

Secrecy is fundamental to computer security, but real systems often cannot avoid leaking some secret information. For this reason, the past decade has seen growing interest in quantitative theories of information flow that allow us to quantify the information being leaked. Within these theories, the system is modeled as an information-theoretic channel that specifies the probability of each output, given each input. Given a prior distribution on those inputs, entropy-like measures quantify the amount of information leakage caused by the channel. ^ This thesis presents new results in the theory of min-entropy leakage. First, we study the perspective of secrecy as a resource that is gradually consumed by a system. We explore this intuition through various models of min-entropy consumption. Next, we consider several composition operators that allow smaller systems to be combined into larger systems, and explore the extent to which the leakage of a combined system is constrained by the leakage of its constituents. Most significantly, we prove upper bounds on the leakage of a cascade of two channels, where the output of the first channel is used as input to the second. In addition, we show how to decompose a channel into a cascade of channels. ^ We also establish fundamental new results about the recently-proposed g-leakage family of measures. These results further highlight the significance of channel cascading. We prove that whenever channel A is composition refined by channel B, that is, whenever A is the cascade of B and R for some channel R, the leakage of A never exceeds that of B, regardless of the prior distribution or leakage measure (Shannon leakage, guessing entropy leakage, min-entropy leakage, or g-leakage). Moreover, we show that composition refinement is a partial order if we quotient away channel structure that is redundant with respect to leakage alone. These results are strengthened by the proof that composition refinement is the only way for one channel to never leak more than another with respect to g-leakage. Therefore, composition refinement robustly answers the question of when a channel is always at least as secure as another from a leakage point of view.^

Coupled symplectic maps as models for subdiffusive processes in disordered Hamiltonian lattices

Acknowledgements One of us (T. B.) acknowledges many interesting discussions on coupled maps with Professor C. Tsallis. We are also grateful to the anonymous referees for their constructive feedback that helped us improve the manuscript and to the HPCS Laboratory of the TEI of Western Greece for providing the computer facilities where all our simulations were performed. C. G. A. was partially supported by the “EPSRC EP/I032606/1” grant of the University of Aberdeen. This research has been co-financed by the European Union (European Social Fund - ESF) and Greek national funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF) - Research Funding Program: THALES - Investing in knowledge society through the European Social Fund.