The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

Evolutionary neurobiology and art

In his important book on evolutionary theory, Darwin's Dangerous Idea, Daniel Dennett warns that Darwin's idea seeps through every area of human discourse like a "universal acid" (Dennett, 1995). Art and the aesthetic response cannot escape its influence. So my approach in this chapter is essentially naturalistic. Friedrich Nietzsche writes of observing the human comedy from afar, "like a cold angel...without anger, but without warmth" (Nietzsche, 1872, p. 164). Whether Nietzsche, of all people, could have done this is a matter of debate. But we know what he means. It describes a stance outside the human world as if looking down on human folly from Mount Olympus. From this stance, humans, their art and neurology are all part of the natural world, all part of the evolutionary process, the struggle for existence. The anthropologist David Dutton, in his contribution to the Routledge Companion to Aesthetics, says that all humans have an aesthetic sense (Dutton, 2001). It is a human universal. Biologists argue that such universals have an evolutionary basis. Furthermore, many have argued that not only humans but also animals, at least the higher mammals and birds, have an appreciation of the beautiful and the ugly (Eibl-Eibesfeldt, 1988).11Charles Darwin indeed writes "Birds appear to be the most aesthetic of all animals, excepting, of course, man, and they have nearly the same sense of the beautiful that we have" (1871, The Descent of Man and Selection in Relation to Sex, London: John Murray, vol.2, xiii, 39). This again suggests that aesthetics has an evolutionary origin. In parenthesis here, I should perhaps say that I am well aware of the criticism leveled at evolutionary psychology. I am well aware that it has been attacked as just so many "just-so" stories. This is neither the time nor the place to mount a defense but simply just to say that I believe that a defense is eminently feasible. © 2006 Elsevier Inc. All rights reserved.

The effective use of implicit parallelism through the use of an object-oriented programming language

This thesis explores translating well-written sequential programs in a subset of the Eiffel programming language - without syntactic or semantic extensions - into parallelised programs for execution on a distributed architecture. The main focus is on constructing two object-oriented models: a theoretical self-contained model of concurrency which enables a simplified second model for implementing the compiling process. There is a further presentation of principles that, if followed, maximise the potential levels of parallelism. Model of Concurrency. The concurrency model is designed to be a straightforward target for mapping sequential programs onto, thus making them parallel. It aids the compilation process by providing a high level of abstraction, including a useful model of parallel behaviour which enables easy incorporation of message interchange, locking, and synchronization of objects. Further, the model is sufficient such that a compiler can and has been practically built. Model of Compilation. The compilation-model's structure is based upon an object-oriented view of grammar descriptions and capitalises on both a recursive-descent style of processing and abstract syntax trees to perform the parsing. A composite-object view with an attribute grammar style of processing is used to extract sufficient semantic information for the parallelisation (i.e. code-generation) phase. Programming Principles. The set of principles presented are based upon information hiding, sharing and containment of objects and the dividing up of methods on the basis of a command/query division. When followed, the level of potential parallelism within the presented concurrency model is maximised. Further, these principles naturally arise from good programming practice. Summary. In summary this thesis shows that it is possible to compile well-written programs, written in a subset of Eiffel, into parallel programs without any syntactic additions or semantic alterations to Eiffel: i.e. no parallel primitives are added, and the parallel program is modelled to execute with equivalent semantics to the sequential version. If the programming principles are followed, a parallelised program achieves the maximum level of potential parallelisation within the concurrency model.

Generalized methods and solvers for noise removal from piecewise constant signals. I. Background theory

Removing noise from piecewise constant (PWC) signals is a challenging signal processing problem arising in many practical contexts. For example, in exploration geosciences, noisy drill hole records need to be separated into stratigraphic zones, and in biophysics, jumps between molecular dwell states have to be extracted from noisy fluorescence microscopy signals. Many PWC denoising methods exist, including total variation regularization, mean shift clustering, stepwise jump placement, running medians, convex clustering shrinkage and bilateral filtering; conventional linear signal processing methods are fundamentally unsuited. This paper (part I, the first of two) shows that most of these methods are associated with a special case of a generalized functional, minimized to achieve PWC denoising. The minimizer can be obtained by diverse solver algorithms, including stepwise jump placement, convex programming, finite differences, iterated running medians, least angle regression, regularization path following and coordinate descent. In the second paper, part II, we introduce novel PWC denoising methods, and comparisons between these methods performed on synthetic and real signals, showing that the new understanding of the problem gained in part I leads to new methods that have a useful role to play.

The effect of flying and low humidity on the admittance of the tympanic membrane and middle ear system

Many passengers experience discomfort during flight because of the effect of low humidity on the skin, eyes, throat, and nose. In this physiological study, we have investigated whether flight and low humidity also affect the tympanic membrane. From previous studies, a decrease in admittance of the tympanic membrane through drying might be expected to affect the buffering capacity of the middle ear and to disrupt automatic pressure regulation. This investigation involved an observational study onboard an aircraft combined with experiments in an environmental chamber, where the humidity could be controlled but could not be made to be as low as during flight. For the flight study, there was a linear relationship between the peak compensated static admittance of the tympanic membrane and relative humidity with a constant of proportionality of 0.00315 mmho/% relative humidity. The low humidity at cruise altitude (minimum 22.7 %) was associated with a mean decrease in admittance of about 20 % compared with measures in the airport. From the chamber study, we further found that a mean decrease in relative humidity of 23.4 % led to a significant decrease in mean admittance by 0.11 mmho [F(1,8) = 18.95, P = 0.002], a decrease of 9.4 %. The order of magnitude for the effect of humidity was similar for the flight and environmental chamber studies. We conclude that admittance changes during flight were likely to have been caused by the low humidity in the aircraft cabin and that these changes may affect the automatic pressure regulation of the middle ear during descent. © 2013 Association for Research in Otolaryngology.

Independent modulation of engagement and connectivity of the facial network during affect processing by CACNA1C and ANK3 risk genes for bipolar disorder

IMPORTANCE Genome-wide association studies (GWASs) indicate that single-nucleotide polymorphisms in the CACNA1C and ANK3 genes increase the risk for bipolar disorder (BD). The genes influence neuronal firing by modulating calcium and sodium channel functions, respectively. Both genes modulate ?-aminobutyric acid-transmitting interneuron function and can thus affect brain regional activation and interregional connectivity. OBJECTIVE To determine whether the genetic risk for BD associated with 2 GWAS-supported risk single-nucleotide polymorphisms at CACNA1C rs1006737 and ANK3 rs10994336 is mediated through changes in regional activation and interregional connectivity of the facial affect-processing network. DESIGN, SETTING, AND PARTICIPANTS Cross-sectional functional magnetic resonance imaging study at a research institute of 41 euthymic patients with BD and 46 healthy participants, all of British white descent. MAIN OUTCOMES AND MEASURES Blood oxygen level-dependent signal and effective connectivity measures during the facial affect-processing task. RESULTS In healthy carriers, both genetic risk variants were independently associated with increased regional engagement throughout the facial affect-processing network and increased effective connectivity between the visual and ventral prefrontal cortical regions. In contrast, BD carriers of either genetic risk variant exhibited pronounced reduction in ventral prefrontal cortical activation and visual-prefrontal effective connectivity. CONCLUSIONS AND RELEVANCE Our data demonstrate that the effect of CACNA1C rs1006737 and ANK3 rs10994336 (or genetic variants in linkage disequilibrium) on the brain converges on the neural circuitry involved in affect processing and provides a mechanism linking BD to genome-wide genetic risk variants.

The rise and fall of diasporic bonds in Japanese-Peruvian “Return” migration

Two decades after Japanese-Peruvians and other South Americans of Japanese descent began to migrate back to Japan, the return-migration phenomenon has ended. Induced by the Japanese government in the name of shared ethnicity, Japanese policymakers now largely regard return migration as a failed policy. It failed because return-migrants did not, in the view of policy- makers, assimilate, integrate, or “make it” in Japan as expected. Thus, once-imagined ethnic bonds ceased to exist in Japan. However, ethnic bonds sustained themselves well outside Japan. The Japanese-Peruvian community in Peru has thrived and maintained continuous ties with Japan. What explains the rise and fall of diasporic ethnic bonds? Drawing on my ethnographic research in Japanese-Peruvian communities in Peru and Japan, I found that diasporic ethnic bonds are cultivated or weakened depending upon where diasporic populations are located in relation to their ancestral homeland, and how such ties are utilized, for what, and by whom.

A statistical physics perspective on alignment-independent protein sequence comparison.

Motivation: Within bioinformatics, the textual alignment of amino acid sequences has long dominated the determination of similarity between proteins, with all that implies for shared structure, function, and evolutionary descent. Despite the relative success of modern-day sequence alignment algorithms, so-called alignment-free approaches offer a complementary means of determining and expressing similarity, with potential benefits in certain key applications, such as regression analysis of protein structure-function studies, where alignment-base similarity has performed poorly. Results: Here, we offer a fresh, statistical physics-based perspective focusing on the question of alignment-free comparison, in the process adapting results from “first passage probability distribution” to summarize statistics of ensemble averaged amino acid propensity values. In this paper, we introduce and elaborate this approach.

Generalized KPCA by adaptive rules in feature space

Principal component analysis (PCA) is well recognized in dimensionality reduction, and kernel PCA (KPCA) has also been proposed in statistical data analysis. However, KPCA fails to detect the nonlinear structure of data well when outliers exist. To reduce this problem, this paper presents a novel algorithm, named iterative robust KPCA (IRKPCA). IRKPCA works well in dealing with outliers, and can be carried out in an iterative manner, which makes it suitable to process incremental input data. As in the traditional robust PCA (RPCA), a binary field is employed for characterizing the outlier process, and the optimization problem is formulated as maximizing marginal distribution of a Gibbs distribution. In this paper, this optimization problem is solved by stochastic gradient descent techniques. In IRKPCA, the outlier process is in a high-dimensional feature space, and therefore kernel trick is used. IRKPCA can be regarded as a kernelized version of RPCA and a robust form of kernel Hebbian algorithm. Experimental results on synthetic data demonstrate the effectiveness of IRKPCA. © 2010 Taylor & Francis.

AltitudeOmics: Red Blood Cell metabolic adaptation to high altitude hypoxia

Red blood cells (RBCs) are key players in systemic oxygen transport. RBCs respond to in vitro hypoxia  through  the so-called  oxygen-dependent  metabolic  regulation,  which  involves  the competitive  binding  of  deoxyhemoglobin  and  glycolytic  enzymes  to  the  N-terminal  cytosolic domain  of  band  3.  This  mechanism  promotes  the  accumulation  of  2,3-DPG,  stabilizing  the deoxygenated state of hemoglobin, and cytosol acidification, triggering oxygen off-loading through the  Bohr  effect.  Despite  in  vitro  studies,  in  vivo adaptations  to  hypoxia  have  not  yet  been completely elucidated. Within  the  framework  of  the AltitudeOmics  study,  erythrocytes  were  collected  from  21 healthy volunteers at sea level, after exposure to high altitude (5260m) for 1, 7 and 16days, and following  reascent  after  7days  at 1525m.  UHPLC-MS  metabolomics  results  were  correlated  to physiological and athletic performance parameters. Immediate  metabolic  adaptations  were  noted as early as a few hours from ascending  to >5000m, and maintained for 16 days at high altitude.  Consistent with the mechanisms elucidated in vitro, hypoxia promoted glycolysis and deregulated the pentose phosphate pathway, as well purine catabolism, glutathione homeostasis, arginine/nitric oxide and sulphur/H2S metabolism. Metabolic adaptations were preserved one week after descent, consistently with improved physical performances in comparison to the first ascendance, suggesting a mechanism of metabolic memory.