The theory of on-line learning: a statistical physics approach

In this paper we review recent theoretical approaches for analysing the dynamics of on-line learning in multilayer neural networks using methods adopted from statistical physics. The analysis is based on monitoring a set of macroscopic variables from which the generalisation error can be calculated. A closed set of dynamical equations for the macroscopic variables is derived analytically and solved numerically. The theoretical framework is then employed for defining optimal learning parameters and for analysing the incorporation of second order information into the learning process using natural gradient descent and matrix-momentum based methods. We will also briefly explain an extension of the original framework for analysing the case where training examples are sampled with repetition.

Evolutionary neurobiology and art

In his important book on evolutionary theory, Darwin's Dangerous Idea, Daniel Dennett warns that Darwin's idea seeps through every area of human discourse like a "universal acid" (Dennett, 1995). Art and the aesthetic response cannot escape its influence. So my approach in this chapter is essentially naturalistic. Friedrich Nietzsche writes of observing the human comedy from afar, "like a cold angel...without anger, but without warmth" (Nietzsche, 1872, p. 164). Whether Nietzsche, of all people, could have done this is a matter of debate. But we know what he means. It describes a stance outside the human world as if looking down on human folly from Mount Olympus. From this stance, humans, their art and neurology are all part of the natural world, all part of the evolutionary process, the struggle for existence. The anthropologist David Dutton, in his contribution to the Routledge Companion to Aesthetics, says that all humans have an aesthetic sense (Dutton, 2001). It is a human universal. Biologists argue that such universals have an evolutionary basis. Furthermore, many have argued that not only humans but also animals, at least the higher mammals and birds, have an appreciation of the beautiful and the ugly (Eibl-Eibesfeldt, 1988).11Charles Darwin indeed writes "Birds appear to be the most aesthetic of all animals, excepting, of course, man, and they have nearly the same sense of the beautiful that we have" (1871, The Descent of Man and Selection in Relation to Sex, London: John Murray, vol.2, xiii, 39). This again suggests that aesthetics has an evolutionary origin. In parenthesis here, I should perhaps say that I am well aware of the criticism leveled at evolutionary psychology. I am well aware that it has been attacked as just so many "just-so" stories. This is neither the time nor the place to mount a defense but simply just to say that I believe that a defense is eminently feasible. © 2006 Elsevier Inc. All rights reserved.

Mach edges: local features predicted by 3rd derivative spatial filtering

Edges are key points of information in visual scenes. One important class of models supposes that edges correspond to the steepest parts of the luminance profile, implying that they can be found as peaks and troughs in the response of a gradient (1st derivative) filter, or as zero-crossings in the 2nd derivative (ZCs). We tested those ideas using a stimulus that has no local peaks of gradient and no ZCs, at any scale. The stimulus profile is analogous to the Mach ramp, but it is the luminance gradient (not the absolute luminance) that increases as a linear ramp between two plateaux; the luminance profile is a blurred triangle-wave. For all image-blurs tested, observers marked edges at or close to the corner points in the gradient profile, even though these were not gradient maxima. These Mach edges correspond to peaks and troughs in the 3rd derivative. Thus Mach edges are inconsistent with many standard edge-detection schemes, but are nicely predicted by a recent model that finds edge points with a 2-stage sequence of 1st then 2nd derivative operators, each followed by a half-wave rectifier.

Mach edges: a key role for 3rd derivative filters in spatial vision

Edges are key points of information in visual scenes. One important class of models supposes that edges correspond to the steepest parts of the luminance profile, implying that they can be found as peaks and troughs in the response of a gradient (first-derivative) filter, or as zero-crossings (ZCs) in the second-derivative. A variety of multi-scale models are based on this idea. We tested this approach by devising a stimulus that has no local peaks of gradient and no ZCs, at any scale. Our stimulus profile is analogous to the classic Mach-band stimulus, but it is the local luminance gradient (not the absolute luminance) that increases as a linear ramp between two plateaux. The luminance profile is a smoothed triangle wave and is obtained by integrating the gradient profile. Subjects used a cursor to mark the position and polarity of perceived edges. For all the ramp-widths tested, observers marked edges at or close to the corner points in the gradient profile, even though these were not gradient maxima. These new Mach edges correspond to peaks and troughs in the third-derivative. They are analogous to Mach bands - light and dark bars are seen where there are no luminance peaks but there are peaks in the second derivative. Here, peaks in the third derivative were seen as light-to-dark edges, troughs as dark-to-light edges. Thus Mach edges are inconsistent with many standard edge detectors, but are nicely predicted by a new model that uses a (nonlinear) third-derivative operator to find edge points.

The effective use of implicit parallelism through the use of an object-oriented programming language

This thesis explores translating well-written sequential programs in a subset of the Eiffel programming language - without syntactic or semantic extensions - into parallelised programs for execution on a distributed architecture. The main focus is on constructing two object-oriented models: a theoretical self-contained model of concurrency which enables a simplified second model for implementing the compiling process. There is a further presentation of principles that, if followed, maximise the potential levels of parallelism. Model of Concurrency. The concurrency model is designed to be a straightforward target for mapping sequential programs onto, thus making them parallel. It aids the compilation process by providing a high level of abstraction, including a useful model of parallel behaviour which enables easy incorporation of message interchange, locking, and synchronization of objects. Further, the model is sufficient such that a compiler can and has been practically built. Model of Compilation. The compilation-model's structure is based upon an object-oriented view of grammar descriptions and capitalises on both a recursive-descent style of processing and abstract syntax trees to perform the parsing. A composite-object view with an attribute grammar style of processing is used to extract sufficient semantic information for the parallelisation (i.e. code-generation) phase. Programming Principles. The set of principles presented are based upon information hiding, sharing and containment of objects and the dividing up of methods on the basis of a command/query division. When followed, the level of potential parallelism within the presented concurrency model is maximised. Further, these principles naturally arise from good programming practice. Summary. In summary this thesis shows that it is possible to compile well-written programs, written in a subset of Eiffel, into parallel programs without any syntactic additions or semantic alterations to Eiffel: i.e. no parallel primitives are added, and the parallel program is modelled to execute with equivalent semantics to the sequential version. If the programming principles are followed, a parallelised program achieves the maximum level of potential parallelisation within the concurrency model.

Generalized methods and solvers for noise removal from piecewise constant signals. I. Background theory

Removing noise from piecewise constant (PWC) signals is a challenging signal processing problem arising in many practical contexts. For example, in exploration geosciences, noisy drill hole records need to be separated into stratigraphic zones, and in biophysics, jumps between molecular dwell states have to be extracted from noisy fluorescence microscopy signals. Many PWC denoising methods exist, including total variation regularization, mean shift clustering, stepwise jump placement, running medians, convex clustering shrinkage and bilateral filtering; conventional linear signal processing methods are fundamentally unsuited. This paper (part I, the first of two) shows that most of these methods are associated with a special case of a generalized functional, minimized to achieve PWC denoising. The minimizer can be obtained by diverse solver algorithms, including stepwise jump placement, convex programming, finite differences, iterated running medians, least angle regression, regularization path following and coordinate descent. In the second paper, part II, we introduce novel PWC denoising methods, and comparisons between these methods performed on synthetic and real signals, showing that the new understanding of the problem gained in part I leads to new methods that have a useful role to play.

The effect of flying and low humidity on the admittance of the tympanic membrane and middle ear system

Many passengers experience discomfort during flight because of the effect of low humidity on the skin, eyes, throat, and nose. In this physiological study, we have investigated whether flight and low humidity also affect the tympanic membrane. From previous studies, a decrease in admittance of the tympanic membrane through drying might be expected to affect the buffering capacity of the middle ear and to disrupt automatic pressure regulation. This investigation involved an observational study onboard an aircraft combined with experiments in an environmental chamber, where the humidity could be controlled but could not be made to be as low as during flight. For the flight study, there was a linear relationship between the peak compensated static admittance of the tympanic membrane and relative humidity with a constant of proportionality of 0.00315 mmho/% relative humidity. The low humidity at cruise altitude (minimum 22.7 %) was associated with a mean decrease in admittance of about 20 % compared with measures in the airport. From the chamber study, we further found that a mean decrease in relative humidity of 23.4 % led to a significant decrease in mean admittance by 0.11 mmho [F(1,8) = 18.95, P = 0.002], a decrease of 9.4 %. The order of magnitude for the effect of humidity was similar for the flight and environmental chamber studies. We conclude that admittance changes during flight were likely to have been caused by the low humidity in the aircraft cabin and that these changes may affect the automatic pressure regulation of the middle ear during descent. © 2013 Association for Research in Otolaryngology.

Independent modulation of engagement and connectivity of the facial network during affect processing by CACNA1C and ANK3 risk genes for bipolar disorder

IMPORTANCE Genome-wide association studies (GWASs) indicate that single-nucleotide polymorphisms in the CACNA1C and ANK3 genes increase the risk for bipolar disorder (BD). The genes influence neuronal firing by modulating calcium and sodium channel functions, respectively. Both genes modulate ?-aminobutyric acid-transmitting interneuron function and can thus affect brain regional activation and interregional connectivity. OBJECTIVE To determine whether the genetic risk for BD associated with 2 GWAS-supported risk single-nucleotide polymorphisms at CACNA1C rs1006737 and ANK3 rs10994336 is mediated through changes in regional activation and interregional connectivity of the facial affect-processing network. DESIGN, SETTING, AND PARTICIPANTS Cross-sectional functional magnetic resonance imaging study at a research institute of 41 euthymic patients with BD and 46 healthy participants, all of British white descent. MAIN OUTCOMES AND MEASURES Blood oxygen level-dependent signal and effective connectivity measures during the facial affect-processing task. RESULTS In healthy carriers, both genetic risk variants were independently associated with increased regional engagement throughout the facial affect-processing network and increased effective connectivity between the visual and ventral prefrontal cortical regions. In contrast, BD carriers of either genetic risk variant exhibited pronounced reduction in ventral prefrontal cortical activation and visual-prefrontal effective connectivity. CONCLUSIONS AND RELEVANCE Our data demonstrate that the effect of CACNA1C rs1006737 and ANK3 rs10994336 (or genetic variants in linkage disequilibrium) on the brain converges on the neural circuitry involved in affect processing and provides a mechanism linking BD to genome-wide genetic risk variants.

The rise and fall of diasporic bonds in Japanese-Peruvian “Return” migration

Two decades after Japanese-Peruvians and other South Americans of Japanese descent began to migrate back to Japan, the return-migration phenomenon has ended. Induced by the Japanese government in the name of shared ethnicity, Japanese policymakers now largely regard return migration as a failed policy. It failed because return-migrants did not, in the view of policy- makers, assimilate, integrate, or “make it” in Japan as expected. Thus, once-imagined ethnic bonds ceased to exist in Japan. However, ethnic bonds sustained themselves well outside Japan. The Japanese-Peruvian community in Peru has thrived and maintained continuous ties with Japan. What explains the rise and fall of diasporic ethnic bonds? Drawing on my ethnographic research in Japanese-Peruvian communities in Peru and Japan, I found that diasporic ethnic bonds are cultivated or weakened depending upon where diasporic populations are located in relation to their ancestral homeland, and how such ties are utilized, for what, and by whom.

A statistical physics perspective on alignment-independent protein sequence comparison.

Motivation: Within bioinformatics, the textual alignment of amino acid sequences has long dominated the determination of similarity between proteins, with all that implies for shared structure, function, and evolutionary descent. Despite the relative success of modern-day sequence alignment algorithms, so-called alignment-free approaches offer a complementary means of determining and expressing similarity, with potential benefits in certain key applications, such as regression analysis of protein structure-function studies, where alignment-base similarity has performed poorly. Results: Here, we offer a fresh, statistical physics-based perspective focusing on the question of alignment-free comparison, in the process adapting results from “first passage probability distribution” to summarize statistics of ensemble averaged amino acid propensity values. In this paper, we introduce and elaborate this approach.

Description Reduction for Restricted Sets of (0,1) Matrices

* The research is supported partly by INTAS: 04-77-7173 project, http://www.intas.be

A PVT-Type Algorithm for Minimizing a Nonsmooth Convex Function

2000 Mathematics Subject Classification: 90C25, 68W10, 49M37.

Generalized KPCA by adaptive rules in feature space

Principal component analysis (PCA) is well recognized in dimensionality reduction, and kernel PCA (KPCA) has also been proposed in statistical data analysis. However, KPCA fails to detect the nonlinear structure of data well when outliers exist. To reduce this problem, this paper presents a novel algorithm, named iterative robust KPCA (IRKPCA). IRKPCA works well in dealing with outliers, and can be carried out in an iterative manner, which makes it suitable to process incremental input data. As in the traditional robust PCA (RPCA), a binary field is employed for characterizing the outlier process, and the optimization problem is formulated as maximizing marginal distribution of a Gibbs distribution. In this paper, this optimization problem is solved by stochastic gradient descent techniques. In IRKPCA, the outlier process is in a high-dimensional feature space, and therefore kernel trick is used. IRKPCA can be regarded as a kernelized version of RPCA and a robust form of kernel Hebbian algorithm. Experimental results on synthetic data demonstrate the effectiveness of IRKPCA. © 2010 Taylor & Francis.

La kabala dentro del discurso contemporaneo y urbano: El ejemplo de Marcos Ricardo Barnatan

This dissertation analyzes the relationship between several mystic Sephardic texts called the Kabalah which include the three basic mystic books of Judaism: Sefer Yetsira, Sefer Bahir, and Sefer Zohar, and Argentine writer of Jewish descent, Marcos Ricardo Barnatan’s experimental work. The premise is that several mystic Sephardic texts have certain literary characteristics that make them fertile sources for the inspiration of writers like Barnatan. This thesis proves that Barnatan’s poetry and his first novel El Laberinto De Sion both use the most general concepts of the Kabalah in developing his literary and artistic creations. ^ This study is focuses on the concept that there exists a possibility of reading kabalistic texts not only in an exegetical way, but also in a poetic way. Barnatan’s literature is prone to this kind of reading of the texts. He creates a surprisingly expressive structure without ties to established models. This expressive structure is built on a vast amount of symbols, which results from this freedom. Barnatan adopts multiple symbols from cosmogenic theories and makes use of them, thus incorporating both the spirit and style of kabalistic texts into his own work. ^ Instead of addressing some of the main concerns of Kabbalistic study and its commentary, Barnatan avoids the concern for Law, the study of the Torah and its commentaries, while avoiding the use of the Hebrew language. For this reason Barnatan is too deviant to be considered an extension from the mainstream Kabbalistic commentary. Barnatan’s work is destined for a reader who cannot only understand his experimental methods, but who can also assemble a disjointed text while accepting a fundamental instability of space and time.^

Ecology and distribution of diatoms in Biscayne Bay, Florida (USA): Implications for bioassessment and paleoenvironmental studies

The spatial and temporal distribution of planktonic, sediment-associated and epiphytic diatoms among 58 sites in Biscayne Bay, Florida was examined in order to identify diatom taxa indicative of different salinity and water quality conditions, geographic locations and habitat types. Assessments were made in contrasting wet and dry seasons in order to develop robust assessment models for salinity and water quality for this region. We found that diatom assemblages differed between nearshore and offshore locations, especially during the wet season when salinity and nutrient gradients were steepest. In the dry season, habitat structure was primary determinant of diatom assemblage composition. Among a suite of physicochemical variables, water depth and sediment total phosphorus (STP) were most strongly associated with diatom assemblage composition in the dry season, while salinity and water total phosphorus (TP) were more important in the wet season. We used indicator species analysis (ISA) to identify taxa that were most abundant and frequent at nearshore and offshore locations, in planktonic, epiphytic and benthic habitats and in contrasting salinity and water quality regimes. Because surface water concentrations of salts, total phosphorus, nitrogen (TN) and organic carbon (TOC) are partly controlled by water management in this region, diatom-based models were produced to infer these variables in modern and retrospective assessments of management-driven changes. Weighted averaging (WA) and weighted averaging partial least squares (WA-PLS) regressions produced reliable estimates of salinity, TP, TN and TOC from diatoms (r2 = 0.92, 0.77, 0.77 and 0.71, respectively). Because of their sensitivity to salinity, nutrient and TOC concentrations diatom assemblages should be useful in developing protective nutrient criteria for estuaries and coastal waters of Florida.