Various algorithms for optimization and learning in adaptive systems

This thesis discusses various methods for learning and optimization in adaptive systems. Overall, it emphasizes the relationship between optimization, learning, and adaptive systems; and it illustrates the influence of underlying hardware upon the construction of efficient algorithms for learning and optimization. Chapter 1 provides a summary and an overview.

Chapter 2 discusses a method for using feed-forward neural networks to filter the noise out of noise-corrupted signals. The networks use back-propagation learning, but they use it in a way that qualifies as unsupervised learning. The networks adapt based only on the raw input data-there are no external teachers providing information on correct operation during training. The chapter contains an analysis of the learning and develops a simple expression that, based only on the geometry of the network, predicts performance.

Chapter 3 explains a simple model of the piriform cortex, an area in the brain involved in the processing of olfactory information. The model was used to explore the possible effect of acetylcholine on learning and on odor classification. According to the model, the piriform cortex can classify odors better when acetylcholine is present during learning but not present during recall. This is interesting since it suggests that learning and recall might be separate neurochemical modes (corresponding to whether or not acetylcholine is present). When acetylcholine is turned off at all times, even during learning, the model exhibits behavior somewhat similar to Alzheimer's disease, a disease associated with the degeneration of cells that distribute acetylcholine.

Chapters 4, 5, and 6 discuss algorithms appropriate for adaptive systems implemented entirely in analog hardware. The algorithms inject noise into the systems and correlate the noise with the outputs of the systems. This allows them to estimate gradients and to implement noisy versions of gradient descent, without having to calculate gradients explicitly. The methods require only noise generators, adders, multipliers, integrators, and differentiators; and the number of devices needed scales linearly with the number of adjustable parameters in the adaptive systems. With the exception of one global signal, the algorithms require only local information exchange.

Electronic structure and phonon thermodynamics of iron alloys

Inelastic neutron scattering (INS) and nuclear-resonant inelastic x-ray scattering (NRIXS) were used to measure phonon spectra of FeV as a B2- ordered compound and as a bcc solid solution. Contrary to the behavior of ordering alloys studied to date, the phonons in the B2-ordered phase are softer than in the solid solution. Ordering increases the vibrational entropy, which stabilizes the ordered phase to higher temperatures. Ab initio calculations show that the number of electronic states at the Fermi level increases upon ordering, enhancing the screening between ions, and reducing the interatomic force constants. The effect of screening is larger at the V atomic sites than at the Fe atomic sites.

The phonon spectra of Au-rich alloys of fcc Au-Fe were also measured. The main effect on the vibrational entropy of alloying comes from a stiffening of the Au partial phonon density of states (DOS) with Fe concentration that increases the miscibility gap temperature. The magnitude of the effect is non- linear and it is reduced at higher Fe concentrations. Force constants were calculated for several compositions and show a local stiffening of Au–Au bonds close to Fe atoms, but Au–Au bonds that are farther away do not show this effect. Phonon DOS curves calculated from the force constants reproduced the experimental trends. The Au–Fe bond is soft and favors ordering, but a charge transfer from the Fe to the Au atoms stiffens the Au–Au bonds enough to favor unmixing. The stiffening is attributed to two main effects comparable in magnitude: an increase in electron density in the free-electron-like states, and stronger sd-hybridization.

INS and NRIXS measurements were performed at elevated temperatures on B2-ordered FeTi and NRIXS measurements were performed at high pressures. The high-pressure behavior is quasi- harmonic. The softening of the phonon DOS curves with temperature is strongly nonharmonic. Calculations of the force constants and Born-von Karman fits to the experimental data show that the bonds between second nearest neighbors (2nn) are much stiffer than those between 1nn, but fits to the high temperature data show that the former softens at a faster rate with temperature. The Fe–Fe bond softens more than the Ti–Ti bond. The unusual stiffness of the 2nn bond is explained by the calculated charge distribution, which is highly aspherical and localized preferentially in the t2g orbitals. Ab initio molecular dynamics (AIMD) simulations show a charge transfer from the t2g orbitals to the eg orbitals at elevated temperatures. The asphericity decreases linearly with temperature and is more severe at the Fe sites.

Optimal Data Distributions in Machine Learning

In the first part of the thesis we explore three fundamental questions that arise naturally when we conceive a machine learning scenario where the training and test distributions can differ. Contrary to conventional wisdom, we show that in fact mismatched training and test distribution can yield better out-of-sample performance. This optimal performance can be obtained by training with the dual distribution. This optimal training distribution depends on the test distribution set by the problem, but not on the target function that we want to learn. We show how to obtain this distribution in both discrete and continuous input spaces, as well as how to approximate it in a practical scenario. Benefits of using this distribution are exemplified in both synthetic and real data sets.

In order to apply the dual distribution in the supervised learning scenario where the training data set is fixed, it is necessary to use weights to make the sample appear as if it came from the dual distribution. We explore the negative effect that weighting a sample can have. The theoretical decomposition of the use of weights regarding its effect on the out-of-sample error is easy to understand but not actionable in practice, as the quantities involved cannot be computed. Hence, we propose the Targeted Weighting algorithm that determines if, for a given set of weights, the out-of-sample performance will improve or not in a practical setting. This is necessary as the setting assumes there are no labeled points distributed according to the test distribution, only unlabeled samples.

Finally, we propose a new class of matching algorithms that can be used to match the training set to a desired distribution, such as the dual distribution (or the test distribution). These algorithms can be applied to very large datasets, and we show how they lead to improved performance in a large real dataset such as the Netflix dataset. Their computational complexity is the main reason for their advantage over previous algorithms proposed in the covariate shift literature.

In the second part of the thesis we apply Machine Learning to the problem of behavior recognition. We develop a specific behavior classifier to study fly aggression, and we develop a system that allows analyzing behavior in videos of animals, with minimal supervision. The system, which we call CUBA (Caltech Unsupervised Behavior Analysis), allows detecting movemes, actions, and stories from time series describing the position of animals in videos. The method summarizes the data, as well as it provides biologists with a mathematical tool to test new hypotheses. Other benefits of CUBA include finding classifiers for specific behaviors without the need for annotation, as well as providing means to discriminate groups of animals, for example, according to their genetic line.