Optimal Data Distributions in Machine Learning

In the first part of the thesis we explore three fundamental questions that arise naturally when we conceive a machine learning scenario where the training and test distributions can differ. Contrary to conventional wisdom, we show that in fact mismatched training and test distribution can yield better out-of-sample performance. This optimal performance can be obtained by training with the dual distribution. This optimal training distribution depends on the test distribution set by the problem, but not on the target function that we want to learn. We show how to obtain this distribution in both discrete and continuous input spaces, as well as how to approximate it in a practical scenario. Benefits of using this distribution are exemplified in both synthetic and real data sets.

In order to apply the dual distribution in the supervised learning scenario where the training data set is fixed, it is necessary to use weights to make the sample appear as if it came from the dual distribution. We explore the negative effect that weighting a sample can have. The theoretical decomposition of the use of weights regarding its effect on the out-of-sample error is easy to understand but not actionable in practice, as the quantities involved cannot be computed. Hence, we propose the Targeted Weighting algorithm that determines if, for a given set of weights, the out-of-sample performance will improve or not in a practical setting. This is necessary as the setting assumes there are no labeled points distributed according to the test distribution, only unlabeled samples.

Finally, we propose a new class of matching algorithms that can be used to match the training set to a desired distribution, such as the dual distribution (or the test distribution). These algorithms can be applied to very large datasets, and we show how they lead to improved performance in a large real dataset such as the Netflix dataset. Their computational complexity is the main reason for their advantage over previous algorithms proposed in the covariate shift literature.

In the second part of the thesis we apply Machine Learning to the problem of behavior recognition. We develop a specific behavior classifier to study fly aggression, and we develop a system that allows analyzing behavior in videos of animals, with minimal supervision. The system, which we call CUBA (Caltech Unsupervised Behavior Analysis), allows detecting movemes, actions, and stories from time series describing the position of animals in videos. The method summarizes the data, as well as it provides biologists with a mathematical tool to test new hypotheses. Other benefits of CUBA include finding classifiers for specific behaviors without the need for annotation, as well as providing means to discriminate groups of animals, for example, according to their genetic line.

High-energy scattering processes with cuts in the angular momentum plane

The experimental consequence of Regge cuts in the angular momentum plane are investigated. The principle tool in the study is the set of diagrams originally proposed by Amati, Fubini, and Stanghellini. Mandelstam has shown that the AFS cuts are actually cancelled on the physical sheet, but they may provide a useful guide to the properties of the real cuts. Inclusion of cuts modifies the simple Regge pole predictions for high-energy scattering data. As an example, an attempt is made to fit high energy elastic scattering data for pp, ṗp, π^±p, and K^±p, by replacing the Igi pole by terms representing the effect of a Regge cut. The data seem to be compatible with either a cut or the Igi pole.

Theoretical investigation of turbulent boundary layer over a wavy surface

The important features of the two-dimensional incompressible turbulent flow over a wavy surface of wavelength comparable with the boundary layer thickness are analyzed.

A turbulent field method using model equation for turbulent shear stress similar to the scheme of Bradshaw, Ferriss and Atwell (1967) is employed with suitable modification to cover the viscous sublayer. The governing differential equations are linearized based on the small but finite amplitude to wavelength ratio. An orthogonal wavy coordinate system, accurate to the second order in the amplitude ratio, is adopted to avoid the severe restriction to the validity of linearization due to the large mean velocity gradient near the wall. Analytic solution up to the second order is obtained by using the method of matched-asymptotic-expansion based on the large Reynolds number and hence the small skin friction coefficient.

In the outer part of the layer, the perturbed flow is practically "inviscid." Solutions for the velocity, Reynolds stress and also the wall pressure distributions agree well with the experimental measurement. In the wall region where the perturbed Reynolds stress plays an important role in the process of momentum transport, only a qualitative agreement is obtained. The results also show that the nonlinear second-order effect is negligible for amplitude ratio of 0.03. The discrepancies in the detailed structure of the velocity, shear stress, and skin friction distributions near the wall suggest modifications to the model are required to describe the present problem.

Stationary absolute distributions for chains of infinite order

Let {Ƶ_n}^∞_{n = -∞} be a stochastic process with state space S₁ = {0, 1, …, D – 1}. Such a process is called a chain of infinite order. The transitions of the chain are described by the functions

Q_i(i⁽⁰⁾) = Ƥ(Ƶ_n = i | Ƶ_{n - 1} = i ⁽⁰⁾₁, Ƶ_{n - 2} = i ⁽⁰⁾₂, …) (i ɛ S₁), where i⁽⁰⁾ = (i⁽⁰⁾₁, i⁽⁰⁾₂, …) ranges over infinite sequences from S₁. If i⁽ⁿ⁾ = (i⁽ⁿ⁾₁, i⁽ⁿ⁾₂, …) for n = 1, 2,…, then i⁽ⁿ⁾ → i⁽⁰⁾ means that for each k, i⁽ⁿ⁾_k = i⁽⁰⁾_k for all n sufficiently large.

Given functions Q_i(i⁽⁰⁾) such that

(i) 0 ≤ Q_i(i⁽⁰) ≤ ξ ˂ 1

(ii)D – 1/Ʃ/i = 0 Q_i(i⁽⁰⁾) Ξ 1

(iii) Q_i(i⁽ⁿ⁾) → Q_i(i⁽⁰⁾) whenever i⁽ⁿ⁾ → i⁽⁰⁾,

we prove the existence of a stationary chain of infinite order {Ƶ_n} whose transitions are given by

Ƥ (Ƶ_n = i | Ƶ_{n - 1}, Ƶ_{n - 2}, …) = Q_i(Ƶ_{n - 1}, Ƶ_{n - 2}, …)

With probability 1. The method also yields stationary chains {Ƶ_n} for which (iii) does not hold but whose transition probabilities are, in a sense, “locally Markovian.” These and similar results extend a paper by T.E. Harris [Pac. J. Math., 5 (1955), 707-724].

Included is a new proof of the existence and uniqueness of a stationary absolute distribution for an Nth order Markov chain in which all transitions are possible. This proof allows us to achieve our main results without the use of limit theorem techniques.

Experimental study study of antiproton-proton annihilation into a pair of charged Pi-Mesons or K-Mesons for incident antiproton mementa in the range from 0.72 GeV/c to 2.62 GeV/c

The cross sections for the two antiproton-proton annihilation-in-flight modes,

ˉp + p → π⁺ + π^-

ˉp + p → k⁺ + k^-

were measured for fifteen laboratory antiproton beam momenta ranging from 0.72 to 2.62 GeV/c. No magnets were used to determine the charges in the final state. As a result, the angular distributions were obtained in the form [dσ/dΩ (Θ_C.M.) + dσ/dΩ (π – Θ_C.M.)] for 45 ≲ Θ_C.M. ≲ 135°.

A hodoscope-counter system was used to discriminate against events with final states having more than two particles and antiproton-proton elastic scattering events. One spark chamber was used to record the track of each of the two charged final particles. A total of about 40,000 pictures were taken. The events were analyzed by measuring the laboratory angle of the track in each chamber. The value of the square of the mass of the final particles was calculated for each event assuming the reaction

ˉp + p → a pair of particles with equal masses.

About 20,000 events were found to be either annihilation into π ^±-pair or k ^±-pair events. The two different charged meson pair modes were also distinctly separated.

The average differential cross section of ˉp + p → π⁺ + π^- varied from ~ 25 µb/sr at antiproton beam momentum 0.72 GeV/c (total energy in center-of-mass system, √s = 2.0 GeV) to ~ 2 µb/sr at beam momentum 2.62 GeV/c (√s = 2.64 GeV). The most striking feature in the angular distribution was a peak at Θ_C.M. = 90° (cos Θ_C.M. = 0) which increased with √s and reached a maximum at √s ~ 2.1 GeV (beam momentum ~ 1.1 GeV/c). Then it diminished and seemed to disappear completely at √s ~ 2.5 GeV (beam momentum ~ 2.13 GeV/c). A valley in the angular distribution occurred at cos Θ_C.M. ≈ 0.4. The differential cross section then increased as cos Θ_C.M. approached 1.

The average differential cross section for ˉp + p → k⁺ + k^- was about one third of that of the π^±-pair mode throughout the energy range of this experiment. At the lower energies, the angular distribution, unlike that of the π^±-pair mode, was quite isotropic. However, a peak at Θ_C.M. = 90° seemed to develop at √s ~ 2.37 GeV (antiproton beam momentum ~ 1.82 GeV/c). No observable change was seen at that energy in the π^±-pair cross section.

The possible connection of these features with the observed meson resonances at 2.2 GeV and 2.38 GeV, and its implications, were discussed.

Part I: Total yield measurements for the ^(21)Ne(α,n)^)(24)Mg reaction. Part II: A study of the ^(12)C(He,p)^(14)N reaction

PART I

The total cross-section for the reaction ²¹Ne(α, n)²⁴Mg has been measured in the energy range 1.49 Mev ≤ E_cm ≤ 2.6 Mev. The cross-section factor, S(O), for this reaction has been determined, by means of an optical model calculation, to be in the range 1.52 x 10¹² mb-Mev to 2.67 x 10¹² mb-Mev, for interaction radii in the range 5.0 fm to 6.6 fm. With S(O) ≈ 2 x 10¹² mb-Mev, the reaction ²¹Ne(α, n)²⁴Mg can produce a large enough neutron flux to be a significant astrophysical source of neutrons.

PART II

The reaction¹²C(³He, p)¹⁴N has been studied over the energy range 12 Mev ≤ E_lab ≤ 18 Mev. Angular distributions of the proton groups leading to the lowest seven levels in ¹⁴N were obtained.

Distorted wave calculations, based on two-nucleon transfer theory, were performed, and were found to be reliable for obtaining the value of the orbital angular momentum transferred. The present work shows that such calculations do not yield unambiguous values for the spectroscopic factors.

Transverse intensity distributions of a broadband laser modulated by a hard-edged aperture

By means of the Huygens-Fresnel diffraction integral, the field representation of a laser beam modulated by a hard-edged aperture is derived. The near-field and far-field transverse intensity distributions of the beams with different bandwidths are analyzed by using the representation. The numerical calculation results indicate that the amplitudes and numbers of the intensity spikes decrease with increasing bandwidth, and beam smoothing is achieved when the bandwidth takes a certain value in the near field. In the far field, the radius of the transverse intensity distribution decreases as the bandwidth increases, and the physical explanation of this fact is also given. (c) 2005 Optical Society of America.

Influence of the chirp on the intensity distributions of an apertured pulse

Starting from the Huygens-Fresnel diffraction integral and the Fourier transform, the propagation expression of a chirped pulse passing through a hard-edged aperture is derived. Using the obtained expression, the intensity distributions of the pulse with different chirp in the near and far fields are analyzed in detail. Due to the modulation of the aperture, many intensity peaks emerge in the intensity distributions of the chirped pulse in the near field. However, the amplitudes of the intensity peaks decrease on increasing the chirp, which results in the smoothing effect in the intensity distributions. The beam smoothing brought by increasing the chirp is explained physically. Also, it is found that the radius of the intensity distribution of the chirped pulse decreases when the chirp increases in the far field. (c) 2005 Elsevier GmbH. All rights reserved.