Optimal Data Distributions in Machine Learning

In the first part of the thesis we explore three fundamental questions that arise naturally when we conceive a machine learning scenario where the training and test distributions can differ. Contrary to conventional wisdom, we show that in fact mismatched training and test distribution can yield better out-of-sample performance. This optimal performance can be obtained by training with the dual distribution. This optimal training distribution depends on the test distribution set by the problem, but not on the target function that we want to learn. We show how to obtain this distribution in both discrete and continuous input spaces, as well as how to approximate it in a practical scenario. Benefits of using this distribution are exemplified in both synthetic and real data sets.

In order to apply the dual distribution in the supervised learning scenario where the training data set is fixed, it is necessary to use weights to make the sample appear as if it came from the dual distribution. We explore the negative effect that weighting a sample can have. The theoretical decomposition of the use of weights regarding its effect on the out-of-sample error is easy to understand but not actionable in practice, as the quantities involved cannot be computed. Hence, we propose the Targeted Weighting algorithm that determines if, for a given set of weights, the out-of-sample performance will improve or not in a practical setting. This is necessary as the setting assumes there are no labeled points distributed according to the test distribution, only unlabeled samples.

Finally, we propose a new class of matching algorithms that can be used to match the training set to a desired distribution, such as the dual distribution (or the test distribution). These algorithms can be applied to very large datasets, and we show how they lead to improved performance in a large real dataset such as the Netflix dataset. Their computational complexity is the main reason for their advantage over previous algorithms proposed in the covariate shift literature.

In the second part of the thesis we apply Machine Learning to the problem of behavior recognition. We develop a specific behavior classifier to study fly aggression, and we develop a system that allows analyzing behavior in videos of animals, with minimal supervision. The system, which we call CUBA (Caltech Unsupervised Behavior Analysis), allows detecting movemes, actions, and stories from time series describing the position of animals in videos. The method summarizes the data, as well as it provides biologists with a mathematical tool to test new hypotheses. Other benefits of CUBA include finding classifiers for specific behaviors without the need for annotation, as well as providing means to discriminate groups of animals, for example, according to their genetic line.

I. Electronic states of heavily-doped molecular crystals--naphthalene. I. Theoretical. II. Experimental. II. Lowest singlet and triplet excited states of pyrazine

PART I

The energy spectrum of heavily-doped molecular crystals was treated in the Green’s function formulation. The mixed crystal Green’s function was obtained by averaging over all possible impurity distributions. The resulting Green’s function, which takes the form of an infinite perturbation expansion, was further approximated by a closed form suitable for numerical calculations. The density-of-states functions and optical spectra for binary mixtures of normal naphthalene and deuterated naphthalene were calculated using the pure crystal density-of-state functions. The results showed that when the trap depth is large, two separate energy bands persist, but when the trap depth is small only a single band exists. Furthermore, in the former case it was found that the intensities of the outer Davydov bands are enhanced whereas the inner bands are weakened. Comparisons with previous theoretical calculations and experimental results are also made.

PART II

The energy states and optical spectra of heavily-doped mixed crystals are investigated. Studies are made for the following binary systems: (1) naphthalene-h₈ and d₈, (2) naphthalene--h₈ and αd₄, and (3) naphthalene--h₈ and βd₁, corresponding to strong, medium and weak perturbations. In addition to ordinary absorption spectra at 4˚K, band-to-band transitions at both 4˚K and 77˚K are also analyzed with emphasis on their relations to cooperative excitation and overall density-of-states functions for mixed crystals. It is found that the theoretical calculations presented in a previous paper agree generally with experiments except for cluster states observed in system (1) at lower guest concentrations. These features are discussed semi-quantitatively. As to the intermolecular interaction parameters, it is found that experimental results compare favorably with calculations based on experimental density-of-states functions but not with those based on octopole interactions or charge-transfer interactions. Previous experimental results of Sheka and the theoretical model of Broude and Rashba are also compared with present investigations.

PART III

The phosphorescence, fluorescence and absorption spectra of pyrazine-h₄ and d₄ have been obtained at 4˚K in a benzene matrix. For comparison, those of the isotopically mixed crystal pyrazine-h₄ in d₄ were also taken. All these spectra show extremely sharp and well-resolved lines and reveal detailed vibronic structure.

The analysis of the weak fluorescence spectrum resolves the long-disputed question of whether one or two transitions are involved in the near-ultraviolet absorption of pyrazine. The “mirror-image relationship” between absorption and emission shows that the lowest singlet state is an allowed transition, properly designated as ¹B_3u ← ¹A_1g. The forbidden component ¹B_2g, predicted by both “exciton” and MO theories to be below the allowed component, must lie higher. Its exact location still remains uncertain.

The phosphorescence spectrum when compared with the excitation phosphorescence spectra, indicates that the lowest triplet state is also symmetry allowed, showing a strong 0-0 band and a “mirror-image relationship” between absorption and emission. In accordance with previous work, the triplet state is designated as ³B_3u.

The vibronic structure of the phosphorescence spectrum is very complicated. Previous work on the analysis of this spectrum all concluded that a long progression of v_6a exists. Under the high resolution attainable in our work, the supposed v_6a progression proves to have a composite triplet structure, starting from the second member of the progression. Not only is the v_9a hydrogen-bending mode present as shown by the appearance of the C-D bending mode in the d₄ spectrum, but a band of 1207 cm^-1 in the pyrazine in benzene system and 1231 cm^-1 in the mixed crystal system is also observed. This band is assigned as 2v_6b and of a_1g symmetry. Its anonymously strong intensity in the phosphorescence spectrum is interpreted as due to the Fermi resonance with the 2v_6a and v_9a band.

To help resolve the present controversy over the crystal phosphorescence spectrum of pyrazine, detailed vibrational analyses of the emission spectra were made. The fluorescence spectrum has essentially the same vibronic structure as the phosphorescence spectrum.

Studies of response to earthquake ground motion

A study is made of the accuracy of electronic digital computer calculations of ground displacement and response spectra from strong-motion earthquake accelerograms. This involves an investigation of methods of the preparatory reduction of accelerograms into a form useful for the digital computation and of the accuracy of subsequent digital calculations. Various checks are made for both the ground displacement and response spectra results, and it is concluded that the main errors are those involved in digitizing the original record. Differences resulting from various investigators digitizing the same experimental record may become as large as 100% of the maximum computed ground displacements. The spread of the results of ground displacement calculations is greater than that of the response spectra calculations. Standardized methods of adjustment and calculation are recommended, to minimize such errors.

Studies are made of the spread of response spectral values about their mean. The distribution is investigated experimentally by Monte Carlo techniques using an electric analog system with white noise excitation, and histograms are presented indicating the dependence of the distribution on the damping and period of the structure. Approximate distributions are obtained analytically by confirming and extending existing results with accurate digital computer calculations. A comparison of the experimental and analytical approaches indicates good agreement for low damping values where the approximations are valid. A family of distribution curves to be used in conjunction with existing average spectra is presented. The combination of analog and digital computations used with Monte Carlo techniques is a promising approach to the statistical problems of earthquake engineering.

Methods of analysis of very small earthquake ground motion records obtained simultaneously at different sites are discussed. The advantages of Fourier spectrum analysis for certain types of studies and methods of calculation of Fourier spectra are presented. The digitizing and analysis of several earthquake records is described and checks are made of the dependence of results on digitizing procedure, earthquake duration and integration step length. Possible dangers of a direct ratio comparison of Fourier spectra curves are pointed out and the necessity for some type of smoothing procedure before comparison is established. A standard method of analysis for the study of comparative ground motion at different sites is recommended.

A theory of strong and weak scintillations with applications to astrophysics

The propagation of waves in an extended, irregular medium is studied under the "quasi-optics" and the "Markov random process" approximations. Under these assumptions, a Fokker-Planck equation satisfied by the characteristic functional of the random wave field is derived. A complete set of the moment equations with different transverse coordinates and different wavenumbers is then obtained from the characteristic functional. The derivation does not require Gaussian statistics of the random medium and the result can be applied to the time-dependent problem. We then solve the moment equations for the phase correlation function, angular broadening, temporal pulse smearing, intensity correlation function, and the probability distribution of the random waves. The necessary and sufficient conditions for strong scintillation are also given.

We also consider the problem of diffraction of waves by a random, phase-changing screen. The intensity correlation function is solved in the whole Fresnel diffraction region and the temporal pulse broadening function is derived rigorously from the wave equation.

The method of smooth perturbations is applied to interplanetary scintillations. We formulate and calculate the effects of the solar-wind velocity fluctuations on the observed intensity power spectrum and on the ratio of the observed "pattern" velocity and the true velocity of the solar wind in the three-dimensional spherical model. The r.m.s. solar-wind velocity fluctuations are found to be ~200 km/sec in the region about 20 solar radii from the Sun.

We then interpret the observed interstellar scintillation data using the theories derived under the Markov approximation, which are also valid for the strong scintillation. We find that the Kolmogorov power-law spectrum with an outer scale of 10 to 100 pc fits the scintillation data and that the ambient averaged electron density in the interstellar medium is about 0.025 cm^-3. It is also found that there exists a region of strong electron density fluctuation with thickness ~10 pc and mean electron density ~7 cm^-3 between the PSR 0833-45 pulsar and the earth.

Stationary absolute distributions for chains of infinite order

Let {Ƶ_n}^∞_{n = -∞} be a stochastic process with state space S₁ = {0, 1, …, D – 1}. Such a process is called a chain of infinite order. The transitions of the chain are described by the functions

Q_i(i⁽⁰⁾) = Ƥ(Ƶ_n = i | Ƶ_{n - 1} = i ⁽⁰⁾₁, Ƶ_{n - 2} = i ⁽⁰⁾₂, …) (i ɛ S₁), where i⁽⁰⁾ = (i⁽⁰⁾₁, i⁽⁰⁾₂, …) ranges over infinite sequences from S₁. If i⁽ⁿ⁾ = (i⁽ⁿ⁾₁, i⁽ⁿ⁾₂, …) for n = 1, 2,…, then i⁽ⁿ⁾ → i⁽⁰⁾ means that for each k, i⁽ⁿ⁾_k = i⁽⁰⁾_k for all n sufficiently large.

Given functions Q_i(i⁽⁰⁾) such that

(i) 0 ≤ Q_i(i⁽⁰) ≤ ξ ˂ 1

(ii)D – 1/Ʃ/i = 0 Q_i(i⁽⁰⁾) Ξ 1

(iii) Q_i(i⁽ⁿ⁾) → Q_i(i⁽⁰⁾) whenever i⁽ⁿ⁾ → i⁽⁰⁾,

we prove the existence of a stationary chain of infinite order {Ƶ_n} whose transitions are given by

Ƥ (Ƶ_n = i | Ƶ_{n - 1}, Ƶ_{n - 2}, …) = Q_i(Ƶ_{n - 1}, Ƶ_{n - 2}, …)

With probability 1. The method also yields stationary chains {Ƶ_n} for which (iii) does not hold but whose transition probabilities are, in a sense, “locally Markovian.” These and similar results extend a paper by T.E. Harris [Pac. J. Math., 5 (1955), 707-724].

Included is a new proof of the existence and uniqueness of a stationary absolute distribution for an Nth order Markov chain in which all transitions are possible. This proof allows us to achieve our main results without the use of limit theorem techniques.

Strong visible up-conversion emission in Tb3+, Tm3+ and Tb3+-Tm3+ co-doped tellurite glasses sensitized by Yb3+

Up-conversion luminescence characteristics under 975 nm excitation have been investigated with Tb3+/Tm3+/Yb3+ triply doped tellurite glasses. Here, green (547 nm: D-5(4) --> F-7(4)) and red (660 nm: D-5(4) --> F-7(2)) up-conversion (UC) luminescence originating from Tb3+ is observed strongly, because of the quadratic dependences of emission intensities on the excitation power. Especially, the UC luminescence was intensified violently with the energy transfer from the Tm3+ ions involves in the Tb3+ excitation. To the Tb3+/Tm3+/Yb3+ triply doped glass system, a novel up-conversion mechanism is proposed as follows: the energy of (3)G(4) level (Tm3+) was transferred to D-5(4) (Tb3+) and the 477-nm UC luminescence of Tm3+ was nearly quenched. (C) 2006 Elsevier B.V. All rights reserved.

I. Non-linear effects in a self-consistent calculation of SU_3 symmetry breaking in strong interaction coupling constants. II. Direct-channel reggeization of strong interaction scattering amplitudes

The thesis is divided into two parts. Part I generalizes a self-consistent calculation of residue shifts from SU₃ symmetry, originally performed by Dashen, Dothan, Frautschi, and Sharp, to include the effects of non-linear terms. Residue factorizability is used to transform an overdetermined set of equations into a variational problem, which is designed to take advantage of the redundancy of the mathematical system. The solution of this problem automatically satisfies the requirement of factorizability and comes close to satisfying all the original equations.

Part II investigates some consequences of direct channel Regge poles and treats the problem of relating Reggeized partial wave expansions made in different reaction channels. An analytic method is introduced which can be used to determine the crossed-channel discontinuity for a large class of direct-channel Regge representations, and this method is applied to some specific representations.

It is demonstrated that the multi-sheeted analytic structure of the Regge trajectory function can be used to resolve apparent difficulties arising from infinitely rising Regge trajectories. Also discussed are the implications of large collections of "daughter trajectories."

Two things are of particular interest: first, the threshold behavior in direct and crossed channels; second, the potentialities of Reggeized representations for us in self-consistent calculations. A new representation is introduced which surpasses previous formulations in these two areas, automatically satisfying direct-channel threshold constraints while being capable of reproducing a reasonable crossed channel discontinuity. A scalar model is investigated for low energies, and a relation is obtained between the mass of the lowest bound state and the slope of the Regge trajectory.

Investigation of strong earthquake ground motion

The pattern of energy release during the Imperial Valley, California, earthquake of 1940 is studied by analysing the El Centro strong motion seismograph record and records from the Tinemaha seismograph station, 546 km from the epicenter. The earthquake was a multiple event sequence with at least 4 events recorded at El Centro in the first 25 seconds, followed by 9 events recorded in the next 5 minutes. Clear P, S and surface waves were observed on the strong motion record. Although the main part of the earthquake energy was released during the first 15 seconds, some of the later events were as large as M = 5.8 and thus are important for earthquake engineering studies. The moment calculated using Fourier analysis of surface waves agrees with the moment estimated from field measurements of fault offset after the earthquake. The earthquake engineering significance of the complex pattern of energy release is discussed. It is concluded that a cumulative increase in amplitudes of building vibration resulting from the present sequence of shocks would be significant only for structures with relatively long natural period of vibration. However, progressive weakening effects may also lead to greater damage for multiple event earthquakes.

The model with surface Love waves propagating through a single layer as a surface wave guide is studied. It is expected that the derived properties for this simple model illustrate well several phenomena associated with strong earthquake ground motion. First, it is shown that a surface layer, or several layers, will cause the main part of the high frequency energy, radiated from the nearby earthquake, to be confined to the layer as a wave guide. The existence of the surface layer will thus increase the rate of the energy transfer into the man-made structures on or near the surface of the layer. Secondly, the surface amplitude of the guided SH waves will decrease if the energy of the wave is essentially confined to the layer and if the wave propagates towards an increasing layer thickness. It is also shown that the constructive interference of SH waves will cause the zeroes and the peaks in the Fourier amplitude spectrum of the surface ground motion to be continuously displaced towards the longer periods as the distance from the source of the energy release increases.

Classical and Quantum Effects in Plasmonic Metals

The field of plasmonics exploits the unique optical properties of metallic nanostructures to concentrate and manipulate light at subwavelength length scales. Metallic nanostructures get their unique properties from their ability to support surface plasmons– coherent wave-like oscillations of the free electrons at the interface between a conductive and dielectric medium. Recent advancements in the ability to fabricate metallic nanostructures with subwavelength length scales have created new possibilities in technology and research in a broad range of applications.

In the first part of this thesis, we present two investigations of the relationship between the charge state and optical state of plasmonic metal nanoparticles. Using experimental bias-dependent extinction measurements, we derive a potential- dependent dielectric function for Au nanoparticles that accounts for changes in the physical properties due to an applied bias that contribute to the optical extinction. We also present theory and experiment for the reverse effect– the manipulation of the carrier density of Au nanoparticles via controlled optical excitation. This plasmoelectric effect takes advantage of the strong resonant properties of plasmonic materials and the relationship between charge state and optical properties to eluci- date a new avenue for conversion of optical power to electrical potential.

The second topic of this thesis is the non-radiative decay of plasmons to a hot-carrier distribution, and the distribution’s subsequent relaxation. We present first-principles calculations that capture all of the significant microscopic mechanisms underlying surface plasmon decay and predict the initial excited carrier distributions so generated. We also preform ab initio calculations of the electron-temperature dependent heat capacities and electron-phonon coupling coefficients of plasmonic metals. We extend these first-principle methods to calculate the electron-temperature dependent dielectric response of hot electrons in plasmonic metals, including direct interband and phonon-assisted intraband transitions. Finally, we combine these first-principles calculations of carrier dynamics and optical response to produce a complete theoretical description of ultrafast pump-probe measurements, free of any fitting parameters that are typical in previous analyses.

Inelastic K+p reactions at incident momenta from 1.37 GeV/c to 2.17 GeV/c

A number of recent experiments have suggested the possibility of a highly inelastic resonance in K⁺p scattering. To study the inelastic K⁺p reactions, a 400 K exposure has been taken at the L.R.L. 25 inch bubble chamber. The data are spread over seven K⁺ momenta between 1.37 and 2.17 GeV/c.

Cross-sections have been measured for the reaction K⁺p → pK°π+ which is dominated by the quasi-two body channels K∆ and K*N. Both these channels are strongly peripheral, as at other momenta. The decay of the ∆ is in good agreement with the predictions of the rho-photon analogy of Stodolsky and Sakurai. The data on the K*p channel show evidence of both pseudo scalar and vector exchange.

Cross-sections for the final state pK⁺π+π- shows a strong contribution from the quasi-two body channel K*∆. This reaction is also very peripheral even at threshold. The decay angular distributions indicate the reaction is dominated as at higher momenta by a pion exchange mechanism. The data are also in good agreement with the quark model predictions of Bialas and Zalewski for the K* and ∆ decay.

Transverse intensity distributions of a broadband laser modulated by a hard-edged aperture

By means of the Huygens-Fresnel diffraction integral, the field representation of a laser beam modulated by a hard-edged aperture is derived. The near-field and far-field transverse intensity distributions of the beams with different bandwidths are analyzed by using the representation. The numerical calculation results indicate that the amplitudes and numbers of the intensity spikes decrease with increasing bandwidth, and beam smoothing is achieved when the bandwidth takes a certain value in the near field. In the far field, the radius of the transverse intensity distribution decreases as the bandwidth increases, and the physical explanation of this fact is also given. (c) 2005 Optical Society of America.

Influence of the chirp on the intensity distributions of an apertured pulse

Starting from the Huygens-Fresnel diffraction integral and the Fourier transform, the propagation expression of a chirped pulse passing through a hard-edged aperture is derived. Using the obtained expression, the intensity distributions of the pulse with different chirp in the near and far fields are analyzed in detail. Due to the modulation of the aperture, many intensity peaks emerge in the intensity distributions of the chirped pulse in the near field. However, the amplitudes of the intensity peaks decrease on increasing the chirp, which results in the smoothing effect in the intensity distributions. The beam smoothing brought by increasing the chirp is explained physically. Also, it is found that the radius of the intensity distribution of the chirped pulse decreases when the chirp increases in the far field. (c) 2005 Elsevier GmbH. All rights reserved.

Strong visible up-conversion emissions and thermometric applications of Er3+-Yb3+ codoped Al2O3 prepared by the sol-gel method

The Er3+-Yb3+ codoped Al2O3 has been prepared by the sol-gel method using the aluminium isopropoxide [Al(OC3H7)(3)]-derived Al2O3 sols with addition of the erbium nitrate [Er(NO3)(3) center dot 5H(2)O] and ytterbium nitrate [Yb(NO3)(3) center dot 5H(2)O]. The phase structure, including only two crystalline types of doped Al2O3 phases, theta and gamma, was obtained for the 1 mol% Er3+ and 5 mol% Yb3+ codoped Al2O3 at the sintering temperature of 1,273 K. By a 978 nm semiconductor laser diodes excitation, the visible up-conversion emissions centered at about 523, 545, and 660 nm were obtained. The temperature dependence of the green up-conversion emissions was studied over a wide temperature range of 300-825 K, and the reasonable agreement between the calculated temperature by the fluorescence intensity ratio (FIR) theory and the measured temperature proved that Er3+-Yb3+ codoped Al2O3 plays an important role in the application of high temperature sensor.