Latent variable models for neural data analysis

The brain is perhaps the most complex system to have ever been subjected to rigorous scientific investigation. The scale is staggering: over 10^11 neurons, each making an average of 10^3 synapses, with computation occurring on scales ranging from a single dendritic spine, to an entire cortical area. Slowly, we are beginning to acquire experimental tools that can gather the massive amounts of data needed to characterize this system. However, to understand and interpret these data will also require substantial strides in inferential and statistical techniques. This dissertation attempts to meet this need, extending and applying the modern tools of latent variable modeling to problems in neural data analysis.

It is divided into two parts. The first begins with an exposition of the general techniques of latent variable modeling. A new, extremely general, optimization algorithm is proposed - called Relaxation Expectation Maximization (REM) - that may be used to learn the optimal parameter values of arbitrary latent variable models. This algorithm appears to alleviate the common problem of convergence to local, sub-optimal, likelihood maxima. REM leads to a natural framework for model size selection; in combination with standard model selection techniques the quality of fits may be further improved, while the appropriate model size is automatically and efficiently determined. Next, a new latent variable model, the mixture of sparse hidden Markov models, is introduced, and approximate inference and learning algorithms are derived for it. This model is applied in the second part of the thesis.

The second part brings the technology of part I to bear on two important problems in experimental neuroscience. The first is known as spike sorting; this is the problem of separating the spikes from different neurons embedded within an extracellular recording. The dissertation offers the first thorough statistical analysis of this problem, which then yields the first powerful probabilistic solution. The second problem addressed is that of characterizing the distribution of spike trains recorded from the same neuron under identical experimental conditions. A latent variable model is proposed. Inference and learning in this model leads to new principled algorithms for smoothing and clustering of spike data.

Chemical and mineralogical characterization of micro-inclusions in diamonds

Secondary-ion mass spectrometry (SIMS), electron probe analysis (EPMA), analytical scanning electron microscopy (SEM) and infrared (IR) spectroscopy were used to determine the chemical composition and the mineralogy of sub-micrometer inclusions in cubic diamonds and in overgrowths (coats) on octahedral diamonds from Zaire, Botswana, and some unknown localities.

The inclusions are sub-micrometer in size. The typical diameter encountered during transmission electron microscope (TEM) examination was 0.1-0.5 µm. The micro-inclusions are sub-rounded and their shape is crystallographically controlled by the diamond. Normally they are not associated with cracks or dislocations and appear to be well isolated within the diamond matrix. The number density of inclusions is highly variable on any scale and may reach 10^(11) inclusions/cm^3 in the most densely populated zones. The total concentration of metal oxides in the diamonds varies between 20 and 1270 ppm (by weight).

SIMS analysis yields the average composition of about 100 inclusions contained in the sputtered volume. Comparison of analyses of different volumes of an individual diamond show roughly uniform composition (typically ±10% relative). The variation among the average compositions of different diamonds is somewhat greater (typically ±30%). Nevertheless, all diamonds exhibit similar characteristics, being rich in water, carbonate, SiO_2, and K_2O, and depleted in MgO. The composition of micro-inclusions in most diamonds vary within the following ranges: SiO_2, 30-53%; K_2O, 12-30%; CaO, 8-19%; FeO, 6-11%; Al_2O_3, 3-6%; MgO, 2-6%; TiO_2, 2-4%; Na_2O, 1-5%; P_2O_5, 1-4%; and Cl, 1-3%. In addition, BaO, 1-4%; SrO, 0.7-1.5%; La_2O_3, 0.1-0.3%; Ce_2O_3, 0.3-0.5%; smaller amounts of other rare-earth elements (REE), as well as Mn, Th, and U were also detected by instrumental neutron activation analysis (INAA). Mg/(Fe+Mg), 0.40-0.62 is low compared with other mantle derived phases; K/ AI ratios of 2-7 are very high, and the chondrite-normalized Ce/Eu ratios of 10-21 are also high, indicating extremely fractionated REE patterns.

SEM analyses indicate that individual inclusions within a single diamond are roughly of similar composition. The average composition of individual inclusions as measured with the SEM is similar to that measured by SIMS. Compositional variations revealed by the SEM are larger than those detected by SIMS and indicate a small variability in the composition of individual inclusions. No compositions of individual inclusions were determined that might correspond to mono-mineralic inclusions.

IR spectra of inclusion- bearing zones exhibit characteristic absorption due to: (1) pure diamonds, (2) nitrogen and hydrogen in the diamond matrix; and (3) mineral phases in the micro-inclusions. Nitrogen concentrations of 500-1100 ppm, typical of the micro-inclusion-bearing zones, are higher than the average nitrogen content of diamonds. Only type IaA centers were detected by IR. A yellow coloration may indicate small concentration of type IB centers.

The absorption due to the micro-inclusions in all diamonds produces similar spectra and indicates the presence of hydrated sheet silicates (most likely, Fe-rich clay minerals), carbonates (most likely calcite), and apatite. Small quantities of molecular CO_2 are also present in most diamonds. Water is probably associated with the silicates but the possibility of its presence as a fluid phase cannot be excluded. Characteristic lines of olivine, pyroxene and garnet were not detected and these phases cannot be significant components of the inclusions. Preliminary quantification of the IR data suggests that water and carbonate account for, on average, 20-40 wt% of the micro-inclusions.

The composition and mineralogy of the micro-inclusions are completely different from those of the more common, larger inclusions of the peridotitic or eclogitic assemblages. Their bulk composition resembles that of potassic magmas, such as kimberlites and lamproites, but is enriched in H_2O, CO_3, K_2O, and incompatible elements, and depleted in MgO.

It is suggested that the composition of the micro-inclusions represents a volatile-rich fluid or a melt trapped by the diamond during its growth. The high content of K, Na, P, and incompatible elements suggests that the trapped material found in the micro-inclusions may represent an effective metasomatizing agent. It may also be possible that fluids of similar composition are responsible for the extreme enrichment of incompatible elements documented in garnet and pyroxene inclusions in diamonds.

The origin of the fluid trapped in the micro-inclusions is still uncertain. It may have been formed by incipient melting of a highly metasomatized mantle rocks. More likely, it is the result of fractional crystallization of a potassic parental magma at depth. In either case, the micro-inclusions document the presence of highly potassic fluids or melts at depths corresponding to the diamond stability field in the upper mantle. The phases presently identified in the inclusions are believed to be the result of closed system reactions at lower pressures.

Variable-angle electron spectroscopic studies of various polyatomic molecular systems

The complementary techniques of low-energy, variable-angle electron-impact spectroscopy and ultraviolet variable-angle photoelectron spectroscopy have been used to study the electronic spectroscopy and structure of several series of molecules. Electron-impact studies were performed at incident beam energies between 25 eV and 100 eV and at scattering angles ranging from 0° to 90°. The energy-loss regions from 0 eV to greater than 15 eV were studied. Photoelectron spectroscopic studies were conducted using a HeI radiation source and spectra were measured at scattering angles from 45° to 90°. The molecules studied were chosen because of their spectroscopic, chemical, and structural interest. The operation of a new electron-impact spectrometer with multiple-mode target source capability is described. This spectrometer has been used to investigate the spin-forbidden transitions in a number of molecular systems.

The electron-impact spectroscopy of the six chloro-substituted ethylenes has been studied over the energy-loss region from 0-15 eV. Spin-forbidden excitations corresponding to the π → π*, N → T transition have been observed at excitation energies ranging from 4.13 eV in vinyl chloride to 3.54 eV in tetrachloroethylene. Symmetry-forbidden transitions of the type π → np have been oberved in trans-dichloroethyene and tetrachlor oethylene. In addition, transitions to many states lying above the first ionization potential were observed for the first time. Many of these bands have been assigned to Rydberg series converging to higher ionization potentials. The trends observed in the measured transition energies for the π → π*, N → T, and N → V as well as the π → 3s excitation are discussed and compared to those observed in the methyl- and fluoro- substituted ethylenes.

The electron energy-loss spectra of the group VIb transition metal hexacarbonyls have been studied in the 0 eV to 15 eV region. The differential cross sections were obtained for several features in the 3-7 eV energy-loss region. The symmetry-forbidden nature of the ¹A_1g → ¹A_1g, 2t_2g(π) → 3t_2g(π*) transition in these compounds was confirmed by the high-energy, low-angle behavior of their relative intensities. Several low lying transitions have been assigned to ligand field transitions on the basis of the energy and angular behavior of the differential cross sections for these transitions. No transitions which could clearly be assigned to singlet → triplet excitations involving metal orbitals were located. A number of states lying above the first ionization potential have been observed for the first time. A number of features in the 6-14 eV energy-loss region of the spectra of these compounds correspond quite well to those observed in free CO.

A number of exploratory studies have been performed. The π → π*, N → T, singlet → triplet excitation has been located in vinyl bromide at 4.05 eV. We have also observed this transition at approximately 3.8 eV in a cis-/trans- mixture of the 1,2-dibromoethylenes. The low-angle spectrum of iron pentacarbonyl was measured over the energy-loss region extending from 2-12 eV. A number of transitions of 8 eV or greater excitation energy were observed for the first time. Cyclopropane was also studied at both high and low angles but no clear evidence for any spin- forbidden transitions was found. The electron-impact spectrum of the methyl radical resulting from the pyrolysis of tetramethyl tin was obtained at 100 eV incident energy and at 0° scattering angle. Transitions observed at 5.70 eV and 8.30 eV agree well with the previous optical results. In addition, a number of bands were observed in the 8-14 eV region which are most likely due to Rydberg transitions converging to the higher ionization potentials of this molecule. This is the first reported electron-impact spectrum of a polyatomic free radical.

Variable-angle photoelectron spectroscopic studies were performed on a series of three-membered-ring heterocyclic compounds. These compounds are of great interest due to their highly unusual structure. Photoelectron angular distributions using HeI radiation have been measured for the first time for ethylene oxide and ethyleneimine. The measured anisotropy parameters, β, along with those measured for cyclopropane were used to confirm the orbital correlations and photoelectron band assignments. No high values of β similar to those expected for alkene π orbitals were observed for the Walsh or Forster-Coulson-Moffit type orbitals.

The variations and climatic significance of D/H ratios in the carbon-bound hydrogen of cellulose in trees

The σD values of nitrated cellulose from a variety of trees covering a wide geographic range have been measured. These measurements have been used to ascertain which factors are likely to cause σD variations in cellulose C-H hydrogen.

It is found that a primary source of tree σD variation is the σD variation of the environmental precipitation. Superimposed on this are isotopic variations caused by the transpiration of the leaf water incorporated by the tree. The magnitude of this transpiration effect appears to be related to relative humidity.

Within a single tree, it is found that the hydrogen isotope variations which occur for a ring sequence in one radial direction may not be exactly the same as those which occur in a different direction. Such heterogeneities appear most likely to occur in trees with asymmetric ring patterns that contain reaction wood. In the absence of reaction wood such heterogeneities do not seem to occur. Thus, hydrogen isotope analyses of tree ring sequences should be performed on trees which do not contain reaction wood.

Comparisons of tree σD variations with variations in local climate are performed on two levels: spatial and temporal. It is found that the σD values of 20 North American trees from a wide geographic range are reasonably well-correlated with the corresponding average annual temperature. The correlation is similar to that observed for a comparison of the σD values of annual precipitation of 11 North American sites with annual temperature. However, it appears that this correlation is significantly disrupted by trees which grew on poorly drained sites such as those in stagnant marshes. Therefore, site selection may be important in choosing trees for climatic interpretation of σD values, although proper sites do not seem to be uncommon.

The measurement of σD values in 5-year samples from the tree ring sequences of 13 trees from 11 North American sites reveals a variety of relationships with local climate. As it was for the spatial σD vs climate comparison, site selection is also apparently important for temporal tree σD vs climate comparisons. Again, it seems that poorly-drained sites are to be avoided. For nine trees from different "well-behaved" sites, it was found that the local climatic variable best related to the σD variations was not the same for all sites.

Two of these trees showed a strong negative correlation with the amount of local summer precipitation. Consideration of factors likely to influence the isotopic composition of summer rain suggests that rainfall intensity may be important. The higher the intensity, the lower the σD value. Such an effect might explain the negative correlation of σD vs summer precipitation amount for these two trees. A third tree also exhibited a strong correlation with summer climate, but in this instance it was a positive correlation of σD with summer temperature.

The remaining six trees exhibited the best correlation between σD values and local annual climate. However, in none of these six cases was it annual temperature that was the most important variable. In fact annual temperature commonly showed no relationship at all with tree σD values. Instead, it was found that a simple mass balance model incorporating two basic assumptions yielded parameters which produced the best relationships with tree σD values. First, it was assumed that the σD values of these six trees reflected the σD values of annual precipitation incorporated by these trees. Second, it was assumed that the σD value of the annual precipitation was a weighted average of two seasonal isotopic components: summer and winter. Mass balance equations derived from these assumptions yielded combinations of variables that commonly showed a relationship with tree σD values where none had previously been discerned.

It was found for these "well-behaved" trees that not all sample intervals in a σD vs local climate plot fell along a well-defined trend. These departures from the local σD VS climate norm were defined as "anomalous". Some of these anomalous intervals were common to trees from different locales. When such widespread commonalty of an anomalous interval occurred, it was observed that the interval corresponded to an interval in which drought had existed in the North American Great Plains.

Consequently, there appears to be a combination of both local and large scale climatic information in the σD variations of tree cellulose C-H hydrogen.

Geochemistry and resonance ionization of platinum-group elements

Experimental studies were conducted with the goals of 1) determining the origin of Pt- group element (PGE) alloys and associated mineral assemblages in refractory inclusions from meteorites and 2) developing a new ultrasensitive method for the in situ chemical and isotopic analysis of PGE. A general review of the geochemistry and cosmochemistry of the PGE is given, and specific research contributions are presented within the context of this broad framework.

An important step toward understanding the cosmochemistry of the PGE is the determination of the origin of POE-rich metallic phases (most commonly εRu-Fe) that are found in Ca, AJ-rich refractory inclusions (CAI) in C3V meteorites. These metals occur along with γNi-Fe metals, Ni-Fe sulfides and Fe oxides in multiphase opaque assemblages. Laboratory experiments were used to show that the mineral assemblages and textures observed in opaque assemblages could be produced by sulfidation and oxidation of once homogeneous Ni-Fe-PGE metals. Phase equilibria, partitioning and diffusion kinetics were studied in the Ni-Fe-Ru system in order to quantify the conditions of opaque assemblage formation. Phase boundaries and tie lines in the Ni-Fe-Ru system were determined at 1273, 1073 and 873K using an experimental technique that allowed the investigation of a large portion of the Ni-Fe-Ru system with a single experiment at each temperature by establishing a concentration gradient within which local equilibrium between coexisting phases was maintained. A wide miscibility gap was found to be present at each temperature, separating a hexagonal close-packed εRu-Fe phase from a face-centered cubic γNi-Fe phase. Phase equilibria determined here for the Ni-Fe-Ru system, and phase equilibria from the literature for the Ni-Fe-S and Ni-Fe-O systems, were compared with analyses of minerals from opaque assemblages to estimate the temperature and chemical conditions of opaque assemblage formation. It was determined that opaque assemblages equilibrated at a temperature of ~770K, a sulfur fugacity 10 times higher than an equilibrium solar gas, and an oxygen fugacity 10⁶ times higher than an equilibrium solar gas.

Diffusion rates between -γNi-Fe and εRu-Fe metal play a critical role in determining the time (with respect to CAI petrogenesis) and duration of the opaque assemblage equilibration process. The diffusion coefficient for Ru in Ni (D^Ru_Ni) was determined as an analog for the Ni-Fe-Ru system by the thin-film diffusion method in the temperature range of 1073 to 1673K and is given by the expression:

D^Ru_Ni (cm² sec^-1) = 5.0(±0.7) x 10^-3 exp(-2.3(±0.1) x 10¹² erg mole^-1/RT) where R is the gas constant and T is the temperature in K. Based on the rates of dissolution and exsolution of metallic phases in the Ni-Fe-Ru system it is suggested that opaque assemblages equilibrated after the melting and crystallization of host CAI during a metamorphic event of ≥ 10³ years duration. It is inferred that opaque assemblages originated as immiscible metallic liquid droplets in the CAI silicate liquid. The bulk compositions of PGE in these precursor alloys reflects an early stage of condensation from the solar nebula and the partitioning of V between the precursor alloys and CAI silicate liquid reflects the reducing nebular conditions under which CAI were melted. The individual mineral phases now observed in opaque assemblages do not preserve an independent history prior to CAI melting and crystallization, but instead provide important information on the post-accretionary history of C3V meteorites and allow the quantification of the temperature, sulfur fugacity and oxygen fugacity of cooling planetary environments. This contrasts with previous models that called upon the formation of opaque assemblages by aggregation of phases that formed independently under highly variable conditions in the solar nebula prior to the crystallization of CAI.

Analytical studies were carried out on PGE-rich phases from meteorites and the products of synthetic experiments using traditional electron microprobe x-ray analytical techniques. The concentrations of PGE in common minerals from meteorites and terrestrial rocks are far below the ~100 ppm detection limit of the electron microprobe. This has limited the scope of analytical studies to the very few cases where PGE are unusually enriched. To study the distribution of PGE in common minerals will require an in situ analytical technique with much lower detection limits than any methods currently in use. To overcome this limitation, resonance ionization of sputtered atoms was investigated for use as an ultrasensitive in situ analytical technique for the analysis of PGE. The mass spectrometric analysis of Os and Re was investigated using a pulsed primary Ar⁺ ion beam to provide sputtered atoms for resonance ionization mass spectrometry. An ionization scheme for Os that utilizes three resonant energy levels (including an autoionizing energy level) was investigated and found to have superior sensitivity and selectivity compared to nonresonant and one and two energy level resonant ionization schemes. An elemental selectivity for Os over Re of ≥ 10³ was demonstrated. It was found that detuning the ionizing laser from the autoionizing energy level to an arbitrary region in the ionization continuum resulted in a five-fold decrease in signal intensity and a ten-fold decrease in elemental selectivity. Osmium concentrations in synthetic metals and iron meteorites were measured to demonstrate the analytical capabilities of the technique. A linear correlation between Os⁺ signal intensity and the known Os concentration was observed over a range of nearly 10⁴ in Os concentration with an accuracy of ~ ±10%, a millimum detection limit of 7 parts per billion atomic, and a useful yield of 1%. Resonance ionization of sputtered atoms samples the dominant neutral-fraction of sputtered atoms and utilizes multiphoton resonance ionization to achieve high sensitivity and to eliminate atomic and molecular interferences. Matrix effects should be small compared to secondary ion mass spectrometry because ionization occurs in the gas phase and is largely independent of the physical properties of the matrix material. Resonance ionization of sputtered atoms can be applied to in situ chemical analysis of most high ionization potential elements (including all of the PGE) in a wide range of natural and synthetic materials. The high useful yield and elemental selectivity of this method should eventually allow the in situ measurement of Os isotope ratios in some natural samples and in sample extracts enriched in PGE by fire assay fusion.

Phase equilibria and diffusion experiments have provided the basis for a reinterpretation of the origin of opaque assemblages in CAI and have yielded quantitative information on conditions in the primitive solar nebula and cooling planetary environments. Development of the method of resonance ionization of sputtered atoms for the analysis of Os has shown that this technique has wide applications in geochemistry and will for the first time allow in situ studies of the distribution of PGE at the low concentration levels at which they occur in common minerals.

Advanced Monte Carlo Simulation and Machine Learning for Frequency Domain Optical Coherence Tomography

Optical Coherence Tomography(OCT) is a popular, rapidly growing imaging technique with an increasing number of bio-medical applications due to its noninvasive nature. However, there are three major challenges in understanding and improving an OCT system: (1) Obtaining an OCT image is not easy. It either takes a real medical experiment or requires days of computer simulation. Without much data, it is difficult to study the physical processes underlying OCT imaging of different objects simply because there aren't many imaged objects. (2) Interpretation of an OCT image is also hard. This challenge is more profound than it appears. For instance, it would require a trained expert to tell from an OCT image of human skin whether there is a lesion or not. This is expensive in its own right, but even the expert cannot be sure about the exact size of the lesion or the width of the various skin layers. The take-away message is that analyzing an OCT image even from a high level would usually require a trained expert, and pixel-level interpretation is simply unrealistic. The reason is simple: we have OCT images but not their underlying ground-truth structure, so there is nothing to learn from. (3) The imaging depth of OCT is very limited (millimeter or sub-millimeter on human tissues). While OCT utilizes infrared light for illumination to stay noninvasive, the downside of this is that photons at such long wavelengths can only penetrate a limited depth into the tissue before getting back-scattered. To image a particular region of a tissue, photons first need to reach that region. As a result, OCT signals from deeper regions of the tissue are both weak (since few photons reached there) and distorted (due to multiple scatterings of the contributing photons). This fact alone makes OCT images very hard to interpret.

This thesis addresses the above challenges by successfully developing an advanced Monte Carlo simulation platform which is 10000 times faster than the state-of-the-art simulator in the literature, bringing down the simulation time from 360 hours to a single minute. This powerful simulation tool not only enables us to efficiently generate as many OCT images of objects with arbitrary structure and shape as we want on a common desktop computer, but it also provides us the underlying ground-truth of the simulated images at the same time because we dictate them at the beginning of the simulation. This is one of the key contributions of this thesis. What allows us to build such a powerful simulation tool includes a thorough understanding of the signal formation process, clever implementation of the importance sampling/photon splitting procedure, efficient use of a voxel-based mesh system in determining photon-mesh interception, and a parallel computation of different A-scans that consist a full OCT image, among other programming and mathematical tricks, which will be explained in detail later in the thesis.

Next we aim at the inverse problem: given an OCT image, predict/reconstruct its ground-truth structure on a pixel level. By solving this problem we would be able to interpret an OCT image completely and precisely without the help from a trained expert. It turns out that we can do much better. For simple structures we are able to reconstruct the ground-truth of an OCT image more than 98% correctly, and for more complicated structures (e.g., a multi-layered brain structure) we are looking at 93%. We achieved this through extensive uses of Machine Learning. The success of the Monte Carlo simulation already puts us in a great position by providing us with a great deal of data (effectively unlimited), in the form of (image, truth) pairs. Through a transformation of the high-dimensional response variable, we convert the learning task into a multi-output multi-class classification problem and a multi-output regression problem. We then build a hierarchy architecture of machine learning models (committee of experts) and train different parts of the architecture with specifically designed data sets. In prediction, an unseen OCT image first goes through a classification model to determine its structure (e.g., the number and the types of layers present in the image); then the image is handed to a regression model that is trained specifically for that particular structure to predict the length of the different layers and by doing so reconstruct the ground-truth of the image. We also demonstrate that ideas from Deep Learning can be useful to further improve the performance.

It is worth pointing out that solving the inverse problem automatically improves the imaging depth, since previously the lower half of an OCT image (i.e., greater depth) can be hardly seen but now becomes fully resolved. Interestingly, although OCT signals consisting the lower half of the image are weak, messy, and uninterpretable to human eyes, they still carry enough information which when fed into a well-trained machine learning model spits out precisely the true structure of the object being imaged. This is just another case where Artificial Intelligence (AI) outperforms human. To the best knowledge of the author, this thesis is not only a success but also the first attempt to reconstruct an OCT image at a pixel level. To even give a try on this kind of task, it would require fully annotated OCT images and a lot of them (hundreds or even thousands). This is clearly impossible without a powerful simulation tool like the one developed in this thesis.