Learning from Geometry

Subspaces and manifolds are two powerful models for high dimensional signals. Subspaces model linear correlation and are a good fit to signals generated by physical systems, such as frontal images of human faces and multiple sources impinging at an antenna array. Manifolds model sources that are not linearly correlated, but where signals are determined by a small number of parameters. Examples are images of human faces under different poses or expressions, and handwritten digits with varying styles. However, there will always be some degree of model mismatch between the subspace or manifold model and the true statistics of the source. This dissertation exploits subspace and manifold models as prior information in various signal processing and machine learning tasks.

A near-low-rank Gaussian mixture model measures proximity to a union of linear or affine subspaces. This simple model can effectively capture the signal distribution when each class is near a subspace. This dissertation studies how the pairwise geometry between these subspaces affects classification performance. When model mismatch is vanishingly small, the probability of misclassification is determined by the product of the sines of the principal angles between subspaces. When the model mismatch is more significant, the probability of misclassification is determined by the sum of the squares of the sines of the principal angles. Reliability of classification is derived in terms of the distribution of signal energy across principal vectors. Larger principal angles lead to smaller classification error, motivating a linear transform that optimizes principal angles. This linear transformation, termed TRAIT, also preserves some specific features in each class, being complementary to a recently developed Low Rank Transform (LRT). Moreover, when the model mismatch is more significant, TRAIT shows superior performance compared to LRT.

The manifold model enforces a constraint on the freedom of data variation. Learning features that are robust to data variation is very important, especially when the size of the training set is small. A learning machine with large numbers of parameters, e.g., deep neural network, can well describe a very complicated data distribution. However, it is also more likely to be sensitive to small perturbations of the data, and to suffer from suffer from degraded performance when generalizing to unseen (test) data.

From the perspective of complexity of function classes, such a learning machine has a huge capacity (complexity), which tends to overfit. The manifold model provides us with a way of regularizing the learning machine, so as to reduce the generalization error, therefore mitigate overfiting. Two different overfiting-preventing approaches are proposed, one from the perspective of data variation, the other from capacity/complexity control. In the first approach, the learning machine is encouraged to make decisions that vary smoothly for data points in local neighborhoods on the manifold. In the second approach, a graph adjacency matrix is derived for the manifold, and the learned features are encouraged to be aligned with the principal components of this adjacency matrix. Experimental results on benchmark datasets are demonstrated, showing an obvious advantage of the proposed approaches when the training set is small.

Stochastic optimization makes it possible to track a slowly varying subspace underlying streaming data. By approximating local neighborhoods using affine subspaces, a slowly varying manifold can be efficiently tracked as well, even with corrupted and noisy data. The more the local neighborhoods, the better the approximation, but the higher the computational complexity. A multiscale approximation scheme is proposed, where the local approximating subspaces are organized in a tree structure. Splitting and merging of the tree nodes then allows efficient control of the number of neighbourhoods. Deviation (of each datum) from the learned model is estimated, yielding a series of statistics for anomaly detection. This framework extends the classical {\em changepoint detection} technique, which only works for one dimensional signals. Simulations and experiments highlight the robustness and efficacy of the proposed approach in detecting an abrupt change in an otherwise slowly varying low-dimensional manifold.

On surfaces with prescribed shape operator

The problem of immersing a simply connected surface with a prescribed shape operator is discussed. I show that, aside from some special degenerate cases, such as when the shape operator can be realized by a surface with one family of principal curves being geodesic, the space of such realizations is a convex set in an affine space of dimension at most 3. The cases where this maximum dimension of realizability is achieved are analyzed and it is found that there are two such families of shape operators, one depending essentially on three arbitrary functions of one variable and another depending essentially on two arbitrary functions of one variable. The space of realizations is discussed in each case, along with some of their remarkable geometric properties. Several explicit examples are constructed.

Age model and pollen analysis from site GIK16867 in the northern Angola Basin

Sporomorphs and dinoflagellate cysts from site GIK16867 in the northern Angola Basin record the vegetation history of the West African forest during the last 700 ka in relation to changes in salinity and productivity of the eastern Gulf of Guinea. During most cool and cold periods, the Afromontane forest, rather than the open grass-rich dry forest, expanded to lower altitudes partly replacing the lowland rain forest of the borderlands east of the Gulf of Guinea. Except in Stage 3, when oceanic productivity was high during a period of decreased atmospheric circulation, high oceanic productivity is correlated to strong winds. The response of marine productivity in the course of a climatic cycle, however, is earlier than that of wind vigour and makes wind-stress-induced oceanic upwelling in the area less likely. Monsoon variation is well illustrated by the pollen record of increased lowland rain forest that is paired to the dinoflagellate cyst record of decreased salinity forced by increased precipitation and run-off.

Pollen flux off Cape Blanc

Fluxes of airborne freshwater diatoms (FD), phytoliths (PH), and pollen grains (PO) collected with sediment traps off Cape Blanc, northwest Africa, from 1988 till 1991 are presented. Both continental rainfall variations and wind mean strength and direction play a key role in the temporal fluctuations of the fluxes of eolian traces in the pelagic realm. Drier conditions in Northern Africa in 1987 could have preceded the high lithogenic input and moderate FD flux in 1988. The PH peak in summer 1988 was probably caused by increased wind velocity. Wetter rainy seasons of 1988/89 might have promoted a significant pollen production in summer 1989, and FD in late 1989 and early 1990, as well as contributed to the reduction of the lithogenic flux in 1989/90. Decreased fluxes of FD, PH and PO, and higher contribution of the 6-11 µm lithogenic fraction in 1991 would mainly reflect minor intensity and decreased amount of continental trade winds. Air-mass backward trajectories confirm that the Saharan Air Layer is predominantly involved in the spring/summer transport. Trade winds play a decisive role in the fall/winter months, but also contribute to the transport during late spring/summer. Origin of wind trajectories does not support a direct relationship between transporting wind-layers and material source areas in Northern Africa. High winter fluxes of eolian tracers and high amount of trade winds with continental origin in summer warn against a simplistic interpretation of the seasonal eolian signal preserved in the sediments off Cape Blanc, and the wind layer involved in its transport.

Pollen analyses of sediment core GeoB1711-4 in the eastern South Atlantic

Influx of aeolian pollen trapped in marine sediments off Namibia provides a wind variation record for the last 135 kyr. The influx of major pollen components is derived from the southwest African desert/semi-desert zone and shows six periods during which enhanced southeast trade winds contributed to strong upwelling and reduced sea surface temperatures. The most prominent of these occurred during 17-23 cal. kyr, 42-56 kyr and before 130 kyr B.P. Correspondence between the pollen influx record and the Vostok deuterium isotope record suggests that pronounced glacial Antarctic cooling was accompanied by intensification of the southeast trades throughout the Late Quaternary. However, during 42-23 kyr B.P. the combination of strong Antarctic glaciation with a decrease of wind zonality induced by low latitude precessional insolation changes caused strong alongshore winds and Ekman pumping that resulted in strong upwelling and reduced sea surface temperatures without pollen influx enhancement.

Oligocene to early Miocene benthic foraminifera in the eastern Equatorial Pacific

We investigated Oligocene and early Miocene benthic foraminiferal faunas (> 105 µm in size) from Ocean Drilling Program (Leg 199) Site 1218 (4826 m water depth and ~3300 to ~4000 m paleo-water depth) and Site 1219 (5063 m water depth and ~4200 to ~4400 m paleo-water depth) to understand the response of abyssal benthic foraminifera to mid-Oligocene glacial events in the eastern Equatorial Pacific Ocean. Two principal factor assemblages were recognized. The Factor 1 assemblage (common Nuttallides umbonifer) is related to either an influx of the Southern Component Water (SCW), possibly carbonate undersaturated, or a decrease in seasonality of the food supply from the surface ocean. The Factor 2 assemblage is characterized by typical deep-sea taxa living under variable trophic conditions, possibly with a seasonal component in food supply. The occurrence of abyssal benthic foraminifera faunas during the mid-Oligocene depends on either the effect of SCW or the seasonality of food resources. The Factor 1 assemblage was most common near 76Ol-C11r, 73Ol-C10rn and 67Ol-C9n (ca. 30.2, 29.1 and 26.8 Ma respectively by Pälike et al. (2006, doi:10.1126/science.1133822)). This indicates that the effect of SCW increased or the seasonal input of food from the surface ocean to benthic environments was weakened close to these glacial events. In contrast, the huge export flux of small biogenic carbonate particles close to these glacial events might be responsible for carbonate-rich sediments buffering carbonate undersaturation. Changes in deep-water masses or the periodicity of food supply from the surface ocean and variation in surface carbonate production affected by orbital forcing had an impact on the mid-Oligocene faunas of abyssal benthic foraminifera around the intervals of glacial events in the eastern Equatorial Pacific Ocean. The Factor 1 assemblage decreased sharply at ? 30 Ma (29.8 Ma by Pälike et al. (2006), 30.0 Ma by CK95) and returned to dominance after ? 29 Ma (28.6 Ma by Pälike et al. (2006), 28.8 Ma by CK95). It is likely that the effect of SCW (possibly carbonate undersaturated) has intensified since the late Oligocene. The faunal transition of benthic foraminifera in the eastern Equatorial Pacific Ocean at ~29 Ma might be attributable to the influence of Northern Component Water (NCW) input to the Southern Ocean and the subsequent formation of SCW at about that time.

Relative abundance of euphausiids (Crustacea) in water samples of the Arabian Sea (Table 2)

In the present paper, the ecology and feeding habits of euphausiids are described. The samples were taken at the time of the NE-monsoon (1964/65) by R. V. "Meteor" in the Arabian Sea and adjacent waters. 24 species were determined. According to distribution of the species, the following marine areas can be distinguished: Arabian Sea: 24 species, dominant are Euphausia diomedeae, E. tenera, E. distinguenda, Stylocheiron carinatum. Gulf of Aden: 10 species, dominant are Euphausia diomedeae, E. distinguenda. Red Sea: 6 species, dominant are Euphausia diomedeae, E. distinguenda. Gulf of Oman : 5 Species, dominant are Euphausia distinguenda, Pseudeupbaufia latifrons. Persian Gulf: 1 species - Pseudeuphausia latifrons. The total number of euphausiids indicate the biomass of this group. High densities of euphausiids (200-299 and > 300 individuals/100 m**3) occur in the innermost part of the Gulf cf Aden, in the area south of the equator near the African east coast, near Karachi (Indian west coast) and in the Persian Gulf. Comparison with data relating to production biology confirms that these are eutrophic zones which coincide with areas in which upwelling occurs at the time of the NE-monsoon. The central part of the Arabian Sea differs from adjacent waters by virtue of less dense euphausiid populations (> 199 individuals/100 m**3). Measurements relating to production biology demonstrate a relatively low concentration of primary food sources. Food material was ascertained by analysis of stomach content. The following omnivorous species were examined: Euphausia diomedeae, E. distinguenda, E. tenera, Pseudeuphausia latifrons and Thysanopoda tricuspidata. Apart from crustacean remains large numbers of Foraminifera, Radiolaria, tintinnids, dinoflagellates were found in the stomachs. Quantitatively crustaceans form the most important item in the diet. Food selection on the basis of size and form appears to be restricted to certain genera of tintinnids. The genera Stylocheiron and Nematoscelis are predators. Only crustacean remains were found in the stomachs of Stylocheiron abbreviatum, whereas Radiolaria, Foraminifera and tintinnids occurred to some extent in Nematasceli sp. Different euphausiids in the food chain in the Arabian Sea. In omnivorous species the position is variable, since they not only feed by filtering autotrophic and heterotrophic Protista, but also by predation on zooplankton. Carnivorous species without filtering apparatus feed exclusively on zooplankton of the size of copepods. Only these species are well established as occupying a higher position in the food chain. The parasitic protozoan Tbalassomyces fagei was found on Euphausia diomedeae, E. fenera, E. distinguenda and E. sanzoi.

Pollen analysis of sediment core GeoB1023-5

A high resolution marine pollen record from site GeoB1023, west of the northern Namib desert provides data on vegetation and climate change for the last 21 ka at an average resolution of 185 y. Pollen and spores are mainly delivered to the site by the Cunene river and by surface and mid-tropospheric wind systems. The main pollen source areas are located between 13°S and 21°S, which includes the northern Namib desert and semi-desert, the Angola-northern Namibian highland, and the north-western Kalahari. The pollen spectra reflect environmental changes in the region. The last glacial maximum (LGM) was characterised by colder and more arid conditions than at present, when a vegetation with temperate elements such as Asteroideae, Ericaceae, and Restionaceae grew north of 21°S. At 17.5 ka cal. B.P., an amelioration both in temperature and humidity terminated the LGM but, in the northern Kalahari, mean annual rainfall in the interval 17.5-14.4 ka cal. B.P. was probably 100-150 mm lower than at present (400-500 mm/y). The Late-glacial to early Holocene transition includes two arid periods, i.e. 14.4-12.5 and 10.9-9.3 ka cal. B.P. The last part of the former period may be correlated with the Younger Dryas. The warmest and most humid period in the Holocene occurred between 6.3 and 4.8 ka cal. B.P. During the last 2000 years, human impact, as reflected by indications of deforestation, enhanced burning and overgrazing, progressively intensified.