Learning Nonlinear Visual Processing from Natural Images

The paradigm of computational vision hypothesizes that any visual function -- such as the recognition of your grandparent -- can be replicated by computational processing of the visual input. What are these computations that the brain performs? What should or could they be? Working on the latter question, this dissertation takes the statistical approach, where the suitable computations are attempted to be learned from the natural visual data itself. In particular, we empirically study the computational processing that emerges from the statistical properties of the visual world and the constraints and objectives specified for the learning process. This thesis consists of an introduction and 7 peer-reviewed publications, where the purpose of the introduction is to illustrate the area of study to a reader who is not familiar with computational vision research. In the scope of the introduction, we will briefly overview the primary challenges to visual processing, as well as recall some of the current opinions on visual processing in the early visual systems of animals. Next, we describe the methodology we have used in our research, and discuss the presented results. We have included some additional remarks, speculations and conclusions to this discussion that were not featured in the original publications. We present the following results in the publications of this thesis. First, we empirically demonstrate that luminance and contrast are strongly dependent in natural images, contradicting previous theories suggesting that luminance and contrast were processed separately in natural systems due to their independence in the visual data. Second, we show that simple cell -like receptive fields of the primary visual cortex can be learned in the nonlinear contrast domain by maximization of independence. Further, we provide first-time reports of the emergence of conjunctive (corner-detecting) and subtractive (opponent orientation) processing due to nonlinear projection pursuit with simple objective functions related to sparseness and response energy optimization. Then, we show that attempting to extract independent components of nonlinear histogram statistics of a biologically plausible representation leads to projection directions that appear to differentiate between visual contexts. Such processing might be applicable for priming, \ie the selection and tuning of later visual processing. We continue by showing that a different kind of thresholded low-frequency priming can be learned and used to make object detection faster with little loss in accuracy. Finally, we show that in a computational object detection setting, nonlinearly gain-controlled visual features of medium complexity can be acquired sequentially as images are encountered and discarded. We present two online algorithms to perform this feature selection, and propose the idea that for artificial systems, some processing mechanisms could be selectable from the environment without optimizing the mechanisms themselves. In summary, this thesis explores learning visual processing on several levels. The learning can be understood as interplay of input data, model structures, learning objectives, and estimation algorithms. The presented work adds to the growing body of evidence showing that statistical methods can be used to acquire intuitively meaningful visual processing mechanisms. The work also presents some predictions and ideas regarding biological visual processing.

Diagnostic Tests Based on Quantile Residuals for Nonlinear Time Series Models

This thesis studies quantile residuals and uses different methodologies to develop test statistics that are applicable in evaluating linear and nonlinear time series models based on continuous distributions. Models based on mixtures of distributions are of special interest because it turns out that for those models traditional residuals, often referred to as Pearson's residuals, are not appropriate. As such models have become more and more popular in practice, especially with financial time series data there is a need for reliable diagnostic tools that can be used to evaluate them. The aim of the thesis is to show how such diagnostic tools can be obtained and used in model evaluation. The quantile residuals considered here are defined in such a way that, when the model is correctly specified and its parameters are consistently estimated, they are approximately independent with standard normal distribution. All the tests derived in the thesis are pure significance type tests and are theoretically sound in that they properly take the uncertainty caused by parameter estimation into account. -- In Chapter 2 a general framework based on the likelihood function and smooth functions of univariate quantile residuals is derived that can be used to obtain misspecification tests for various purposes. Three easy-to-use tests aimed at detecting non-normality, autocorrelation, and conditional heteroscedasticity in quantile residuals are formulated. It also turns out that these tests can be interpreted as Lagrange Multiplier or score tests so that they are asymptotically optimal against local alternatives. Chapter 3 extends the concept of quantile residuals to multivariate models. The framework of Chapter 2 is generalized and tests aimed at detecting non-normality, serial correlation, and conditional heteroscedasticity in multivariate quantile residuals are derived based on it. Score test interpretations are obtained for the serial correlation and conditional heteroscedasticity tests and in a rather restricted special case for the normality test. In Chapter 4 the tests are constructed using the empirical distribution function of quantile residuals. So-called Khmaladze s martingale transformation is applied in order to eliminate the uncertainty caused by parameter estimation. Various test statistics are considered so that critical bounds for histogram type plots as well as Quantile-Quantile and Probability-Probability type plots of quantile residuals are obtained. Chapters 2, 3, and 4 contain simulations and empirical examples which illustrate the finite sample size and power properties of the derived tests and also how the tests and related graphical tools based on residuals are applied in practice.

Ecological restoration of forests in Fennoscandia : Defining reference stand structures and immediate effects of restoration.

The first aim of this thesis was to explore the structural characteristics of near-natural forests and to quantify how human utilization has changed them. For this, we examined the stand characteristics in Norway spruce Picea abies (L.) Karst-dominated old-growth stands in northwestern Russia and in old Scots pine Pinus sylvestris L.-dominated stands in three regions from southern Finland to northwestern Russia. In the second study, we also compared stands with different degrees of human impact, from near-natural stands and stands selectively cut in the past to managed stands. Secondly, we used an experimental approach to study the short-term effects of different restorative treatments on forest structure and regeneration in managed Picea abies stands in southern Finland. Restorative treatments consisted of a partial cut combined with three levels of coarse woody debris retention, and a fire/no-fire treatment. In addition, we examined burned and unburned reference stands without cutting treatments. Results from near-natural Picea abies forests emphasize the dynamic character of old-growth forests, the variety of late-successional forest structures, and the fact that extended time periods are needed to attain certain late-successional stages with specific structural and habitat attributes, such as large-diameter deciduous trees and a variety of deadwood. The results from old Pinus sylvestris-dominated forests showed that human impact in the form of forest utilization and fire exclusion has strongly modified and reduced the structural complexity of stands. Consequently, small protected forest fragments in Finland may not serve as valid natural reference areas for forest restoration. However, results from the restoration experiment showed that early-successional natural stand characteristics can be restored to structurally impoverished managed Picea abies stands, despite a significant portion of wood volume being harvested. A variety of restoration methods is needed, due to differences in the condition of the forest when restoration is initiated and the variety of successional stages of forest structures after anthropogenic and natural disturbances. Keywords: dead wood, disturbance dynamic, fire, near-natural stand, rehabilitation, succession

Food and Indoor Air Isolated Bacillus Non-Protein Toxins: : Structures, Physico-Chemical Properties and Mechanisms of Effects on Eukaryotic Cells

We report here the structures and properties of heat-stable, non-protein, and mammalian cell-toxic compounds produced by spore-forming bacilli isolated from indoor air of buildings and from food. Little information is available on the effects and occurrence of heat-stable non-protein toxins produced by bacilli in moisture-damaged buildings. Bacilli emit spores that move in the air and can serve as the carriers of toxins, in a manner similar to that of the spores of toxic fungi found in contaminated indoor air. Bacillus spores in food cause problems because they tolerate the temperatures applied in food manufacture and the spores later initiate growth when food storage conditions are more favorable. Detection of the toxic compounds in Bacillus is based on using the change in mobility of boar spermatozoa as an indicator of toxic exposure. GC, LC, MS, and nuclear magnetic resonance NMR spectroscopy were used for purification, detection, quantitation, and analysis of the properties and structures of the compounds. Toxicity and the mechanisms of toxicity of the compounds were studied using boar spermatozoa, feline lung cells, human neural cells, and mitochondria isolated from rat liver. The ionophoric properties were studied using the BLM (black-lipid membrane) method. One novel toxin, forming ion channels permeant to K+ > Na+ > Ca2+, was found and named amylosin. It is produced by B. amyloliquefaciens isolated from indoor air of moisture-damaged buildings. Amylosin was purified with an RP-HPLC and a monoisotopic mass of 1197 Da was determined with ESI-IT-MS. Furthermore, acid hydrolysis of amylosin followed by analysis of the amino acids with the GS-MS showed that it was a peptide. The presence of a chromophoric polyene group was found using a NMR spectroscopy. The quantification method developed for amylosin based on RP-HPLC-UV, using the macrolactone polyene, amphotericin B (MW 924), as a reference compound. The B. licheniformis strains isolated from a food poisoning case produced a lipopeptide, lichenysin A, that ruptured mammalian cell membranes and was purified with a LC. Lichenysin A was identified by its protonated molecules and sodium- and potassium- cationized molecules with MALDI-TOF-MS. Its protonated forms were observed at m/z 1007, 1021 and 1035. The amino acids of lichenysin A were analyzed with ESI-TQ-MS/MS and, after acid hydrolysis, the stereoisomeric forms of the amino acids with RP-HPLC. The indoor air isolates of the strain of B. amyloliquefaciens produced not only amylosin but also lipopeptides: the cell membrane-damaging surfactin and the fungicidal fengycin. They were identified with ESI-IT-MS observing their protonated molecules, the sodium- and potassium-cationized molecules and analysing the MS/MS spectra. The protonated molecules of surfactin and fengycin showed m/z values of 1009, 1023, and 1037 and 1450, 1463, 1493, and 1506, respectively. Cereulide (MW 1152) was purified with RP-HPLC from a food poisoning strain of B. cereus. Cereulide was identified with ESI-TQ-MS according to the protonated molecule observed at m/z 1154 and the ammonium-, sodium- and potassium-cationized molecules observed at m/z 1171, 1176, and 1192, respectively. The fragment ions of the MS/MS spectrum obtained from the protonated molecule of cereulide at m/z 1154 were also interpreted. We developed a quantification method for cereulide, using RP-HPLC-UV and valinomycin (MW 1110, which structurally resembles cereulide) as the reference compound. Furthermore, we showed empirically, using the BLM method, that the emetic toxin cereulide is a specific and effective potassium ionophore of whose toxicity target is especially the mitochondria.

Discovering hidden structures in molecular data using a Bayesian partition model approach

Advancements in the analysis techniques have led to a rapid accumulation of biological data in databases. Such data often are in the form of sequences of observations, examples including DNA sequences and amino acid sequences of proteins. The scale and quality of the data give promises of answering various biologically relevant questions in more detail than what has been possible before. For example, one may wish to identify areas in an amino acid sequence, which are important for the function of the corresponding protein, or investigate how characteristics on the level of DNA sequence affect the adaptation of a bacterial species to its environment. Many of the interesting questions are intimately associated with the understanding of the evolutionary relationships among the items under consideration. The aim of this work is to develop novel statistical models and computational techniques to meet with the challenge of deriving meaning from the increasing amounts of data. Our main concern is on modeling the evolutionary relationships based on the observed molecular data. We operate within a Bayesian statistical framework, which allows a probabilistic quantification of the uncertainties related to a particular solution. As the basis of our modeling approach we utilize a partition model, which is used to describe the structure of data by appropriately dividing the data items into clusters of related items. Generalizations and modifications of the partition model are developed and applied to various problems. Large-scale data sets provide also a computational challenge. The models used to describe the data must be realistic enough to capture the essential features of the current modeling task but, at the same time, simple enough to make it possible to carry out the inference in practice. The partition model fulfills these two requirements. The problem-specific features can be taken into account by modifying the prior probability distributions of the model parameters. The computational efficiency stems from the ability to integrate out the parameters of the partition model analytically, which enables the use of efficient stochastic search algorithms.

Paleomagnetic and rock magnetic study with emphasis on the Precambrian intrusions and impact structures in Fennoscandia and South Africa

The importance of supercontinents in our understanding of the geological evolution of the planet Earth has been recently emphasized. The role of paleomagnetism in reconstructing lithospheric blocks in their ancient paleopositions is vital. Paleomagnetism is the only quantitative tool for providing ancient latitudes and azimuthal orientations of continents. It also yields information of content of the geomagnetic field in the past. In order to obtain a continuous record on the positions of continents, dated intrusive rocks are required in temporal progression. This is not always possible due to pulse-like occurrences of dykes. In this work we demonstrate that studies of meteorite impact-related rocks may fill some gaps in the paleomagnetic record. This dissertation is based on paleomagnetic and rock magnetic data obtained from samples of the Jänisjärvi impact structure (Russian Karelia, most recent 40Ar-39Ar age of 682 Ma), the Salla diabase dyke (North Finland, U-Pb 1122 Ma), the Valaam monzodioritic sill (Russian Karelia, U-Pb 1458 Ma), and the Vredefort impact structure (South Africa, 2023 Ma). The paleomagnetic study of Jänisjärvi samples was made in order to obtain a pole for Baltica, which lacks paleomagnetic data from 750 to ca. 600 Ma. The position of Baltica at ca. 700 Ma is relevant in order to verify whether the supercontinent Rodinia was already fragmented. The paleomagnetic study of the Salla dyke was conducted to examine the position of Baltica at the onset of supercontinent Rodinia's formation. The virtual geomagnetic pole (VGP) from Salla dyke provides hints that the Mesoproterozoic Baltica - Laurentia unity in the Hudsonland (Columbia, Nuna) supercontinent assembly may have lasted until 1.12 Ga. Moreover, the new VGP of Salla dyke provides new constraint on the timing of the rotation of Baltica relative to Laurentia (e.g. Gower et al., 1990). A paleomagnetic study of the Valaam sill was carried out in order to shed light into the question of existence of Baltica-Laurentia unity in the supercontinent Hudsonland. Combined with results from dyke complex of the Lake Ladoga region (Schehrbakova et al., 2008) a new robust paleomagnetic pole for Baltica is obtained. This pole places Baltica on a latitude of 10°. This low latitude location is supported also by Mesoproterozoic 1.5 1.3 Ga red-bed sedimentation (for example the Satakunta sandstone). The Vredefort impactite samples provide a well dated (2.02 Ga) pole for the Kaapvaal Craton. Rock magnetic data reveal unusually high Koenigsberger ratios (Q values) in all studied lithologies of the Vredefort dome. The high Q values are now first time also seen in samples from the Johannesburg Dome (ca. 120 km away) where there is no impact evidence. Thus, a direct causative link of high Q values to the Vredefort impact event can be ruled out.

Homogeneous model theory of metric structures

This thesis studies homogeneous classes of complete metric spaces. Over the past few decades model theory has been extended to cover a variety of nonelementary frameworks. Shelah introduced the abstact elementary classes (AEC) in the 1980s as a common framework for the study of nonelementary classes. Another direction of extension has been the development of model theory for metric structures. This thesis takes a step in the direction of combining these two by introducing an AEC-like setting for studying metric structures. To find balance between generality and the possibility to develop stability theoretic tools, we work in a homogeneous context, thus extending the usual compact approach. The homogeneous context enables the application of stability theoretic tools developed in discrete homogeneous model theory. Using these we prove categoricity transfer theorems for homogeneous metric structures with respect to isometric isomorphisms. We also show how generalized isomorphisms can be added to the class, giving a model theoretic approach to, e.g., Banach space isomorphisms or operator approximations. The novelty is the built-in treatment of these generalized isomorphisms making, e.g., stability up to perturbation the natural stability notion. With respect to these generalized isomorphisms we develop a notion of independence. It behaves well already for structures which are omega-stable up to perturbation and coincides with the one from classical homogeneous model theory over saturated enough models. We also introduce a notion of isolation and prove dominance for it.

Mathematical models on the impact of noise and dyadic molecular structures on the properties of a cardiac myocyte

In cardiac myocytes (heart muscle cells), coupling of electric signal known as the action potential to contraction of the heart depends crucially on calcium-induced calcium release (CICR) in a microdomain known as the dyad. During CICR, the peak number of free calcium ions (Ca) present in the dyad is small, typically estimated to be within range 1-100. Since the free Ca ions mediate CICR, noise in Ca signaling due to the small number of free calcium ions influences Excitation-Contraction (EC) coupling gain. Noise in Ca signaling is only one noise type influencing cardiac myocytes, e.g., ion channels playing a central role in action potential propagation are stochastic machines, each of which gates more or less randomly, which produces gating noise present in membrane currents. How various noise sources influence macroscopic properties of a myocyte, how noise is attenuated and taken advantage of are largely open questions. In this thesis, the impact of noise on CICR, EC coupling and, more generally, macroscopic properties of a cardiac myocyte is investigated at multiple levels of detail using mathematical models. Complementarily to the investigation of the impact of noise on CICR, computationally-efficient yet spatially-detailed models of CICR are developed. The results of this thesis show that (1) gating noise due to the high-activity mode of L-type calcium channels playing a major role in CICR may induce early after-depolarizations associated with polymorphic tachycardia, which is a frequent precursor to sudden cardiac death in heart failure patients; (2) an increased level of voltage noise typically increases action potential duration and it skews distribution of action potential durations toward long durations in cardiac myocytes; and that (3) while a small number of Ca ions mediate CICR, Excitation-Contraction coupling is robust against this noise source, partly due to the shape of ryanodine receptor protein structures present in the cardiac dyad.

Tannakian Formalism Applied to the Category of Mixed Hodge Structures

Many problems in analysis have been solved using the theory of Hodge structures. P. Deligne started to treat these structures in a categorical way. Following him, we introduce the categories of mixed real and complex Hodge structures. Category of mixed Hodge structures over the field of real or complex numbers is a rigid abelian tensor category, and in fact, a neutral Tannakian category. Therefore it is equivalent to the category of representations of an affine group scheme. The direct sums of pure Hodge structures of different weights over real or complex numbers can be realized as a representation of the torus group, whose complex points is the Cartesian product of two punctured complex planes. Mixed Hodge structures turn out to consist of information of a direct sum of pure Hodge structures of different weights and a nilpotent automorphism. Therefore mixed Hodge structures correspond to the representations of certain semidirect product of a nilpotent group and the torus group acting on it.

Mining Sequential Data – in Search of Segmental Structures

Segmentation is a data mining technique yielding simplified representations of sequences of ordered points. A sequence is divided into some number of homogeneous blocks, and all points within a segment are described by a single value. The focus in this thesis is on piecewise-constant segments, where the most likely description for each segment and the most likely segmentation into some number of blocks can be computed efficiently. Representing sequences as segmentations is useful in, e.g., storage and indexing tasks in sequence databases, and segmentation can be used as a tool in learning about the structure of a given sequence. The discussion in this thesis begins with basic questions related to segmentation analysis, such as choosing the number of segments, and evaluating the obtained segmentations. Standard model selection techniques are shown to perform well for the sequence segmentation task. Segmentation evaluation is proposed with respect to a known segmentation structure. Applying segmentation on certain features of a sequence is shown to yield segmentations that are significantly close to the known underlying structure. Two extensions to the basic segmentation framework are introduced: unimodal segmentation and basis segmentation. The former is concerned with segmentations where the segment descriptions first increase and then decrease, and the latter with the interplay between different dimensions and segments in the sequence. These problems are formally defined and algorithms for solving them are provided and analyzed. Practical applications for segmentation techniques include time series and data stream analysis, text analysis, and biological sequence analysis. In this thesis segmentation applications are demonstrated in analyzing genomic sequences.