Approximate Approximations and a Boundary Point Method for the Linearized Stokes System

The method of approximate approximations, introduced by Maz'ya [1], can also be used for the numerical solution of boundary integral equations. In this case, the matrix of the resulting algebraic system to compute an approximate source density depends only on the position of a finite number of boundary points and on the direction of the normal vector in these points (Boundary Point Method). We investigate this approach for the Stokes problem in the whole space and for the Stokes boundary value problem in a bounded convex domain G subset R^2, where the second part consists of three steps: In a first step the unknown potential density is replaced by a linear combination of exponentially decreasing basis functions concentrated near the boundary points. In a second step, integration over the boundary partial G is replaced by integration over the tangents at the boundary points such that even analytical expressions for the potential approximations can be obtained. In a third step, finally, the linear algebraic system is solved to determine an approximate density function and the resulting solution of the Stokes boundary value problem. Even not convergent the method leads to an efficient approximation of the form O(h^2) + epsilon, where epsilon can be chosen arbitrarily small.

Zur Konvergenz der Randpunktmethode

Das von Maz'ya eingeführte Approximationsverfahren, die Methode der näherungsweisen Näherungen (Approximate Approximations), kann auch zur numerischen Lösung von Randintegralgleichungen verwendet werden (Randpunktmethode). In diesem Fall hängen die Komponenten der Matrix des resultierenden Gleichungssystems zur Berechnung der Näherung für die Dichte nur von der Position der Randpunkte und der Richtung der äußeren Einheitsnormalen in diesen Punkten ab. Dieses numerisches Verfahren wird am Beispiel des Dirichlet Problems für die Laplace Gleichung und die Stokes Gleichungen in einem beschränkten zweidimensionalem Gebiet untersucht. Die Randpunktmethode umfasst drei Schritte: Im ersten Schritt wird die unbekannte Dichte durch eine Linearkombination von radialen, exponentiell abklingenden Basisfunktionen approximiert. Im zweiten Schritt wird die Integration über den Rand durch die Integration über die Tangenten in Randpunkten ersetzt. Für die auftretende Näherungspotentiale können sogar analytische Ausdrücke gewonnen werden. Im dritten Schritt wird das lineare Gleichungssystem gelöst, und eine Näherung für die unbekannte Dichte und damit auch für die Lösung der Randwertaufgabe konstruiert. Die Konvergenz dieses Verfahrens wird für glatte konvexe Gebiete nachgewiesen.

Development and applications of density matrix functional theory of generalized Hubbard Models

In dieser Doktorarbeit wird eine akkurate Methode zur Bestimmung von Grundzustandseigenschaften stark korrelierter Elektronen im Rahmen von Gittermodellen entwickelt und angewandt. In der Dichtematrix-Funktional-Theorie (LDFT, vom englischen lattice density functional theory) ist die Ein-Teilchen-Dichtematrix γ die fundamentale Variable. Auf der Basis eines verallgemeinerten Hohenberg-Kohn-Theorems ergibt sich die Grundzustandsenergie Egs[γgs] = min° E[γ] durch die Minimierung des Energiefunktionals E[γ] bezüglich aller physikalischer bzw. repräsentativer γ. Das Energiefunktional kann in zwei Beiträge aufgeteilt werden: Das Funktional der kinetischen Energie T[γ], dessen lineare Abhängigkeit von γ genau bekannt ist, und das Funktional der Korrelationsenergie W[γ], dessen Abhängigkeit von γ nicht explizit bekannt ist. Das Auffinden präziser Näherungen für W[γ] stellt die tatsächliche Herausforderung dieser These dar. Einem Teil dieser Arbeit liegen vorausgegangene Studien zu Grunde, in denen eine Näherung des Funktionals W[γ] für das Hubbardmodell, basierend auf Skalierungshypothesen und exakten analytischen Ergebnissen für das Dimer, hergeleitet wird. Jedoch ist dieser Ansatz begrenzt auf spin-unabhängige und homogene Systeme. Um den Anwendungsbereich von LDFT zu erweitern, entwickeln wir drei verschiedene Ansätze zur Herleitung von W[γ], die das Studium von Systemen mit gebrochener Symmetrie ermöglichen. Zuerst wird das bisherige Skalierungsfunktional erweitert auf Systeme mit Ladungstransfer. Eine systematische Untersuchung der Abhängigkeit des Funktionals W[γ] von der Ladungsverteilung ergibt ähnliche Skalierungseigenschaften wie für den homogenen Fall. Daraufhin wird eine Erweiterung auf das Hubbardmodell auf bipartiten Gittern hergeleitet und an sowohl endlichen als auch unendlichen Systemen mit repulsiver und attraktiver Wechselwirkung angewandt. Die hohe Genauigkeit dieses Funktionals wird aufgezeigt. Es erweist sich jedoch als schwierig, diesen Ansatz auf komplexere Systeme zu übertragen, da bei der Berechnung von W[γ] das System als ganzes betrachtet wird. Um dieses Problem zu bewältigen, leiten wir eine weitere Näherung basierend auf lokalen Skalierungseigenschaften her. Dieses Funktional ist lokal bezüglich der Gitterplätze formuliert und ist daher anwendbar auf jede Art von geordneten oder ungeordneten Hamiltonoperatoren mit lokalen Wechselwirkungen. Als Anwendungen untersuchen wir den Metall-Isolator-Übergang sowohl im ionischen Hubbardmodell in einer und zwei Dimensionen als auch in eindimensionalen Hubbardketten mit nächsten und übernächsten Nachbarn. Schließlich entwickeln wir ein numerisches Verfahren zur Berechnung von W[γ], basierend auf exakten Diagonalisierungen eines effektiven Vielteilchen-Hamilton-Operators, welcher einen von einem effektiven Medium umgebenen Cluster beschreibt. Dieser effektive Hamiltonoperator hängt von der Dichtematrix γ ab und erlaubt die Herleitung von Näherungen an W[γ], dessen Qualität sich systematisch mit steigender Clustergröße verbessert. Die Formulierung ist spinabhängig und ermöglicht eine direkte Verallgemeinerung auf korrelierte Systeme mit mehreren Orbitalen, wie zum Beispiel auf den spd-Hamilton-Operator. Darüber hinaus berücksichtigt sie die Effekte kurzreichweitiger Ladungs- und Spinfluktuationen in dem Funktional. Für das Hubbardmodell wird die Genauigkeit der Methode durch Vergleich mit Bethe-Ansatz-Resultaten (1D) und Quanten-Monte-Carlo-Simulationen (2D) veranschaulicht. Zum Abschluss wird ein Ausblick auf relevante zukünftige Entwicklungen dieser Theorie gegeben.

A Vector Signal Processing Approach to Color

Surface (Lambertain) color is a useful visual cue for analyzing material composition of scenes. This thesis adopts a signal processing approach to color vision. It represents color images as fields of 3D vectors, from which we extract region and boundary information. The first problem we face is one of secondary imaging effects that makes image color different from surface color. We demonstrate a simple but effective polarization based technique that corrects for these effects. We then propose a systematic approach of scalarizing color, that allows us to augment classical image processing tools and concepts for multi-dimensional color signals.

Learning-Based Approach to Real Time Tracking and Analysis of Faces

This paper describes a trainable system capable of tracking faces and facialsfeatures like eyes and nostrils and estimating basic mouth features such as sdegrees of openness and smile in real time. In developing this system, we have addressed the twin issues of image representation and algorithms for learning. We have used the invariance properties of image representations based on Haar wavelets to robustly capture various facial features. Similarly, unlike previous approaches this system is entirely trained using examples and does not rely on a priori (hand-crafted) models of facial features based on optical flow or facial musculature. The system works in several stages that begin with face detection, followed by localization of facial features and estimation of mouth parameters. Each of these stages is formulated as a problem in supervised learning from examples. We apply the new and robust technique of support vector machines (SVM) for classification in the stage of skin segmentation, face detection and eye detection. Estimation of mouth parameters is modeled as a regression from a sparse subset of coefficients (basis functions) of an overcomplete dictionary of Haar wavelets.

Comparing Support Vector Machines with Gaussian Kernels to Radial Basis Function Classifiers

The Support Vector (SV) machine is a novel type of learning machine, based on statistical learning theory, which contains polynomial classifiers, neural networks, and radial basis function (RBF) networks as special cases. In the RBF case, the SV algorithm automatically determines centers, weights and threshold such as to minimize an upper bound on the expected test error. The present study is devoted to an experimental comparison of these machines with a classical approach, where the centers are determined by $k$--means clustering and the weights are found using error backpropagation. We consider three machines, namely a classical RBF machine, an SV machine with Gaussian kernel, and a hybrid system with the centers determined by the SV method and the weights trained by error backpropagation. Our results show that on the US postal service database of handwritten digits, the SV machine achieves the highest test accuracy, followed by the hybrid approach. The SV approach is thus not only theoretically well--founded, but also superior in a practical application.

Learning Linear, Sparse, Factorial Codes

In previous work (Olshausen & Field 1996), an algorithm was described for learning linear sparse codes which, when trained on natural images, produces a set of basis functions that are spatially localized, oriented, and bandpass (i.e., wavelet-like). This note shows how the algorithm may be interpreted within a maximum-likelihood framework. Several useful insights emerge from this connection: it makes explicit the relation to statistical independence (i.e., factorial coding), it shows a formal relationship to the algorithm of Bell and Sejnowski (1995), and it suggests how to adapt parameters that were previously fixed.

Vector-Based Integration of Local and Long-Range Information in Visual Cortex

Integration of inputs by cortical neurons provides the basis for the complex information processing performed in the cerebral cortex. Here, we propose a new analytic framework for understanding integration within cortical neuronal receptive fields. Based on the synaptic organization of cortex, we argue that neuronal integration is a systems--level process better studied in terms of local cortical circuitry than at the level of single neurons, and we present a method for constructing self-contained modules which capture (nonlinear) local circuit interactions. In this framework, receptive field elements naturally have dual (rather than the traditional unitary influence since they drive both excitatory and inhibitory cortical neurons. This vector-based analysis, in contrast to scalarsapproaches, greatly simplifies integration by permitting linear summation of inputs from both "classical" and "extraclassical" receptive field regions. We illustrate this by explaining two complex visual cortical phenomena, which are incompatible with scalar notions of neuronal integration.

Improving Multiclass Text Classification with the Support Vector Machine

We compare Naive Bayes and Support Vector Machines on the task of multiclass text classification. Using a variety of approaches to combine the underlying binary classifiers, we find that SVMs substantially outperform Naive Bayes. We present full multiclass results on two well-known text data sets, including the lowest error to date on both data sets. We develop a new indicator of binary performance to show that the SVM's lower multiclass error is a result of its improved binary performance. Furthermore, we demonstrate and explore the surprising result that one-vs-all classification performs favorably compared to other approaches even though it has no error-correcting properties.

Properties of Support Vector Machines

Support Vector Machines (SVMs) perform pattern recognition between two point classes by finding a decision surface determined by certain points of the training set, termed Support Vectors (SV). This surface, which in some feature space of possibly infinite dimension can be regarded as a hyperplane, is obtained from the solution of a problem of quadratic programming that depends on a regularization parameter. In this paper we study some mathematical properties of support vectors and show that the decision surface can be written as the sum of two orthogonal terms, the first depending only on the margin vectors (which are SVs lying on the margin), the second proportional to the regularization parameter. For almost all values of the parameter, this enables us to predict how the decision surface varies for small parameter changes. In the special but important case of feature space of finite dimension m, we also show that there are at most m+1 margin vectors and observe that m+1 SVs are usually sufficient to fully determine the decision surface. For relatively small m this latter result leads to a consistent reduction of the SV number.

Sparse Representations of Multiple Signals

We discuss the problem of finding sparse representations of a class of signals. We formalize the problem and prove it is NP-complete both in the case of a single signal and that of multiple ones. Next we develop a simple approximation method to the problem and we show experimental results using artificially generated signals. Furthermore,we use our approximation method to find sparse representations of classes of real signals, specifically of images of pedestrians. We discuss the relation between our formulation of the sparsity problem and the problem of finding representations of objects that are compact and appropriate for detection and classification.

From Regression to Classification in Support Vector Machines

We study the relation between support vector machines (SVMs) for regression (SVMR) and SVM for classification (SVMC). We show that for a given SVMC solution there exists a SVMR solution which is equivalent for a certain choice of the parameters. In particular our result is that for $epsilon$ sufficiently close to one, the optimal hyperplane and threshold for the SVMC problem with regularization parameter C_c are equal to (1-epsilon)^{- 1} times the optimal hyperplane and threshold for SVMR with regularization parameter C_r = (1-epsilon)C_c. A direct consequence of this result is that SVMC can be seen as a special case of SVMR.

On the Noise Model of Support Vector Machine Regression

Support Vector Machines Regression (SVMR) is a regression technique which has been recently introduced by V. Vapnik and his collaborators (Vapnik, 1995; Vapnik, Golowich and Smola, 1996). In SVMR the goodness of fit is measured not by the usual quadratic loss function (the mean square error), but by a different loss function called Vapnik"s $epsilon$- insensitive loss function, which is similar to the "robust" loss functions introduced by Huber (Huber, 1981). The quadratic loss function is well justified under the assumption of Gaussian additive noise. However, the noise model underlying the choice of Vapnik's loss function is less clear. In this paper the use of Vapnik's loss function is shown to be equivalent to a model of additive and Gaussian noise, where the variance and mean of the Gaussian are random variables. The probability distributions for the variance and mean will be stated explicitly. While this work is presented in the framework of SVMR, it can be extended to justify non-quadratic loss functions in any Maximum Likelihood or Maximum A Posteriori approach. It applies not only to Vapnik's loss function, but to a much broader class of loss functions.

Selecting Relevant Genes with a Spectral Approach

Array technologies have made it possible to record simultaneously the expression pattern of thousands of genes. A fundamental problem in the analysis of gene expression data is the identification of highly relevant genes that either discriminate between phenotypic labels or are important with respect to the cellular process studied in the experiment: for example cell cycle or heat shock in yeast experiments, chemical or genetic perturbations of mammalian cell lines, and genes involved in class discovery for human tumors. In this paper we focus on the task of unsupervised gene selection. The problem of selecting a small subset of genes is particularly challenging as the datasets involved are typically characterized by a very small sample size ?? the order of few tens of tissue samples ??d by a very large feature space as the number of genes tend to be in the high thousands. We propose a model independent approach which scores candidate gene selections using spectral properties of the candidate affinity matrix. The algorithm is very straightforward to implement yet contains a number of remarkable properties which guarantee consistent sparse selections. To illustrate the value of our approach we applied our algorithm on five different datasets. The first consists of time course data from four well studied Hematopoietic cell lines (HL-60, Jurkat, NB4, and U937). The other four datasets include three well studied treatment outcomes (large cell lymphoma, childhood medulloblastomas, breast tumors) and one unpublished dataset (lymph status). We compared our approach both with other unsupervised methods (SOM,PCA,GS) and with supervised methods (SNR,RMB,RFE). The results clearly show that our approach considerably outperforms all the other unsupervised approaches in our study, is competitive with supervised methods and in some case even outperforms supervised approaches.