Evaluating the state of the art of parallel programming systems

This paper describes our plans to evaluate the present state of affairs concerning parallel programming and its systems. Three subprojects are proposed: a survey among programmers and scientists, a comparison of parallel programming systems using a standard set of test programs, and a wiki resource for the parallel programming community - the Parawiki. We would like to invite you to participate and turn these subprojects into true community efforts.

Observations on the Publicity and Usage of Parallel Programming Systems and Languages: A Survey Approach

In this publication, we report on an online survey that was carried out among parallel programmers. More than 250 people worldwide have submitted answers to our questions, and their responses are analyzed here. Although not statistically sound, the data we provide give useful insights about which parallel programming systems and languages are known and in actual use. For instance, the collected data indicate that for our survey group MPI and (to a lesser extent) C are the most widely used parallel programming system and language, respectively.

Genetic Programming meets Model-Driven Development

Genetic programming is known to provide good solutions for many problems like the evolution of network protocols and distributed algorithms. In such cases it is most likely a hardwired module of a design framework that assists the engineer to optimize specific aspects of the system to be developed. It provides its results in a fixed format through an internal interface. In this paper we show how the utility of genetic programming can be increased remarkably by isolating it as a component and integrating it into the model-driven software development process. Our genetic programming framework produces XMI-encoded UML models that can easily be loaded into widely available modeling tools which in turn posses code generation as well as additional analysis and test capabilities. We use the evolution of a distributed election algorithm as an example to illustrate how genetic programming can be combined with model-driven development. This example clearly illustrates the advantages of our approach – the generation of source code in different programming languages.

A User-Centric Perspective on Parallel Programming with Focus on OpenMP

The process of developing software that takes advantage of multiple processors is commonly referred to as parallel programming. For various reasons, this process is much harder than the sequential case. For decades, parallel programming has been a problem for a small niche only: engineers working on parallelizing mostly numerical applications in High Performance Computing. This has changed with the advent of multi-core processors in mainstream computer architectures. Parallel programming in our days becomes a problem for a much larger group of developers. The main objective of this thesis was to find ways to make parallel programming easier for them. Different aims were identified in order to reach the objective: research the state of the art of parallel programming today, improve the education of software developers about the topic, and provide programmers with powerful abstractions to make their work easier. To reach these aims, several key steps were taken. To start with, a survey was conducted among parallel programmers to find out about the state of the art. More than 250 people participated, yielding results about the parallel programming systems and languages in use, as well as about common problems with these systems. Furthermore, a study was conducted in university classes on parallel programming. It resulted in a list of frequently made mistakes that were analyzed and used to create a programmers' checklist to avoid them in the future. For programmers' education, an online resource was setup to collect experiences and knowledge in the field of parallel programming - called the Parawiki. Another key step in this direction was the creation of the Thinking Parallel weblog, where more than 50.000 readers to date have read essays on the topic. For the third aim (powerful abstractions), it was decided to concentrate on one parallel programming system: OpenMP. Its ease of use and high level of abstraction were the most important reasons for this decision. Two different research directions were pursued. The first one resulted in a parallel library called AthenaMP. It contains so-called generic components, derived from design patterns for parallel programming. These include functionality to enhance the locks provided by OpenMP, to perform operations on large amounts of data (data-parallel programming), and to enable the implementation of irregular algorithms using task pools. AthenaMP itself serves a triple role: the components are well-documented and can be used directly in programs, it enables developers to study the source code and learn from it, and it is possible for compiler writers to use it as a testing ground for their OpenMP compilers. The second research direction was targeted at changing the OpenMP specification to make the system more powerful. The main contributions here were a proposal to enable thread-cancellation and a proposal to avoid busy waiting. Both were implemented in a research compiler, shown to be useful in example applications, and proposed to the OpenMP Language Committee.

Evolving Distributed Algorithms with Genetic Programming

Distributed systems are one of the most vital components of the economy. The most prominent example is probably the internet, a constituent element of our knowledge society. During the recent years, the number of novel network types has steadily increased. Amongst others, sensor networks, distributed systems composed of tiny computational devices with scarce resources, have emerged. The further development and heterogeneous connection of such systems imposes new requirements on the software development process. Mobile and wireless networks, for instance, have to organize themselves autonomously and must be able to react to changes in the environment and to failing nodes alike. Researching new approaches for the design of distributed algorithms may lead to methods with which these requirements can be met efficiently. In this thesis, one such method is developed, tested, and discussed in respect of its practical utility. Our new design approach for distributed algorithms is based on Genetic Programming, a member of the family of evolutionary algorithms. Evolutionary algorithms are metaheuristic optimization methods which copy principles from natural evolution. They use a population of solution candidates which they try to refine step by step in order to attain optimal values for predefined objective functions. The synthesis of an algorithm with our approach starts with an analysis step in which the wanted global behavior of the distributed system is specified. From this specification, objective functions are derived which steer a Genetic Programming process where the solution candidates are distributed programs. The objective functions rate how close these programs approximate the goal behavior in multiple randomized network simulations. The evolutionary process step by step selects the most promising solution candidates and modifies and combines them with mutation and crossover operators. This way, a description of the global behavior of a distributed system is translated automatically to programs which, if executed locally on the nodes of the system, exhibit this behavior. In our work, we test six different ways for representing distributed programs, comprising adaptations and extensions of well-known Genetic Programming methods (SGP, eSGP, and LGP), one bio-inspired approach (Fraglets), and two new program representations called Rule-based Genetic Programming (RBGP, eRBGP) designed by us. We breed programs in these representations for three well-known example problems in distributed systems: election algorithms, the distributed mutual exclusion at a critical section, and the distributed computation of the greatest common divisor of a set of numbers. Synthesizing distributed programs the evolutionary way does not necessarily lead to the envisaged results. In a detailed analysis, we discuss the problematic features which make this form of Genetic Programming particularly hard. The two Rule-based Genetic Programming approaches have been developed especially in order to mitigate these difficulties. In our experiments, at least one of them (eRBGP) turned out to be a very efficient approach and in most cases, was superior to the other representations.

Development and applications of density matrix functional theory of generalized Hubbard Models

In dieser Doktorarbeit wird eine akkurate Methode zur Bestimmung von Grundzustandseigenschaften stark korrelierter Elektronen im Rahmen von Gittermodellen entwickelt und angewandt. In der Dichtematrix-Funktional-Theorie (LDFT, vom englischen lattice density functional theory) ist die Ein-Teilchen-Dichtematrix γ die fundamentale Variable. Auf der Basis eines verallgemeinerten Hohenberg-Kohn-Theorems ergibt sich die Grundzustandsenergie Egs[γgs] = min° E[γ] durch die Minimierung des Energiefunktionals E[γ] bezüglich aller physikalischer bzw. repräsentativer γ. Das Energiefunktional kann in zwei Beiträge aufgeteilt werden: Das Funktional der kinetischen Energie T[γ], dessen lineare Abhängigkeit von γ genau bekannt ist, und das Funktional der Korrelationsenergie W[γ], dessen Abhängigkeit von γ nicht explizit bekannt ist. Das Auffinden präziser Näherungen für W[γ] stellt die tatsächliche Herausforderung dieser These dar. Einem Teil dieser Arbeit liegen vorausgegangene Studien zu Grunde, in denen eine Näherung des Funktionals W[γ] für das Hubbardmodell, basierend auf Skalierungshypothesen und exakten analytischen Ergebnissen für das Dimer, hergeleitet wird. Jedoch ist dieser Ansatz begrenzt auf spin-unabhängige und homogene Systeme. Um den Anwendungsbereich von LDFT zu erweitern, entwickeln wir drei verschiedene Ansätze zur Herleitung von W[γ], die das Studium von Systemen mit gebrochener Symmetrie ermöglichen. Zuerst wird das bisherige Skalierungsfunktional erweitert auf Systeme mit Ladungstransfer. Eine systematische Untersuchung der Abhängigkeit des Funktionals W[γ] von der Ladungsverteilung ergibt ähnliche Skalierungseigenschaften wie für den homogenen Fall. Daraufhin wird eine Erweiterung auf das Hubbardmodell auf bipartiten Gittern hergeleitet und an sowohl endlichen als auch unendlichen Systemen mit repulsiver und attraktiver Wechselwirkung angewandt. Die hohe Genauigkeit dieses Funktionals wird aufgezeigt. Es erweist sich jedoch als schwierig, diesen Ansatz auf komplexere Systeme zu übertragen, da bei der Berechnung von W[γ] das System als ganzes betrachtet wird. Um dieses Problem zu bewältigen, leiten wir eine weitere Näherung basierend auf lokalen Skalierungseigenschaften her. Dieses Funktional ist lokal bezüglich der Gitterplätze formuliert und ist daher anwendbar auf jede Art von geordneten oder ungeordneten Hamiltonoperatoren mit lokalen Wechselwirkungen. Als Anwendungen untersuchen wir den Metall-Isolator-Übergang sowohl im ionischen Hubbardmodell in einer und zwei Dimensionen als auch in eindimensionalen Hubbardketten mit nächsten und übernächsten Nachbarn. Schließlich entwickeln wir ein numerisches Verfahren zur Berechnung von W[γ], basierend auf exakten Diagonalisierungen eines effektiven Vielteilchen-Hamilton-Operators, welcher einen von einem effektiven Medium umgebenen Cluster beschreibt. Dieser effektive Hamiltonoperator hängt von der Dichtematrix γ ab und erlaubt die Herleitung von Näherungen an W[γ], dessen Qualität sich systematisch mit steigender Clustergröße verbessert. Die Formulierung ist spinabhängig und ermöglicht eine direkte Verallgemeinerung auf korrelierte Systeme mit mehreren Orbitalen, wie zum Beispiel auf den spd-Hamilton-Operator. Darüber hinaus berücksichtigt sie die Effekte kurzreichweitiger Ladungs- und Spinfluktuationen in dem Funktional. Für das Hubbardmodell wird die Genauigkeit der Methode durch Vergleich mit Bethe-Ansatz-Resultaten (1D) und Quanten-Monte-Carlo-Simulationen (2D) veranschaulicht. Zum Abschluss wird ein Ausblick auf relevante zukünftige Entwicklungen dieser Theorie gegeben.

Time Discretization of the SST-generalized Navier-Stokes Equations: Positive and Negative Results

In the theory of the Navier-Stokes equations, the proofs of some basic known results, like for example the uniqueness of solutions to the stationary Navier-Stokes equations under smallness assumptions on the data or the stability of certain time discretization schemes, actually only use a small range of properties and are therefore valid in a more general context. This observation leads us to introduce the concept of SST spaces, a generalization of the functional setting for the Navier-Stokes equations. It allows us to prove (by means of counterexamples) that several uniqueness and stability conjectures that are still open in the case of the Navier-Stokes equations have a negative answer in the larger class of SST spaces, thereby showing that proof strategies used for a number of classical results are not sufficient to affirmatively answer these open questions. More precisely, in the larger class of SST spaces, non-uniqueness phenomena can be observed for the implicit Euler scheme, for two nonlinear versions of the Crank-Nicolson scheme, for the fractional step theta scheme, and for the SST-generalized stationary Navier-Stokes equations. As far as stability is concerned, a linear version of the Euler scheme, a nonlinear version of the Crank-Nicolson scheme, and the fractional step theta scheme turn out to be non-stable in the class of SST spaces. The positive results established in this thesis include the generalization of classical uniqueness and stability results to SST spaces, the uniqueness of solutions (under smallness assumptions) to two nonlinear versions of the Euler scheme, two nonlinear versions of the Crank-Nicolson scheme, and the fractional step theta scheme for general SST spaces, the second order convergence of a version of the Crank-Nicolson scheme, and a new proof of the first order convergence of the implicit Euler scheme for the Navier-Stokes equations. For each convergence result, we provide conditions on the data that guarantee the existence of nonstationary solutions satisfying the regularity assumptions needed for the corresponding convergence theorem. In the case of the Crank-Nicolson scheme, this involves a compatibility condition at the corner of the space-time cylinder, which can be satisfied via a suitable prescription of the initial acceleration.