A User-Centric Perspective on Parallel Programming with Focus on OpenMP

The process of developing software that takes advantage of multiple processors is commonly referred to as parallel programming. For various reasons, this process is much harder than the sequential case. For decades, parallel programming has been a problem for a small niche only: engineers working on parallelizing mostly numerical applications in High Performance Computing. This has changed with the advent of multi-core processors in mainstream computer architectures. Parallel programming in our days becomes a problem for a much larger group of developers. The main objective of this thesis was to find ways to make parallel programming easier for them. Different aims were identified in order to reach the objective: research the state of the art of parallel programming today, improve the education of software developers about the topic, and provide programmers with powerful abstractions to make their work easier. To reach these aims, several key steps were taken. To start with, a survey was conducted among parallel programmers to find out about the state of the art. More than 250 people participated, yielding results about the parallel programming systems and languages in use, as well as about common problems with these systems. Furthermore, a study was conducted in university classes on parallel programming. It resulted in a list of frequently made mistakes that were analyzed and used to create a programmers' checklist to avoid them in the future. For programmers' education, an online resource was setup to collect experiences and knowledge in the field of parallel programming - called the Parawiki. Another key step in this direction was the creation of the Thinking Parallel weblog, where more than 50.000 readers to date have read essays on the topic. For the third aim (powerful abstractions), it was decided to concentrate on one parallel programming system: OpenMP. Its ease of use and high level of abstraction were the most important reasons for this decision. Two different research directions were pursued. The first one resulted in a parallel library called AthenaMP. It contains so-called generic components, derived from design patterns for parallel programming. These include functionality to enhance the locks provided by OpenMP, to perform operations on large amounts of data (data-parallel programming), and to enable the implementation of irregular algorithms using task pools. AthenaMP itself serves a triple role: the components are well-documented and can be used directly in programs, it enables developers to study the source code and learn from it, and it is possible for compiler writers to use it as a testing ground for their OpenMP compilers. The second research direction was targeted at changing the OpenMP specification to make the system more powerful. The main contributions here were a proposal to enable thread-cancellation and a proposal to avoid busy waiting. Both were implemented in a research compiler, shown to be useful in example applications, and proposed to the OpenMP Language Committee.

Repeated and predictable patterns of ecotypic differentiation during a biological invasion: lake-stream divergence in parapatric Swiss stickleback

The relative importance of ecological selection and geographical isolation in promoting and constraining genetic and phenotypic differentiation among populations is not always obvious. Interacting with divergent selection, restricted opportunity for gene flow may in some cases be as much a cause as a consequence of adaptation, with the latter being a hallmark of ecologi- cal speciation. Ecological speciation is well studied in parts of the native range of the three-spined stickleback. Here, we study this process in a recently invaded part of its range. Switzerland was colonized within the past 140 years from at least three different colonization events involving differ- ent stickleback lineages. They now occupy diverse habitats, ranging from small streams to the pelagic zone of large lakes. We use replicated systems of parapatric lake and stream populations, some of which trace their origins to different invasive lineages, to ask (i) whether phenotypic divergence occurred among populations inhabiting distinct habitats, (ii) whether trajec- tories of phenotypic divergence follow predictable parallel patterns and (iii) whether gene flow constrains divergent adaptation or vice versa. We find consistent phenotypic divergence between populations occupying distinct habitats. This involves parallel evolution in several traits with known eco- logical relevance in independent evolutionary lineages. Adaptive divergence supersedes homogenizing gene flow even at a small spatial scale. We find evidence that adaptive phenotypic divergence places constraints on gene flow over and above that imposed by geographical distance, signalling the early onset of ecological speciation.

A green perspective on Structured parallel programming

Structured parallel programming, and in particular programming models using the algorithmic skeleton or parallel design pattern concepts, are increasingly considered to be the only viable means of supporting effective development of scalable and efficient parallel programs. Structured parallel programming models have been assessed in a number of works in the context of performance. In this paper we consider how the use of structured parallel programming models allows knowledge of the parallel patterns present to be harnessed to address both performance and energy consumption. We consider different features of structured parallel programming that may be leveraged to impact the performance/energy trade-off and we discuss a preliminary set of experiments validating our claims.

A parallel pattern for iterative stencil + reduce

We advocate the Loop-of-stencil-reduce pattern as a means of simplifying the implementation of data-parallel programs on heterogeneous multi-core platforms. Loop-of-stencil-reduce is general enough to subsume map, reduce, map-reduce, stencil, stencil-reduce, and, crucially, their usage in a loop in both data-parallel and streaming applications, or a combination of both. The pattern makes it possible to deploy a single stencil computation kernel on different GPUs. We discuss the implementation of Loop-of-stencil-reduce in FastFlow, a framework for the implementation of applications based on the parallel patterns. Experiments are presented to illustrate the use of Loop-of-stencil-reduce in developing data-parallel kernels running on heterogeneous systems.

Late pleistocene ice export events into the Arctic Ocean from the M'Clure Strait Ice Stream, Canadian Arctic Archipelago

Rapidly-flowing sectors of an ice sheet (ice streams) can play ail important role in abrupt climate change through tile delivery of icebergs and meltwater and tile Subsequent disruption of ocean thermohaline circulation (e.g., the North Atlantic's Heinrich events). Recently, several cores have been raised from the Arctic Ocean which document the existence of massive ice export events during tile Late Pleistocene and whose provenance has been linked to Source regions in the Canadian Arctic Archipelago. In this paper, satellite imagery is used to map glacial geomorphology in the vicinity of Victoria Island, Banks Island and Prince of Wales Island (Canadian Arctic) in order to reconstruct ice flow patterns in the highly complex glacial landscape. A total of 88 discrete flow-sets are mapped and of these, 13 exhibit the characteristic geomorphology of palaeo-ice streams (i.e., parallel patterns of large, highly elongated mega-scale glacial lineations forming a convergent flow pattern with abrupt lateral margins). Previous studies by other workers and cross-cutting relationships indicate that the majority of these ice streams are relatively young and operated during or immediately prior to deglaciation. Our new mapping, however, documents a large (> 700 km long; 110 km wide) and relatively old ice stream imprint centred in M'Clintock Channel and converging into Viscount Melville Sound. A trough mouth fan located on the continental shelf Suggests that it extended along M'Clure Strait and was grounded at tile shelf edge. The location of the M'Clure Strait Ice Stream exactly matches the Source area of 4 (possibly 5) major ice export events recorded in core PS 1230 raised from Fram Strait, the major ice exit for the Arctic Ocean. These ice export events occur at similar to 12.9, similar to 15.6, similar to 22 and 29.8 ka (C-14 yr BP) and we argue that they record vigorous episodes of activity of the M'Clure Strait Ice Stream. The timing of these events is remarkably similar to the North Atlantic's Heinrich events and we take this as evidence that the M'Clure Strait Ice Stream was also activated around the same time. This may hold important implications for tile cause of the North Atlantic's Heinrich events and hints at tile possibility of a pall-ice sheet response. (c) 2005 Elsevier B.V. All rights reserved.

Positive relationship between genetic- and species diversity on limestone outcrops in the Carpathian Mountains

We asked if the genetic diversity of Saponaria bellidifolia (a habitat specialist plant) and the species diversity of its habitat are driven by parallel landscape level processes in an island-like system of limestone outcrops in the Carpathian Mountains. We tested the relationship of these two diversity levels at local and regional geographic scales. Local genetic and species diversity showed parallel patterns influenced by the number of plant communities. Likewise, at regional level there was strong evidence for parallel equilibrial dynamics of genotypes and species. However, a superimposed matrix effect enhanced the regional species diversity only. Genetic diversity of habitat specialist organisms and species diversity of these limestone outcrop islands on mainland are modulated by parallel landscape-level processes at different geographic scales, and mechanisms may be identified at very high spatial resolutions.

The Loop-of-Stencil-Reduce Paradigm

In this paper we advocate the Loop-of-stencil-reduce pattern as a way to simplify the parallel programming of heterogeneous platforms (multicore+GPUs). Loop-of-Stencil-reduce is general enough to subsume map, reduce, map-reduce, stencil, stencil-reduce, and, crucially, their usage in a loop. It transparently targets (by using OpenCL) combinations of CPU cores and GPUs, and it makes it possible to simplify the deployment of a single stencil computation kernel on different GPUs. The paper discusses the implementation of Loop-of-stencil-reduce within the FastFlow parallel framework, considering a simple iterative data-parallel application as running example (Game of Life) and a highly effective parallel filter for visual data restoration to assess performance. Thanks to the high-level design of the Loop-of-stencil-reduce, it was possible to run the filter seamlessly on a multicore machine, on multi-GPUs, and on both.

Bifurcation in growth patterns for arrays of parallel Griffith, edge and sliding cracks

This paper presents the recent finding by Muhlhaus et al [1] that bifurcation of crack growth patterns exists for arrays of two-dimensional cracks. This bifurcation is a result of the nonlinear effect due to crack interaction, which is, in the present analysis, approximated by the dipole asymptotic or pseudo-traction method. The nonlinear parameter for the problem is the crack length/ spacing ratio lambda = a/h. For parallel and edge crack arrays under far field tension, uniform crack growth patterns (all cracks having same size) yield to nonuniform crack growth patterns (i.e. bifurcation) if lambda is larger than a critical value lambda(cr) (note that such bifurcation is not found for collinear crack arrays). For parallel and edge crack arrays respectively, the value of lambda(cr) decreases monotonically from (2/9)(1/2) and (2/15.096)(1/2) for arrays of 2 cracks, to (2/3)(1/2)/pi and (2/5.032)(1/2)/pi for infinite arrays of cracks. The critical parameter lambda(cr) is calculated numerically for arrays of up to 100 cracks, whilst discrete Fourier transform is used to obtain the exact solution of lambda(cr) for infinite crack arrays. For geomaterials, bifurcation can also occurs when array of sliding cracks are under compression.

Patterns of calcium-binding proteins support parallel and hierarchical organization of human auditory areas.

The human primary auditory cortex (AI) is surrounded by several other auditory areas, which can be identified by cyto-, myelo- and chemoarchitectonic criteria. We report here on the pattern of calcium-binding protein immunoreactivity within these areas. The supratemporal regions of four normal human brains (eight hemispheres) were processed histologically, and serial sections were stained for parvalbumin, calretinin or calbindin. Each calcium-binding protein yielded a specific pattern of labelling, which differed between auditory areas. In AI, defined as area TC [see C. von Economo and L. Horn (1930) Z. Ges. Neurol. Psychiatr.,130, 678-757], parvalbumin labelling was dark in layer IV; several parvalbumin-positive multipolar neurons were distributed in layers III and IV. Calbindin yielded dark labelling in layers I-III and V; it revealed numerous multipolar and pyramidal neurons in layers II and III. Calretinin labelling was lighter than that of parvalbumin or calbindin in AI; calretinin-positive bipolar and bitufted neurons were present in supragranular layers. In non-primary auditory areas, the intensity of labelling tended to become progressively lighter while moving away from AI, with qualitative differences between the cytoarchitectonically defined areas. In analogy to non-human primates, our results suggest differences in intrinsic organization between auditory areas that are compatible with parallel and hierarchical processing of auditory information.

Patterns of calcium-binding proteins in human inferior colliculus: identification of subdivisions and evidence for putative parallel systems.

The subdivisions of human inferior colliculus are currently based on Golgi and Nissl-stained preparations. We have investigated the distribution of calcium-binding protein immunoreactivity in the human inferior colliculus and found complementary or mutually exclusive localisations of parvalbumin versus calbindin D-28k and calretinin staining. The central nucleus of the inferior colliculus but not the surrounding regions contained parvalbumin-positive neuronal somata and fibres. Calbindin-positive neurons and fibres were concentrated in the dorsal aspect of the central nucleus and in structures surrounding it: the dorsal cortex, the lateral lemniscus, the ventrolateral nucleus, and the intercollicular region. In the dorsal cortex, labelling of calbindin and calretinin revealed four distinct layers.Thus, calcium-binding protein reactivity reveals in the human inferior colliculus distinct neuronal populations that are anatomically segregated. The different calcium-binding protein-defined subdivisions may belong to parallel auditory pathways that were previously demonstrated in non-human primates, and they may constitute a first indication of parallel processing in human subcortical auditory structures.

Parallel evolution of facial stripe patterns in the Neolamprologus brichardi/pulcher species complex endemic to Lake Tanganyika.

Colour pattern diversity can be due to random processes or to natural or sexual selection. Consequently, similarities in colour patterns are not always correlated with common ancestry, but may result from convergent evolution under shared selection pressures or drift. Neolamprologus brichardi and Neolamprologus pulcher have been described as two distinct species based on differences in the arrangement of two dark bars on the operculum. Our study uses DNA sequences of the mitochondrial control region to show that relatedness of haplotypes disagrees with species assignment based on head colour pattern. This suggests repeated parallel evolution of particular stripe patterns. The complete lack of shared haplotypes between populations of the same or different phenotypes reflects strong philopatric behaviour, possibly induced by the cooperative breeding mode in which offspring remain in their natal territory and serve as helpers until they disperse to nearby territories or take over a breeding position. Concordant phylogeographic patterns between N. brichardi/N. pulcher populations and other rock-dwelling cichlids suggest that the same colonization routes have been taken by sympatric species and that these routes were affected by lake level fluctuations in the past.

Parallel evolution of facial stripe patterns in the Neolamprologus brichardilpulcher species complex endemic to Lake Tanganyika

Colour pattern diversity can be due to random processes or to natural or sexual selection. Consequently, similarities in colour patterns are not always correlated with common ancestry, but may result from convergent evolution under shared selection pressures or drift. Neolamprologus brichardi and Neolamprologus pulcher have been described as two distinct species based on differences in the arrangement of two dark bars on the operculum. Our study uses DNA sequences of the mitochondrial control region to show that relatedness of haplotypes disagrees with species assignment based on head colour pattern. This suggests repeated parallel evolution of particular stripe patterns. The complete lack of shared haplotypes between populations of the same or different phenotypes reflects strong philopatric behaviour, possibly induced by the cooperative breeding mode in which offspring remain in their natal territory and serve as helpers until they disperse to nearby territories or take over a breeding position. Concordant phylogeographic patterns between N. brichardi/N. pulcher populations and other rock-dwelling cichlids suggest that the same colonization routes have been taken by sympatric species and that these routes were affected by lake level fluctuations in the past. (C) 2007 Elsevier Inc. All rights reserved.

The new metal worker pattern book; a complete course of instruction in the modern methods of developing and cutting the patterns for sheet metal work, giving the principles underlying practically every problem that is likely to come up in practice and explaining the selection and use of drawing tools and linear and geometrical drawing so clearly that one, who has had no previous knowledge of arithmetic or drawing, may understand these essentials and apply them in using the 253 problems in the development of patterns by the parallel line, conical or flaring, and triangulation systems.

Mode of access: Internet.

Patterns of variation within self-incompatibility loci

Diverse self-incompatibility (SI) mechanisms permit flowering plants to inhibit fertilization by pollen that express specificities in common with the pistil. Characteristic of at least two model systems is greatly reduced recombination across large genomic tracts surrounding the S-locus, which regulates SI. In three angiosperm families, including the Solanaceae, the gene that controls the expression of gametophytic SI in the pistil encodes a ribonuclease (S-RNase). The gene that controls pollen SI expression is currently unknown, although several candidates have recently been proposed. Although each candidate shows a high level of polymorphism and complete allelic disequilibrium with the S-RNase gene, such properties may merely reflect tight linkage to the S-locus, irrespective of any functional role in SI. We analyzed the magnitude and nature of nucleotide variation, with the objective of distinguishing likely candidates for regulators of SI from other genes embedded in the S-locus region. We studied the S-RNase gene of the Solanaceae and 48A, a candidate for the pollen gene in this system, and we also conducted a parallel analysis of the regulators of sporophytic SI in Brassica, a system in which both the pistil and pollen genes are known. Although the pattern of variation shown by the pollen gene of the Brassica system is consistent with its role as a determinant of pollen specificity, that of 48A departs from expectation. Our analysis further suggests that recombination between 48A and S-RNase may have occurred during the interval spanned by the gene genealogy, another indication that 48A may not regulate SI expression in pollen.