Evolving Classifiers - Evolutionary Algorithms in Data Mining

Data mining means to summarize information from large amounts of raw data. It is one of the key technologies in many areas of economy, science, administration and the internet. In this report we introduce an approach for utilizing evolutionary algorithms to breed fuzzy classifier systems. This approach was exercised as part of a structured procedure by the students Achler, Göb and Voigtmann as contribution to the 2006 Data-Mining-Cup contest, yielding encouragingly positive results.

Global Optimization Algorithms and their Application to Distributed Systems

In this report, we discuss the application of global optimization and Evolutionary Computation to distributed systems. We therefore selected and classified many publications, giving an insight into the wide variety of optimization problems which arise in distributed systems. Some interesting approaches from different areas will be discussed in greater detail with the use of illustrative examples.

Evolving Distributed Algorithms with Genetic Programming

Distributed systems are one of the most vital components of the economy. The most prominent example is probably the internet, a constituent element of our knowledge society. During the recent years, the number of novel network types has steadily increased. Amongst others, sensor networks, distributed systems composed of tiny computational devices with scarce resources, have emerged. The further development and heterogeneous connection of such systems imposes new requirements on the software development process. Mobile and wireless networks, for instance, have to organize themselves autonomously and must be able to react to changes in the environment and to failing nodes alike. Researching new approaches for the design of distributed algorithms may lead to methods with which these requirements can be met efficiently. In this thesis, one such method is developed, tested, and discussed in respect of its practical utility. Our new design approach for distributed algorithms is based on Genetic Programming, a member of the family of evolutionary algorithms. Evolutionary algorithms are metaheuristic optimization methods which copy principles from natural evolution. They use a population of solution candidates which they try to refine step by step in order to attain optimal values for predefined objective functions. The synthesis of an algorithm with our approach starts with an analysis step in which the wanted global behavior of the distributed system is specified. From this specification, objective functions are derived which steer a Genetic Programming process where the solution candidates are distributed programs. The objective functions rate how close these programs approximate the goal behavior in multiple randomized network simulations. The evolutionary process step by step selects the most promising solution candidates and modifies and combines them with mutation and crossover operators. This way, a description of the global behavior of a distributed system is translated automatically to programs which, if executed locally on the nodes of the system, exhibit this behavior. In our work, we test six different ways for representing distributed programs, comprising adaptations and extensions of well-known Genetic Programming methods (SGP, eSGP, and LGP), one bio-inspired approach (Fraglets), and two new program representations called Rule-based Genetic Programming (RBGP, eRBGP) designed by us. We breed programs in these representations for three well-known example problems in distributed systems: election algorithms, the distributed mutual exclusion at a critical section, and the distributed computation of the greatest common divisor of a set of numbers. Synthesizing distributed programs the evolutionary way does not necessarily lead to the envisaged results. In a detailed analysis, we discuss the problematic features which make this form of Genetic Programming particularly hard. The two Rule-based Genetic Programming approaches have been developed especially in order to mitigate these difficulties. In our experiments, at least one of them (eRBGP) turned out to be a very efficient approach and in most cases, was superior to the other representations.

Algorithms for Tamagawa Number Conjectures

In dieser Arbeit werden Algorithmen zur Untersuchung der äquivarianten Tamagawazahlvermutung von Burns und Flach entwickelt. Zunächst werden Algorithmen angegeben mit denen die lokale Fundamentalklasse, die globale Fundamentalklasse und Tates kanonische Klasse berechnet werden können. Dies ermöglicht unter anderem Berechnungen in Brauergruppen von Zahlkörpererweiterungen. Anschließend werden diese Algorithmen auf die Tamagawazahlvermutung angewendet. Die Epsilonkonstantenvermutung kann dadurch für alle Galoiserweiterungen L|K bewiesen werden, bei denen L in einer Galoiserweiterung E|Q vom Grad kleiner gleich 15 eingebettet werden kann. Für die Tamagawazahlvermutung an der Stelle 1 wird ein Algorithmus angegeben, der die Vermutung für ein gegebenes Fallbeispiel L|Q numerischen verifizieren kann. Im Spezialfall, dass alle Charaktere rational oder abelsch sind, kann dieser Algorithmus die Vermutung für L|Q sogar beweisen.

Systematics and evolution of the genus Deuterocohnia Mez (Bromeliaceae)

The present study investigates the systematics and evolution of the Neotropical genus Deuterocohnia Mez (Bromeliaceae). It provides a comprehensive taxonomic revision as well as phylogenetic analyses based on chloroplast and nuclear DNA sequences and presents a hypothesis on the evolution of the genus. A broad morphological, anatomical, biogeographical and ecological overview of the genus is given in the first part of the study. For morphological character assessment more than 700 herbarium specimens from 39 herbaria as well as living plant material in the field and in the living collections of botanical gardens were carefully examined. The arid habitats, in which the species of Deuterocohnia grow, are reflected by the morphological and anatomical characters of the species. Important characters for species delimitation were identified, like the length of the inflorescence, the branching order, the density of flowers on partial inflorescences, the relation of the length of the primary bracts to that of the partial inflorescence, the sizes of floral bracts, sepals and petals, flower colour, the presence or absence of a pedicel, the curvature of the stamina and the petals during anthesis. After scrutinizing the nomenclatural history of the taxa belonging to Deuterocohnia – including the 1992 syonymized genus Abromeitiella – 17 species, 4 subspecies and 4 varieties are accepted in the present revision. Taxonomic changes were made in the following cases: (I) New combinations: A. abstrusa (A. Cast.) N. Schütz is re-established – as defined by Castellanos (1931) – and transfered to D. abstrusa; D. brevifolia (Griseb.) M.A. Spencer & L.B. Sm. includes accessions of the former D. lorentziana (Mez) M.A. Spencer & L.B. Sm., which are not assigned to D. abstrusa; D. bracteosa W. Till is synonymized to D. strobilifera Mez; D. meziana Kuntze ex Mez var. carmineo-viridiflora Rauh is classified as a subspecies of D. meziana (ssp. carmineo-viridiflora (Rauh) N. Schütz); D. pedicellata W. Till is classified as a subspecies of D. meziana (ssp. pedicellata (W. Till) N. Schütz); D. scapigera (Rauh & L. Hrom.) M.A. Spencer & L.B. Sm ssp. sanctae-crucis R. Vásquez & Ibisch is classified as a species (D. sanctae-crucis (R. Vásquez & Ibisch) N. Schütz); (II) New taxa: a new subspecies of D. meziana Kuntze ex Mez is established; a new variety of D. scapigera is established; (the new taxa will be validly published elsewhere); (III) New type: an epitype for D. longipetala was chosen. All other species were kept according to Spencer and Smith (1992) or – in the case of more recently described species – according to the protologue. Beside the nomenclatural notes and the detailed descriptions, information on distribution, habitat and ecology, etymology and taxonomic delimitation is provided for the genus and for each of its species. An key was constructed for the identification of currently accepted species, subspecies and varieties. The key is based on easily detectable morphological characters. The former synonymization of the genus Abromeitiella into Deuterocohnia (Spencer and Smith 1992) is re-evalutated in the present study. Morphological as well as molecular investigations revealed Deuterocohnia incl. Abromeitiella as being monophyletic, with some indications that a monophyletic Abromeitiella lineage arose from within Deuterocohnia. Thus the union of both genera is confirmed. The second part of the present thesis describes and discusses the molecular phylogenies and networks. Molecular analyses of three chloroplast intergenic spacers (rpl32-trnL, rps16-trnK, trnS-ycf3) were conducted with a sample set of 119 taxa. This set included 103 Deuterocohnia accessions from all 17 described species of the genus and 16 outgroup taxa from the remainder of Pitcairnioideae s.str. (Dyckia (8 sp.), Encholirium (2 sp.), Fosterella (4 sp.) and Pitcairnia (2 sp.)). With its high sampling density, the present investigation by far represents the most comprehensive molecular study of Deuterocohnia up till now. All data sets were analyzed separately as well as in combination, and various optimality criteria for phylogenetic tree construction were applied (Maximum Parsimony, Maximum Likelihood, Bayesian inferences and the distance method Neighbour Joining). Congruent topologies were generally obtained with different algorithms and optimality criteria, but individual clades received different degrees of statistical support in some analyses. The rps16-trnK locus was the most informative among the three spacer regions examined. The results of the chloroplast DNA analyses revealed a highly supported paraphyly of Deuterocohnia. Thus, the cpDNA trees divide the genus into two subclades (A and B), of which Deuterocohnia subclade B is sister to the included Dyckia and Encholirium accessions, and both together are sister to Deuterocohnia subclade A. To further examine the relationship between Deuterocohnia and Dyckia/Encholirium at the generic level, two nuclear low copy markers (PRK exon2-5 and PHYC exon1) were analysed with a reduced taxon set. This set included 22 Deuterocohnia accessions (including members of both cpDNA subclades), 2 Dyckia, 2 Encholirium and 2 Fosterella species. Phylogenetic trees were constructed as described above, and for comparison the same reduced taxon set was also analysed at the three cpDNA data loci. In contrast to the cpDNA results, the nuclear DNA data strongly supported the monophyly of Deuterocohnia, which takes a sister position to a clade of Dyckia and Encholirium samples. As morphology as well as nuclear DNA data generated in the present study and in a former AFLP analysis (Horres 2003) all corroborate the monophyly of Deuterocohnia, the apparent paraphyly displayed in cpDNA analyses is interpreted to be the consequence of a chloroplast capture event. This involves the introgression of the chloroplast genome from the common ancestor of the Dyckia/ Encholirium lineage into the ancestor of Deuterocohnia subclade B species. The chloroplast haplotypes are not species-specific in Deuterocohnia. Thus, one haplotype was sometimes shared by several species, where the same species may harbour different haplotypes. The arrangement of haplotypes followed geographical patterns rather than taxonomic boundaries, which may indicate some residual gene flow among populations from different Deuteroccohnia species. Phenotypic species coherence on the background of ongoing gene flow may then be maintained by sets of co-adapted alleles, as was suggested by the porous genome concept (Wu 2001, Palma-Silva et al. 2011). The results of the present study suggest the following scenario for the evolution of Deuterocohnia and its species. Deuterocohnia longipetala may be envisaged as a representative of the ancestral state within the genus. This is supported by (1) the wide distribution of this species; (2) the overlap in distribution area with species of Dyckia; (3) the laxly flowered inflorescences, which are also typical for Dyckia; (4) the yellow petals with a greenish tip, present in most other Deuterocohnia species. The following six extant lineages within Deuterocohnia might have independently been derived from this ancestral state with a few changes each: (I) D. meziana, D. brevispicata and D. seramisiana (Bolivia, lowland to montane areas, mostly reddish-greenish coloured, very laxly to very densely flowered); (II) D. strobilifera (Bolivia, high Andean mountains, yellow flowers, densely flowered); (III) D. glandulosa (Bolivia, montane areas, yellow-greenish flowers, densely flowered); (IV) D. haumanii, D. schreiteri, D. digitata, and D. chrysantha (Argentina, Chile, E Andean mountains and Atacama desert, yellow-greenish flowers, densely flowered); (V) D. recurvipetala (Argentina, foothills of the Andes, recurved yellow flowers, laxly flowered); (VI) D. gableana, D. scapigera, D. sanctae-crucis, D. abstrusa, D. brevifolia, D. lotteae (former Abromeitiella species, Bolivia, Argentina, higher Andean mountains, greenish-yellow flowers, inflorescence usually simple). Originating from the lower montane Andean regions, at least four lineages of the genus (I, II, IV, VI) adapted in part to higher altitudes by developing densely flowered partial inflorescences, shorter flowers and – in at least three lineages (II, IV, VI) – smaller rosettes, whereas species spreading into the lowlands (I, V) developed larger plants, laxly flowered, amply branched inflorescences and in part larger flowers (I).

Multi-Robot Task Allocation for Inspection Problems with Cooperative Tasks Using Hybrid Genetic Algorithms

In dieser Dissertation werden Methoden zur optimalen Aufgabenverteilung in Multirobotersystemen (engl. Multi-Robot Task Allocation – MRTA) zur Inspektion von Industrieanlagen untersucht. MRTA umfasst die Verteilung und Ablaufplanung von Aufgaben für eine Gruppe von Robotern unter Berücksichtigung von operativen Randbedingungen mit dem Ziel, die Gesamteinsatzkosten zu minimieren. Dank zunehmendem technischen Fortschritt und sinkenden Technologiekosten ist das Interesse an mobilen Robotern für den Industrieeinsatz in den letzten Jahren stark gestiegen. Viele Arbeiten konzentrieren sich auf Probleme der Mobilität wie Selbstlokalisierung und Kartierung, aber nur wenige Arbeiten untersuchen die optimale Aufgabenverteilung. Da sich mit einer guten Aufgabenverteilung eine effizientere Planung erreichen lässt (z. B. niedrigere Kosten, kürzere Ausführungszeit), ist das Ziel dieser Arbeit die Entwicklung von Lösungsmethoden für das aus Inspektionsaufgaben mit Einzel- und Zweiroboteraufgaben folgende Such-/Optimierungsproblem. Ein neuartiger hybrider Genetischer Algorithmus wird vorgestellt, der einen teilbevölkerungbasierten Genetischen Algorithmus zur globalen Optimierung mit lokalen Suchheuristiken kombiniert. Zur Beschleunigung dieses Algorithmus werden auf die fittesten Individuen einer Generation lokale Suchoperatoren angewendet. Der vorgestellte Algorithmus verteilt die Aufgaben nicht nur einfach und legt den Ablauf fest, sondern er bildet auch temporäre Roboterverbünde für Zweiroboteraufgaben, wodurch räumliche und zeitliche Randbedingungen entstehen. Vier alternative Kodierungsstrategien werden für den vorgestellten Algorithmus entworfen: Teilaufgabenbasierte Kodierung: Hierdurch werden alle möglichen Lösungen abgedeckt, allerdings ist der Suchraum sehr groß. Aufgabenbasierte Kodierung: Zwei Möglichkeiten zur Zuweisung von Zweiroboteraufgaben wurden implementiert, um die Effizienz des Algorithmus zu steigern. Gruppierungsbasierte Kodierung: Zeitliche Randbedingungen zur Gruppierung von Aufgaben werden vorgestellt, um gute Lösungen innerhalb einer kleinen Anzahl von Generationen zu erhalten. Zwei Umsetzungsvarianten werden vorgestellt. Dekompositionsbasierte Kodierung: Drei geometrische Zerlegungen wurden entworfen, die Informationen über die räumliche Anordnung ausnutzen, um Probleme zu lösen, die Inspektionsgebiete mit rechteckigen Geometrien aufweisen. In Simulationsstudien wird die Leistungsfähigkeit der verschiedenen hybriden Genetischen Algorithmen untersucht. Dazu wurde die Inspektion von Tanklagern einer Erdölraffinerie mit einer Gruppe homogener Inspektionsroboter als Anwendungsfall gewählt. Die Simulationen zeigen, dass Kodierungsstrategien, die auf der geometrischen Zerlegung basieren, bei einer kleinen Anzahl an Generationen eine bessere Lösung finden können als die anderen untersuchten Strategien. Diese Arbeit beschäftigt sich mit Einzel- und Zweiroboteraufgaben, die entweder von einem einzelnen mobilen Roboter erledigt werden können oder die Zusammenarbeit von zwei Robotern erfordern. Eine Erweiterung des entwickelten Algorithmus zur Behandlung von Aufgaben, die mehr als zwei Roboter erfordern, ist möglich, würde aber die Komplexität der Optimierungsaufgabe deutlich vergrößern.

Boundaries and Topological Algorithms

This thesis develops a model for the topological structure of situations. In this model, the topological structure of space is altered by the presence or absence of boundaries, such as those at the edges of objects. This allows the intuitive meaning of topological concepts such as region connectivity, function continuity, and preservation of topological structure to be modeled using the standard mathematical definitions. The thesis shows that these concepts are important in a wide range of artificial intelligence problems, including low-level vision, high-level vision, natural language semantics, and high-level reasoning.

A Theory of Quantitative Inference Applied to a Mechanical Design Compiler

This thesis presents the ideas underlying a computer program that takes as input a schematic of a mechanical or hydraulic power transmission system, plus specifications and a utility function, and returns catalog numbers from predefined catalogs for the optimal selection of components implementing the design. Unlike programs for designing single components or systems, the program provides the designer with a high level "language" in which to compose new designs. It then performs some of the detailed design process. The process of "compilation" is based on a formalization of quantitative inferences about hierarchically organized sets of artifacts and operating conditions. This allows the design compilation without the exhaustive enumeration of alternatives.

From Genetic Algorithms to Efficient Organization

The work described in this thesis began as an inquiry into the nature and use of optimization programs based on "genetic algorithms." That inquiry led, eventually, to three powerful heuristics that are broadly applicable in gradient-ascent programs: First, remember the locations of local maxima and restart the optimization program at a place distant from previously located local maxima. Second, adjust the size of probing steps to suit the local nature of the terrain, shrinking when probes do poorly and growing when probes do well. And third, keep track of the directions of recent successes, so as to probe preferentially in the direction of most rapid ascent. These algorithms lie at the core of a novel optimization program that illustrates the power to be had from deploying them together. The efficacy of this program is demonstrated on several test problems selected from a variety of fields, including De Jong's famous test-problem suite, the traveling salesman problem, the problem of coordinate registration for image guided surgery, the energy minimization problem for determining the shape of organic molecules, and the problem of assessing the structure of sedimentary deposits using seismic data.

A Statistical Image-Based Shape Model for Visual Hull Reconstruction and 3D Structure Inference

We present a statistical image-based shape + structure model for Bayesian visual hull reconstruction and 3D structure inference. The 3D shape of a class of objects is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras. Bayesian reconstructions of new shapes are then estimated using a prior density constructed with a mixture model and probabilistic principal components analysis. We show how the use of a class-specific prior in a visual hull reconstruction can reduce the effect of segmentation errors from the silhouette extraction process. The proposed method is applied to a data set of pedestrian images, and improvements in the approximate 3D models under various noise conditions are shown. We further augment the shape model to incorporate structural features of interest; unknown structural parameters for a novel set of contours are then inferred via the Bayesian reconstruction process. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and works even with only a single input view. Using a data set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images.

On the Convergence of Stochastic Iterative Dynamic Programming Algorithms

Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD(lambda) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD(lambda) and Q-learning belong.

Statistical Models for Co-occurrence Data

Modeling and predicting co-occurrences of events is a fundamental problem of unsupervised learning. In this contribution we develop a statistical framework for analyzing co-occurrence data in a general setting where elementary observations are joint occurrences of pairs of abstract objects from two finite sets. The main challenge for statistical models in this context is to overcome the inherent data sparseness and to estimate the probabilities for pairs which were rarely observed or even unobserved in a given sample set. Moreover, it is often of considerable interest to extract grouping structure or to find a hierarchical data organization. A novel family of mixture models is proposed which explain the observed data by a finite number of shared aspects or clusters. This provides a common framework for statistical inference and structure discovery and also includes several recently proposed models as special cases. Adopting the maximum likelihood principle, EM algorithms are derived to fit the model parameters. We develop improved versions of EM which largely avoid overfitting problems and overcome the inherent locality of EM--based optimization. Among the broad variety of possible applications, e.g., in information retrieval, natural language processing, data mining, and computer vision, we have chosen document retrieval, the statistical analysis of noun/adjective co-occurrence and the unsupervised segmentation of textured images to test and evaluate the proposed algorithms.

Memory Usage Inference for Object-Oriented Programs

We present a type-based approach to statically derive symbolic closed-form formulae that characterize the bounds of heap memory usages of programs written in object-oriented languages. Given a program with size and alias annotations, our inference system will compute the amount of memory required by the methods to execute successfully as well as the amount of memory released when methods return. The obtained analysis results are useful for networked devices with limited computational resources as well as embedded software.

Scaling algorithms for distributed max flow

Bibliography: p. 22-24.

Inference of distributional parameters from compositional samples containing nondetects

Low concentrations of elements in geochemical analyses have the peculiarity of being compositional data and, for a given level of significance, are likely to be beyond the capabilities of laboratories to distinguish between minute concentrations and complete absence, thus preventing laboratories from reporting extremely low concentrations of the analyte. Instead, what is reported is the detection limit, which is the minimum concentration that conclusively differentiates between presence and absence of the element. A spatially distributed exhaustive sample is employed in this study to generate unbiased sub-samples, which are further censored to observe the effect that different detection limits and sample sizes have on the inference of population distributions starting from geochemical analyses having specimens below detection limit (nondetects). The isometric logratio transformation is used to convert the compositional data in the simplex to samples in real space, thus allowing the practitioner to properly borrow from the large source of statistical techniques valid only in real space. The bootstrap method is used to numerically investigate the reliability of inferring several distributional parameters employing different forms of imputation for the censored data. The case study illustrates that, in general, best results are obtained when imputations are made using the distribution best fitting the readings above detection limit and exposes the problems of other more widely used practices. When the sample is spatially correlated, it is necessary to combine the bootstrap with stochastic simulation