A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

Of milquetoasts and daredevils : personalities in female eiders

Traditionally biologists have often considered individual differences in behaviour or physiology as a nuisance when investigating a population of individuals. These differences have mostly been dismissed as measurement errors or as non-adaptive variation around an adaptive population mean. Recent research, however, challenges this view. While long acknowledged in human personality studies, the importance of individual variation has recently entered into ecological and evolutionary studies in the form of animal personality. The concept of animal personality focuses on consistent differences within and between individuals in behavioural and physiological traits across time and contexts and its ecological and evolutionary consequences. Nevertheless, a satisfactory explanation for the existence of personality is still lacking. Although there is a growing number of explanatory theoretical models, there is still a lack of empirical studies on wild populations showing how traditional life-history tradeoffs can explain the maintenance of variation in personality traits. In this thesis, I first investigate the validity of variation in allostatic load or baseline corticosterone (CORT) concentrations as a measure for differences in individual quality. The association between CORT and quality has recently been summarised under the “CORT-fitness hypothesis”, which states that a general negative relationship between baseline CORT and fitness exists. I then continue to apply the concept of animal personality to depict how the life-history trade-off between survival and fecundity is mediated in incubating female eiders (Somateria mollissima), thereby maintaining variation in behaviour and physiology. To this end, I investigated breeding female eiders from a wild population that breeds in the archipelago around Tvärminne Zoological Station, SW Finland. The field data used was collected from 2008 to 2012. The overall aim of the thesis was to show how differences in personality and stress responsiveness are linked to a life-history context. In the four chapters I examine how the life-history trade-off between survival and fecundity could be resolved depending on consistent individual differences in escape behaviour, stress physiology, individual quality and nest-site selection. First, I corroborated the validity of the “CORT-fitness hypothesis”, by showing that reproductive success is generally negatively correlated with serum and faecal baseline CORT levels. The association between individual quality and baseline CORT is, however, context dependent. Poor body condition was associated with elevated serum baseline CORT only in older breeders, while a larger reproductive investment (clutch mass) was associated with elevated serum baseline CORT among females breeding late in the season. Interestingly, good body condition was associated with elevated faecal baseline CORT levels in late breeders. High faecal baseline CORT levels were positively related to high baseline body temperature, and breeders in poor condition showed an elevated baseline body temperature, but only on open islands. The relationship between stress physiology and individual quality is modulated by breeding experience and breeding phenology. Consequently, the context dependency highlights that this relationship has to be interpreted cautiously. Additionally, I verified if stress responsiveness is related to risk-taking behaviour. Females who took fewer risks (longer flight initiation distance) showed a stronger stress response (measured as an increase in CORT concentration after capture and handling of the bird). However, this association was modulated by breeding experience and body condition, with young breeders and those in poor body condition showing the strongest relationship between risktaking and stress responsiveness. Shy females (longer flight initiation distance) also incubated their clutch for a shorter time. Additionally, I demonstrated that stress responsiveness and predation risk interact with maternal investment and reproductive success. Under high risk of predation, females that incubated a larger clutch showed a stronger stress response. Surprisingly, these females also exhibited higher reproductive success than females with a weaker stress response. Again, these context dependent results suggest that the relationship between stress responsiveness and risk-taking behaviour should not be studied in isolation from individual quality and that stress responsiveness may show adaptive plasticity when individuals are exposed to different predation regimes. Finally, female risk-taking behaviour and stress coping styles were also related to nest-site choice. Less stress responsive females more frequently occupied nests with greater coverage that were farther away from the shoreline. Females nesting in nests with medium cover and farther from the shoreline had higher reproductive success. These results suggest that different personality types are distributed non-randomly in space. In this thesis I was able to demonstrate that personalities and stress coping strategies are persistent individual characteristics, which express measurable effects on fitness. This suggests that those traits are exposed to natural selection and thereby can evolve. Furthermore, individual variation in personality and stress coping strategy is linked to the alternative ways in which animals resolve essential life-history trade-offs.

All-Optical Label Stacking: Easing the Trade-offs Between Routing and Architecture Cost in All-Optical Packet Switching

All-optical label swapping (AOLS) forms a key technology towards the implementation of all-optical packet switching nodes (AOPS) for the future optical Internet. The capital expenditures of the deployment of AOLS increases with the size of the label spaces (i.e. the number of used labels), since a special optical device is needed for each recognized label on every node. Label space sizes are affected by the way in which demands are routed. For instance, while shortest-path routing leads to the usage of fewer labels but high link utilization, minimum interference routing leads to the opposite. This paper studies all-optical label stacking (AOLStack), which is an extension of the AOLS architecture. AOLStack aims at reducing label spaces while easing the compromise with link utilization. In this paper, an integer lineal program is proposed with the objective of analyzing the softening of the aforementioned trade-off due to AOLStack. Furthermore, a heuristic aiming at finding good solutions in polynomial-time is proposed as well. Simulation results show that AOLStack either a) reduces the label spaces with a low increase in the link utilization or, similarly, b) uses better the residual bandwidth to decrease the number of labels even more

Feeding–Danger Trade-Offs Underlie Stopover Site Selection by Migrants

To migrate successfully, birds need to store adequate fat reserves to fuel each leg of the journey. Migrants acquire their fuel reserves at stopover sites; this often entails exposure to predators. Therefore, the safety attributes of sites may be as important as the feeding opportunities. Furthermore, site choice might depend on fuel load, with lean birds more willing to accept danger to obtain good feeding. Here, we evaluate the factors underlying stopover-site usage by migrant Western Sandpipers (Calidris mauri) on a landscape scale. We measured the food and danger attributes of 17 potential stopover sites in the Strait of Georgia and Puget Sound region. We used logistic regression models to test whether food, safety, or both were best able to predict usage of these sites by Western Sandpipers. Eight of the 17 sites were used by sandpipers on migration. Generally, sites that were high in food and safety were used, whereas sites that were low in food and safety were not. However, dangerous sites were used if there was ample food abundance, and sites with low food abundance were used if they were safe. The model including both food and safety best-predicted site usage by sandpipers. Furthermore, lean sandpipers used the most dangerous sites, whereas heavier birds (which do not need to risk feeding in dangerous locations) used safer sites. This study demonstrates that both food and danger attributes are considered by migrant birds when selecting stopover sites, thus both these attributes should be considered to prioritize and manage stopover sites for conservation.

Noise, predation risk compensation and vigilance in the chaffinch Fringilla coelebs

Background noise should in theory hinder detection of auditory cues associated with approaching danger. We tested whether foraging chaffinches Fringilla coelebs responded to background noise by increasing vigilance, and examined whether this was explained by predation risk compensation or by a novel stimulus hypothesis. The former predicts that only inter-scan interval should be modified in the presence of background noise, not vigilance levels generally. This is because noise hampers auditory cue detection and increases perceived predation risk primarily when in the head-down position, and also because previous tests have shown that only interscan interval is correlated with predator detection ability in this system. Chaffinches only modified interscan interval supporting this hypothesis. At the same time they made significantly fewer pecks when feeding during the background noise treatment and so the increased vigilance led to a reduction in intake rate, suggesting that compensating for the increased predation risk could indirectly lead to a fitness cost. Finally, the novel stimulus hypothesis predicts that chaffinches should habituate to the noise, which did not occur within a trial or over 5 subsequent trials. We conclude that auditory cues may be an important component of the trade-off between vigilance and feeding, and discuss possible implications for anti-predation theory and ecological processes

Floral volatiles controlling ant behaviour

P>1. Ants show complex interactions with plants, both facultative and mutualistic, ranging from grazers through seed predators and dispersers to herders of some herbivores and guards against others. But ants are rarely pollinators, and their visits to flowers may be detrimental to plant fitness. 2. Plants therefore have various strategies to control ant distributions, and restrict them to foliage rather than flowers. These 'filters' may involve physical barriers on or around flowers, or 'decoys and bribes' sited on the foliage (usually extrafloral nectaries - EFNs). Alternatively, volatile organic compounds (VOCs) are used as signals to control ant behaviour, attracting ants to leaves and/or deterring them from functional flowers. Some of the past evidence that flowers repel ants by VOCs has been equivocal and we describe the shortcomings of some experimental approaches, which involve behavioural tests in artificial conditions. 3. We review our previous study of myrmecophytic acacias, which used in situ experiments to show that volatiles derived from pollen can specifically and transiently deter ants during dehiscence, the effects being stronger in ant-guarded species and more effective on resident ants, both in African and Neotropical species. In these plants, repellence involves at least some volatiles that are known components of ant alarm pheromones, but are not repellent to beneficial bee visitors. 4. We also present new evidence of ant repellence by VOCs in temperate flowers, which is usually pollen-based and active on common European ants. We use these data to indicate that across a wide range of plants there is an apparent trade-off in ant-controlling filter strategies between the use of defensive floral volatiles and the alternatives of decoying EFNs or physical barriers.

Stops making sense: translational trade-offs and stop codon reassignment

Background Efficient gene expression involves a trade-off between (i) premature termination of protein synthesis; and (ii) readthrough, where the ribosome fails to dissociate at the terminal stop. Sense codons that are similar in sequence to stop codons are more susceptible to nonsense mutation, and are also likely to be more susceptible to transcriptional or translational errors causing premature termination. We therefore expect this trade-off to be influenced by the number of stop codons in the genetic code. Although genetic codes are highly constrained, stop codon number appears to be their most volatile feature. Results In the human genome, codons readily mutable to stops are underrepresented in coding sequences. We construct a simple mathematical model based on the relative likelihoods of premature termination and readthrough. When readthrough occurs, the resultant protein has a tail of amino acid residues incorrectly added to the C-terminus. Our results depend strongly on the number of stop codons in the genetic code. When the code has more stop codons, premature termination is relatively more likely, particularly for longer genes. When the code has fewer stop codons, the length of the tail added by readthrough will, on average, be longer, and thus more deleterious. Comparative analysis of taxa with a range of stop codon numbers suggests that genomes whose code includes more stop codons have shorter coding sequences. Conclusions We suggest that the differing trade-offs presented by alternative genetic codes may result in differences in genome structure. More speculatively, multiple stop codons may mitigate readthrough, counteracting the disadvantage of a higher rate of nonsense mutation. This could help explain the puzzling overrepresentation of stop codons in the canonical genetic code and most variants.

Selection experiments reveal trade-offs between swimming and twitching motilities in pseudomonas aeruginosa

Bacteria possess a range of mechanisms to move in different environments, and these mechanisms have important direct and correlated impacts on the virulence of opportunistic pathogens. Bacteria use two surface organelles to facilitate motility: a single polar flagellum, and type IV pili, enabling swimming in aqueous habitats and twitching along hard surfaces, respectively. Here, we address whether there are trade-offs between these motility mechanisms, and hence whether different environments could select for altered motility. We experimentally evolved initially isogenic Pseudomonas aeruginosa under conditions that favored the different types of motility, and found evidence for a trade-off mediated by antagonistic pleiotropy between swimming and twitching. Moreover, changes in motility resulted in correlated changes in other behaviors, including biofilm formation and growth within an insect host. This suggests environmental origins of a particular motile opportunistic pathogen could predictably influence motility and virulence

Using experimental evolution to explore natural patterns between bacterial motility and resistance to bacteriophages

Resistance of bacteria to phages may be gained by alteration of surface proteins to which phages bind, a mechanism that is likely to be costly as these molecules typically have critical functions such as movement or nutrient uptake. To address this potential trade-off, we combine a systematic study of natural bacteria and phage populations with an experimental evolution approach. We compare motility, growth rate and susceptibility to local phages for 80 bacteria isolated from horse chestnut leaves and, contrary to expectation, find no negative association between resistance to phages and bacterial motility or growth rate. However, because correlational patterns (and their absence) are open to numerous interpretations, we test for any causal association between resistance to phages and bacterial motility using experimental evolution of a subset of bacteria in both the presence and absence of naturally associated phages. Again, we find no clear link between the acquisition of resistance and bacterial motility, suggesting that for these natural bacterial populations, phage-mediated selection is unlikely to shape bacterial motility, a key fitness trait for many bacteria in the phyllosphere. The agreement between the observed natural pattern and the experimental evolution results presented here demonstrates the power of this combined approach for testing evolutionary trade-offs.

Are the antiglobalizers right?: the issue of trade without a Walrasian auctioneer

This paper analyzes the determinants of expectational coordination on the perfect foresight equilibrium of an open economy in the class of one-dimensional models where the price is determined by price expectations. In this class of models, we relate autarky expectational stability conditions to regional integration ones, providing an intuitive open economy interpretation ofthe elasticities condition obtained by Guesnerie [11]. There, we show that the degree of structural heterogeneity trades-off the existence of standard efficiency gains -due to the increase in competition (spatial price stabilization)- and coordination upon the welfare enhancing free-trade equilibrium (stabilizing price expectations). This trade-off provides a new rationale for an exogenous price intervention at the international levei. Through the coordinational concern of the authority, trading countries are ab]e to fully reap the bene:fits from trade. We illustrate this point showing that classical measures evaluating ex-ante the desirability of economic integration (net welfare gains) do not always advise integration between two expectationally stable economies.

Digging effort in leaf-cutting ant queens (Atta sexdens rubropilosa) and its effects on survival and colony growth during the claustral phase

Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP)

Investimento materno em anfíbios anuros: a questão da variação no tamanho do ovo

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Risk of Disease from Wildlife Reservoirs: Badgers, Cattle, and Bovine Tuberculosis

Livestock face complex foraging options associated with optimizing nutrient intake while being able to avoid areas posing risk of parasites or disease. Areas of tall nutrient-rich swards around fecal deposits may be attractive for grazing, but might incur fitness costs from parasites. We use the example of dairy cattle and the risks of tuberculosis transmission posed to them by pastures contaminated with badger excreta to examine this trade-off. A risk may be posed either by aerosolized inhalation through investigation or by ingestion via grazing contaminated swards. We quantified the levels of investigation and grazing of 150 dairy cows at badger latrines (accumulations of feces and urine) and crossing points (urination-only sites). Grazing behavior was compared between strip-grazed and rotation-grazed fields. Strip grazing had fields subdivided for grazing periods of <24 h, whereas rotational grazing involved access to whole fields for 1 to 7 d each. A higher proportion of the herd investigated badger latrines than crossing points or controls. Cattle initially avoided swards around badger latrines but not around crossing points. Avoidance periods were shorter in strip- compared with rotation-grazing systems. In rotation-grazing management, latrines were avoided for longer times, but there were more investigative contacts than with strip-grazing management. If investigation is a major route of tuberculosis transmission, the risk to cattle is greatest in extensive rotation-grazing systems. However, if ingestion of fresh urine is the primary method of transmission, strip-grazing management may pose a greater threat. Farming systems affect the level and type of contact between livestock and wildlife excreta and thus the risks of disease.

Demography in eusocial Hymenoptera

In my doctoral thesis I investigated the evolution of demographic traits within eusocial Hymenoptera. In the social bees, wasps and ants, eusociality has a unique effect on life span evolution as female larvae with the same genetic background can develop through phenotypic plasticity to a queen or a worker with vastly diverging life-history traits. Ant queens belong to the longest-lived insect species, while workers in most species live only a fraction of the queen’s life span. The average colony size of a species is positively correlated with social complexity, division of labor and diverging morphological female phenotypes all of which also affect life span. Therefore the demographic traits of interest in this thesis were life span and colony size. To understand the evolution of worker life span I applied a trade-off model that includes both hierarchical levels important in eusocial systems, namely the colony- and the individual-level. I showed that the evolution of worker life span may be an adaptive trait on the colony level to optimize resource allocation and therefore fitness in response to different levels of extrinsic mortality. A shorter worker life span as a result of reduced resource investments under high levels of extrinsic mortality increases colony fitness. In a further study I showed that Lasius niger colonies produce different aging phenotypes throughout colony development. Smaller colonies which apply a different foraging strategy than larger colonies produced smaller workers, which in turn have a longer life span as compared to larger workers produced in larger colonies. With the switch to cooperative foraging in growing colonies individual workers become less important for the colony caused by their increasing redundancy. Alternatively a trade of between growth and life span may lead to the results found in this study. A further comparative analysis to study the effect of colony size on life span showed a correlation between queen and worker life span when colony size is taken into account. While neither worker nor queen life span was associated with colony size, the differences between queen and worker life span increase with larger average colony sizes across all eusocial Hymenoptera. As colony size affects both queen and worker life span, I aimed to understand which factors lead to the small colony sizes displayed by some ant species. I therefore analyzed per-capita productivity at different colony sizes of eight cavity dwelling ant species. Most colonies of the study species grew larger than optimal productivity predicted. Larger colony size was shown to increase colony homeostasis, the predictability of future productivity and in turn the survival probability of the colony. I also showed that species that deploy an individual foraging mode may circumvent the density dependent decline in foraging success by splitting the colony to several nest sites.

The role of chemotactile cues in interspecific interactions among Central-European arthropod communities

In einem Ökosystem beeinflussen sich Tiere gegenseitig in erster Linie durch direkte Interaktionen. Ihr Verhalten kann aber auch indirekt durch chemotaktile Stoffe die andere Tiere in der Umwelt hinterlassen beeinflusst werden. Vergleichbar zu direkten Interaktionen können indirekt ausgelöste Verhaltensänderungen einen starken Einfluss auf Populationsdynamiken und Gemeinschaftsstrukturen eines Ökosystems haben. Obwohl das daran gehegte Interesse der Ökologen in den letzten Jahrzenten stark gestiegen ist, fehlen immer noch Studien, welche über mehrere Arten hinweg versuchen die übergreifende Relevanz von chemotaktilen Stoffen herauszufinden. Im Rahmen meiner Doktorarbeit untersuchte ich daher wie sich mehrere mitteleuropäische Arthropodenarten, abhängig von deren interspezifischen Räuber-Beute- und Konkurrenzbeziehungen, mittels chemotaktiler Stoffe beeinflussen können. Mithilfe unterschiedlicher Verhaltensversuche konnte ich empirisch nachweisen, dass verschiedene Arthropoden chemotaktile Stoffe zu ihrem eigenen Vorteil nutzen können. Außerdem zeigen meine Ergebnisse, dass die Verhaltensänderungen artspezifisch und abhängig von den jeweiligen Lebenszyklen und den damit verbundenen Eigenschaften (z.B. Körpergröße, Häufigkeit oder Rangordnung) der beteiligten Arten sind. Ich vermute daher, dass Arthropoden chemotaktile Stoffe ihrer Gegenspieler wahrnehmen und interpretieren können. Eine Verhaltensänderung scheint jedoch nur dann statt zu finden wenn ein Nichtreagieren starke Fitnesskosten mit sich führen würde. Zusammenfassend zeigen die Ergebnisse meiner Doktorarbeit, wie wichtig es ist, die Bedeutung von chemotaktilen Stoffen innerhalb vieler Arten einer Gemeinschaft zu testen, um die den Verhaltensänderungen zugrundeliegenden Ursachen identifizieren zu können. Dies wiederum stellt die Grundlage, um die ökologische Relevanz von chemotaktilen Stoffen und deren mögliche Effekte auf Ökosystemfunktionen besser zu verstehen.