Towards Direct Policy Search Reinforcement Learning for Robot Control

This paper proposes a high-level reinforcement learning (RL) control system for solving the action selection problem of an autonomous robot. Although the dominant approach, when using RL, has been to apply value function based algorithms, the system here detailed is characterized by the use of direct policy search methods. Rather than approximating a value function, these methodologies approximate a policy using an independent function approximator with its own parameters, trying to maximize the future expected reward. The policy based algorithm presented in this paper is used for learning the internal state/action mapping of a behavior. In this preliminary work, we demonstrate its feasibility with simulated experiments using the underwater robot GARBI in a target reaching task

Systematic survey of clonal complexity in tuberculosis at a populational level and detailed characterization of the isolates involved.

Clonally complex infections by Mycobacterium tuberculosis are progressively more accepted. Studies of their dimension in epidemiological scenarios where the infective pressure is not high are scarce. Our study systematically searched for clonally complex infections (mixed infections by more than one strain and simultaneous presence of clonal variants) by applying mycobacterial interspersed repetitive-unit (MIRU)-variable-number tandem-repeat (VNTR) analysis to M. tuberculosis isolates from two population-based samples of respiratory (703 cases) and respiratory-extrapulmonary (R+E) tuberculosis (TB) cases (71 cases) in a context of moderate TB incidence. Clonally complex infections were found in 11 (1.6%) of the respiratory TB cases and in 10 (14.1%) of those with R+E TB. Among the 21 cases with clonally complex TB, 9 were infected by 2 independent strains and the remaining 12 showed the simultaneous presence of 2 to 3 clonal variants. For the 10 R+E TB cases with clonally complex infections, compartmentalization (different compositions of strains/clonal variants in independent infected sites) was found in 9 of them. All the strains/clonal variants were also genotyped by IS6110-based restriction fragment length polymorphism analysis, which split two MIRU-defined clonal variants, although in general, it showed a lower discriminatory power to identify the clonal heterogeneity revealed by MIRU-VNTR analysis. The comparative analysis of IS6110 insertion sites between coinfecting clonal variants showed differences in the genes coding for a cutinase, a PPE family protein, and two conserved hypothetical proteins. Diagnostic delay, existence of previous TB, risk for overexposure, and clustered/orphan status of the involved strains were analyzed to propose possible explanations for the cases with clonally complex infections. Our study characterizes in detail all the clonally complex infections by M. tuberculosis found in a systematic survey and contributes to the characterization that these phenomena can be found to an extent higher than expected, even in an unselected population-based sample lacking high infective pressure.

Proposal of a parallel architecture for a motion detection algorithm

This paper proposes a parallel architecture for estimation of the motion of an underwater robot. It is well known that image processing requires a huge amount of computation, mainly at low-level processing where the algorithms are dealing with a great number of data. In a motion estimation algorithm, correspondences between two images have to be solved at the low level. In the underwater imaging, normalised correlation can be a solution in the presence of non-uniform illumination. Due to its regular processing scheme, parallel implementation of the correspondence problem can be an adequate approach to reduce the computation time. Taking into consideration the complexity of the normalised correlation criteria, a new approach using parallel organisation of every processor from the architecture is proposed

On optimal computation of MPLS label binding for multipoint-to-point connections

Most network operators have considered reducing Label Switched Routers (LSR) label spaces (i.e. the number of labels that can be used) as a means of simplifying management of underlaying Virtual Private Networks (VPNs) and, hence, reducing operational expenditure (OPEX). This letter discusses the problem of reducing the label spaces in Multiprotocol Label Switched (MPLS) networks using label merging - better known as MultiPoint-to-Point (MP2P) connections. Because of its origins in IP, MP2P connections have been considered to have tree- shapes with Label Switched Paths (LSP) as branches. Due to this fact, previous works by many authors affirm that the problem of minimizing the label space using MP2P in MPLS - the Merging Problem - cannot be solved optimally with a polynomial algorithm (NP-complete), since it involves a hard- decision problem. However, in this letter, the Merging Problem is analyzed, from the perspective of MPLS, and it is deduced that tree-shapes in MP2P connections are irrelevant. By overriding this tree-shape consideration, it is possible to perform label merging in polynomial time. Based on how MPLS signaling works, this letter proposes an algorithm to compute the minimum number of labels using label merging: the Full Label Merging algorithm. As conclusion, we reclassify the Merging Problem as Polynomial-solvable, instead of NP-complete. In addition, simulation experiments confirm that without the tree-branch selection problem, more labels can be reduced

Speculative parallelization of multipath radiosity algorithm

In computer graphics, global illumination algorithms take into account not only the light that comes directly from the sources, but also the light interreflections. This kind of algorithms produce very realistic images, but at a high computational cost, especially when dealing with complex environments. Parallel computation has been successfully applied to such algorithms in order to make it possible to compute highly-realistic images in a reasonable time. We introduce here a speculation-based parallel solution for a global illumination algorithm in the context of radiosity, in which we have taken advantage of the hierarchical nature of such an algorithm

Prevalent role of TCR alpha-chain in the selection of the preimmune repertoire specific for a human tumor-associated self-antigen.

The specificity of recognition of pMHC complexes by T lymphocytes is determined by the V regions of the TCR alpha- and beta-chains. Recent experimental evidence has suggested that Ag-specific TCR repertoires may exhibit a more V alpha- than V beta-restricted usage. Whether V alpha usage is narrowed during immune responses to Ag or if, on the contrary, restricted V alpha usage is already defined at the early stages of TCR repertoire selection, however, has remained unexplored. Here, we analyzed V and CDR3 TCR regions of single circulating naive T cells specifically detected ex vivo and isolated with HLA-A2/melan-A peptide multimers. Similarly to what was previously observed for melan-A-specific Ag-experienced T cells, we found a relatively wide V beta usage, but a preferential V alpha 2.1 usage. Restricted V alpha 2.1 usage was also found among single CD8(+) A2/melan-A multimer(+) thymocytes, indicating that V alpha-restricted selection takes place in the thymus. V alpha 2.1 usage, however, was independent from functional avidity of Ag recognition. Thus, interaction of the pMHC complex with selected V alpha-chains contributes to set the broad Ag specificity, as underlined by preferential binding of A2/melan-A multimers to V alpha 2.1-bearing TCRs, whereas functional outcomes result from the sum of these with other interactions between pMHC complex and TCR.

A Monte Carlo-Based Fiber Tracking Algorithm using Diffusion Tensor MRI

Diffusion tensor magnetic resonance imaging, which measures directional information of water diffusion in the brain, has emerged as a powerful tool for human brain studies. In this paper, we introduce a new Monte Carlo-based fiber tracking approach to estimate brain connectivity. One of the main characteristics of this approach is that all parameters of the algorithm are automatically determined at each point using the entropy of the eigenvalues of the diffusion tensor. Experimental results show the good performance of the proposed approach

Intra- and intersexual selection and their potential consequences on fitness-relevant traits in several fish species

Résumé : Les mécanismes de sélection sexuelle, en particulier la compétition entre mâles (sélection inter-sexuelle) et le choix des femelles (sélection intra-sexuelle), peuvent fortement influencer le succès reproducteur d'un individu, c'est-à-dire son nombre de descendants. On observe ainsi que les mâles dominants et les mâles élaborant des caractères sexuels secondaires marqués ont un succès reproducteur élevé. Toutefois, le succès reproducteur ne suffit pas pour garantir une contribution génétique élevée, parce que la fitness dépend également de la performance des descendants (c'est-à-dire de leur survie et de leur propre succès reproducteur). Si cette performance dépend en partie des gènes paternels, les males ont un avantage certain à signaler leur qualité aux femelles afin d'atteindre des taux de reproduction élevé. Ce mécanisme de signalisation est connu sous le nom de 'good genes hypothesis', toutefois très peu d'études ont clairement démontré le lien entre la qualité génétique des individus et la signalisation. De plus, la performance des descendants peut aussi dépendre des effets génétiques de compatibilité entre mâles et femelles ('compatible genes'). C'est-à-dire que certains allèles paternels n'apporteraient un avantage aux descendants qu'en combinaison avec certains allèles maternels. Nous avons déterminé, durant la période de reproduction, le statut de dominance des mâles pour deux espèces de poissons d'eau douce : la truite (Salmo trotta) et le vairon (Phoxinus phoxinus), puis nous avons évalué la relation entre le succès reproducteur et le statut de dominance et/ou la quantité de signalisation des caractères sexuels secondaires. Nous avons également fécondés artificiellement des oeufs de truites et de corégones (Coregonus palaea), en croisant chaque mâle avec chaque femelle (full-factorial breeding design). Ce type de design autorise la quantification précise des effets génétiques et permet de séparer les effets de 'good genes' et de 'compatible genes'. Cela a été fait sous différentes intensités de stress bactérien, ainsi que dans des conditions naturelles, et nous avons pu ainsi tester si certains indicateurs de qualité génétique des mâles ('good genes') étaient liés a) à la dominance et/ou b) à l'expression des caractères sexuels secondaires des mâles comme l'intensité mélanique ou la taille des tubercules sexuels. En outre, nous cherchons à savoir si la survie des descendants est liée à certaines combinaison des gènes du complexe d'histocompatibilité majeur (MHC) et/ou à la parenté génétique des parents, les deux traits étant soupçonnés d'avoir des influences génétique de compatibilité (`compatible genes') à la performance des descendants. Nous avons constaté que la dominance des mâles est directement liée à la taille et au poids des mâles (truites, vairons), mais également aux caractères sexuels secondaires (tubercules). De plus, les mâles vairons dominant ont eu un succès de fécondation plus élevés que les mâles subordonnés. Nous montrons que les truites et corégones mâles diffèrent dans leur qualité génétique, qui a été mesurée avéc la survie embryonnaire, le temps avant l'éclosion et enfin la croissance juvénile. Contrairement aux prédictions, la dominance (ou les traits indicatifs de dominance) n'était liée à la qualité génétique, dans aucun des traitements, et ne fonctionne donc pas comme indicateur de qualité. Par contre, la qualité génétique était liée aux caractères sexuels secondaires, particulièrement par la teinte mélanique chez les truites. Les embryons de truites issus de pères sombres survivaient mieux que ceux issus de pères clairs dans des environnements difficiles, de plus leur croissance était plus élevée lors de leur première année dans des conditions naturelles. La taille des juvéniles lors de leur première année est un trait important lié au succès dans la compétition pour des ressources telles qu'abri ou nourriture. De plus, les femelles truites peuvent augmenter la survie de leurs descendants en choisissant des mâles selon leur type de MHC ou selon leur degré de parenté. En outre, chez les corégones, la morphologie des tubercules sexuels ne semble pas signaler la qualité génétique. Nous avons également remarqué que l'exposition à des pathogènes non-létaux pouvait influencer la performance des alevins à court et long terme, probablement en affaiblissant leur système immunitaire. Cette thèse montre que les mâles diffèrent dans leur qualité génétique et que différents mécanismes de sélection inter- ou intra-sexuelle (par exemple la préférence pour des mâles sombres, pour des génotypes MHC ou pour des couples avec degré de parenté basse) pouvait avoir un effet positif sur la qualité des descendants, bien que cet effet génétique pouvait changer au cours du temps et entre différents environnements. Contrairement à nos attentes, le résultat de la compétition intra-sexuelle (la hiérarchie de dominance entre mâles) n'était pas lié à la qualité génétique individuelle ('good genes'). Dans ce sens, ce travail permet également de contribuer à l'explication du fait que la sélection sexuelle, de par sa forte sélection directionnelle, ne conduit pas à la diminution de la variance génétique, mais plutôt à la maintenance du polymorphisme génétique. Summary : Sexual selection mechanisms, especially male-male competition (inteasexual selection) and female mate choice (inteasexual selection), can strongly influence individual mating success, often resulting in dominant males and males with elaborate secondary sexual characters having higher fertilisation success. However, siring a high number of offspring alone does not guarantee high individual fitness, as fitness does also strongly depend on offspring performance (i.e. survival, fecundity). If this superiority in offspring performance depends on paternally inherited genes, the fathers are expected to signal this potential indirect benefit to females in order to attain high mating rates. This mechanism is also known as the 'good genes' hypothesis of sexual selection but until now most studies failed to conclusively show the relation of an individual genetic quality and its potential signalling traits. Further, offspring performance could also depend on compatible gene effects. These are alleles that increase offspring performance only in combination with other specific alleles. We first determined male dominance status from intrasexual competition during mating season for brown trout (Salmo trutta) and European minnows (Phoxinus phoxinus). For minnows we additionally checked if dominance and/or secondary sexual traits were linked to fertilisation success. Further, we artificially fertilised brown trout and alpine whitefish (Coregonus palaea) eggs, following full factorial breeding designs, enabling to properly measure `good gene' and `compatible gene' effects on offspring performance. This was done under different intensities of natural stressors, as well as under natural conditions. This procedure allowed us to test if the obtained male genetic quality measures (good genes effects) were indicated by a) dominance or lay traits linked to dominance and/or by b) secondary sexual characteristics such as melanin-based male skin darkness or breeding tubercles. Further, we investigated if offspring survival was linked to the MHC (major histocompatibility complex) gene combinations and/or to the parental genetic relatedness, as both traits were shown to have 'compatible gene' effects that may influence offspring performance. We found that male dominance in intrasexual competition was positively linked to body size, body weight (brown trout, minnows) but also to elaborate secondary sexual characteristics (breeding tubercles in minnows). Further, dominant minnow males did have an increased fertilisation success compared to subordinate ones. We show that brown trout and whitefish males do usually differ in their genetic quality, which was measured as embryo survival, hatching timing and finally as juvenile growth. Contrary to prediction male dominance or dominance indicating traits do not function as a quality signal as they were not linked to genetic quality. This result was constant when measuring genetic quality under different levels of natural stressors and under natural conditions (brown trout). On the other hand genetic quality seemed to be indicated by secondary sexual characteristics, specifically by melanin-based skin darkness in brown trout as brown trout embryos sired by darker fathers had increased survival rates when raised under harsh conditions and. they grew larger as juveniles after one year of growth in a natural stream, which is an important trait influencing success of juveniles in competition for hidings, food and other resources. Furthermore, brown trout females may increase the survival of their embryos when choosing males according to their MHC genotypes or to the general genetic relatedness between themselves and their potential mates. In whitefish on the other hand breeding tubercle morphology did not seem to signal genetic quality. Eventually, we saw that anon-lethal exposure to pathogens might influence short term and long term offspring performance probably by weakening an exposed individual's immune system. This thesis shows that males usually differ in their genetic quality and that different inter- or intrasexual selection mechanisms (e.g. mate selection favouring dark males, preference for MHC genotype combinations or for unrelated mates) may have strong positive effects on genetically dependent offspring performance but that such genetìc effects can change over time and environments. In contrast to our a priori expectations, the outcome of intrasexual selection, namely male dominance hierarchies, with dominant males often having high fertilisation success, was not linked to individual genetic quality (`good genes'). In this sense the present thesis may also be a helpful contribution to understand why sexual selection does not lead to rapid loss of genetic variation by strong directional selection but could even lead to the maintenance of genetic variation in natural populations.

Constraint algorithm for k-presymplectic Hamiltonian systems. Application to singular field theories

The k-symplectic formulation of field theories is especially simple, since only tangent and cotangent bundles are needed in its description. Its defining elements show a close relationship with those in the symplectic formulation of mechanics. It will be shown that this relationship also stands in the presymplectic case. In a natural way,one can mimick the presymplectic constraint algorithm to obtain a constraint algorithmthat can be applied to k-presymplectic field theory, and more particularly to the Lagrangian and Hamiltonian formulations offield theories defined by a singular Lagrangian, as well as to the unified Lagrangian-Hamiltonian formalism (Skinner--Rusk formalism) for k-presymplectic field theory. Two examples of application of the algorithm are also analyzed.

The stepwise selection for ketoconazole resistance induces upregulation of C14-demethylase (CYP51) in Leishmania amazonensis

Ketoconazole is a clinically safe antifungal agent that also inhibits the growth of Leishmania spp. A study was undertaken to determine whether Leishmania parasites are prone to becoming resistant to ketoconazole by upregulating C14-demethylase after stepwise pharmacological pressure. Leishmania amazonensis promastigotes [inhibitory concentration (IC)50 = 2 µM] were subjected to stepwise selection with ketoconazole and two resistant lines were obtained, La8 (IC50 = 8 µM) and La10 (IC50 = 10 µM). As a result, we found that the resistance level was directly proportional to the C14-demethylase mRNA expression level; we also observed that expression levels were six and 12 times higher in La8 and La10, respectively. This is the first demonstration that L. amazonensis can up-regulate C14-demethylase in response to drug pressure and this report contributes to the understanding of the mechanisms of parasite resistance.

Retroviral Integration Site Selection

The stable insertion of a copy of their genome into the host cell genome is an essential step of the life cycle of retroviruses. The site of viral DNA integration, mediated by the viral-encoded integrase enzyme, has important consequences for both the virus and the host cell. The analysis of retroviral integration site distribution was facilitated by the availability of the human genome sequence, revealing the non-random feature of integration site selection and identifying different favored and disfavored genomic locations for individual retroviruses. This review will summarize the current knowledge about retroviral differences in their integration site preferences as well as the mechanisms involved in this process.

Human genetic selection on the MTHFR 677C>T polymorphism.

BACKGROUND The prevalence of genotypes of the 677C>T polymorphism for the MTHFR gene varies among humans. In previous studies, we found changes in the genotypic frequencies of this polymorphism in populations of different ages, suggesting that this could be caused by an increase in the intake of folate and multivitamins by women during the periconceptional period. The aim was to analyze changes in the allelic frequencies of this polymorphism in a Spanish population, including samples from spontaneous abortions (SA). METHODS A total of 1305 subjects born in the 20th century were genotyped for the 677C>T polymorphism using allele specific real-time PCR with Taqman probes. A section of our population (n = 276) born in 1980-1989 was compared with fetal samples (n = 344) from SA of unknown etiology from the same period. RESULTS An increase in the frequency of the T allele (0.38 vs 0.47; p < 0.001) and of the TT genotype (0.14 vs 0.24; p < 0.001) in subjects born in the last quarter of the century was observed. In the 1980-1989 period, the results show that the frequency of the wild type genotype (CC) is about tenfold lower in the SA samples than in the controls (0.03 vs 0.33; p < 0.001) and that the frequency of the TT genotype increases in the controls (0.19 to 0.27) and in the SA samples (0.20 to 0.33 (p < 0.01)); r = 0.98. CONCLUSION Selection in favor of the T allele has been detected. This selection could be due to the increased fetal viability in early stages of embryonic development, as is deduced by the increase of mutants in both living and SA populations.

Clonal multidrug-resistant Corynebacterium striatum within a nosocomial environment, Rio de Janeiro, Brazil

Corynebacterium striatum is a potentially pathogenic microorganism with the ability to produce outbreaks of nosocomial infections. Here, we document a nosocomial outbreak caused by multidrug-resistant (MDR) C. striatum in Rio de Janeiro, Brazil. C. striatum identification was confirmed by 16S rRNA and rpoB gene sequencing. Fifteen C. striatum strains were isolated from adults (half of whom were 50 years of age and older). C. striatum was mostly isolated in pure culture from tracheal aspirates of patients undergoing endotracheal intubation procedures. The analysis by pulsed-field gel electrophoresis (PFGE) indicated the presence of four PFGE profiles, including two related clones of MDR strains (PFGE I and II). The data demonstrated the predominance of PFGE type I, comprising 11 MDR isolates that were mostly isolated from intensive care units and surgical wards. A potential causal link between death and MDR C. striatum (PFGE types I and II) infection was observed in five cases.

Significant genetic and phenotypic changes arising from clonal growth of a single spore of an arbuscular mycorrhizal fungus over multiple generations.

Arbuscular mycorrhizal fungi (AMF) are highly successful plant symbionts. They reproduce clonally producing multinucleate spores. It has been suggested that some AMF harbor genetically different nuclei. However, recent advances in sequencing the Glomus irregulare genome have indicated very low within-fungus polymorphism. We tested the null hypothesis that, with no genetic differences among nuclei, no significant genetic or phenotypic variation would occur among clonal single spore lines generated from one initial AMF spore. Furthermore, no additional variation would be expected in the following generations of single spore lines. Genetic diversity contained in one initial spore repeatedly gave rise to genetically different variants of the fungus with novel phenotypes. The genetic changes represented quantitative changes in allele frequencies, most probably as a result of changes in the frequency of genetic variation partitioned on different nuclei. The genetic and phenotypic variation is remarkable, given that it arose repeatedly from one clonal individual. Our results highlight the dynamic nature of AMF genetics. Even though within-fungus genetic variation is low, some is probably partitioned among nuclei and potentially causes changes in the phenotype. Our results are important for understanding AMF genetics, as well as for researchers and biotechnologists hoping to use AMF genetic diversity for the improvement of AMF inoculum.