Co-evolutionary and Reinforcement Learning Techniques Applied to Computer Go players

The objective of this thesis is model some processes from the nature as evolution and co-evolution, and proposing some techniques that can ensure that these learning process really happens and useful to solve some complex problems as Go game. The Go game is ancient and very complex game with simple rules which still is a challenge for the Artificial Intelligence. This dissertation cover some approaches that were applied to solve this problem, proposing solve this problem using competitive and cooperative co-evolutionary learning methods and other techniques proposed by the author. To study, implement and prove these methods were used some neural networks structures, a framework free available and coded many programs. The techniques proposed were coded by the author, performed many experiments to find the best configuration to ensure that co-evolution is progressing and discussed the results. Using co-evolutionary learning processes can be observed some pathologies which could impact co-evolution progress. In this dissertation is introduced some techniques to solve pathologies as loss of gradients, cycling dynamics and forgetting. According to some authors, one solution to solve these co-evolution pathologies is introduce more diversity in populations that are evolving. In this thesis is proposed some techniques to introduce more diversity and some diversity measurements for neural networks structures to monitor diversity during co-evolution. The genotype diversity evolved were analyzed in terms of its impact to global fitness of the strategies evolved and their generalization. Additionally, it was introduced a memory mechanism in the network neural structures to reinforce some strategies in the genes of the neurons evolved with the intention that some good strategies learned are not forgotten. In this dissertation is presented some works from other authors in which cooperative and competitive co-evolution has been applied. The Go board size used in this thesis was 9x9, but can be easily escalated to more bigger boards.The author believe that programs coded and techniques introduced in this dissertation can be used for other domains.

Computational Study of the Fe(CN)(2)CO Cofactor and Its Binding to HypC Protein.

In the intricate maturation process of [NiFe]-hydrogenases, the Fe(CN)2CO cofactor is first assembled in a HypCD complex with iron coordinated by cysteines from both proteins and CO is added after ligation of cyanides. The small accessory protein HypC is known to play a role in delivering the cofactor needed for assembling the hydrogenase active site. However, the chemical nature of the Fe(CN)2CO moiety and the stability of the cofactor–HypC complex are open questions. In this work, we address geometries, properties, and the nature of bonding of all chemical species involved in formation and binding of the cofactor by means of quantum calculations. We also study the influence of environmental effects and binding to cysteines on vibrational frequencies of stretching modes of CO and CN used to detect the presence of Fe(CN)2CO. Carbon monoxide is found to be much more sensitive to sulfur binding and the polarity of the medium than cyanides. The stability of the HypC–cofactor complex is analyzed by means of molecular dynamics simulation of cofactor-free and cofactor-bound forms of HypC. The results show that HypC is stable enough to carry the cofactor, but since its binding cysteine is located at the N-terminal unstructured tail, it presents large motions in solution, which suggests the need for a guiding interaction to achieve delivery of the cofactor.

Functional and expression analysis of the metal-inducible dmeRF system from Rhizobium legumionosarum bv. viciae

A gene encoding a homolog to the cation diffusion facilitator protein DmeF from Cupriavidus metallidurans has been identified in the genome of Rhizobium leguminosarum UPM791. The R. leguminosarum dmeF gene is located downstream of an open reading frame (designated dmeR) encoding a protein homologous to the nickel- and cobalt-responsive transcriptional regulator RcnR from Escherichia coli. Analysis of gene expression showed that the R. leguminosarum dmeRF genes are organized as a transcriptional unit whose expression is strongly induced by nickel and cobalt ions, likely by alleviating the repressor activity of DmeR on dmeRF transcription. An R. leguminosarum dmeRF mutant strain displayed increased sensitivity to Co(II) and Ni(II), whereas no alterations of its resistance to Cd(II), Cu(II), or Zn(II) were observed. A decrease of symbiotic performance was observed when pea plants inoculated with an R. leguminosarum dmeRF deletion mutant strain were grown in the presence of high concentrations of nickel and cobalt. The same mutant induced significantly lower activity levels of NiFe hydrogenase in microaerobic cultures. These results indicate that the R. leguminosarum DmeRF system is a metal-responsive efflux mechanism acting as a key element for metal homeostasis in R. leguminosarum under free-living and symbiotic conditions. The presence of similar dmeRF gene clusters in other Rhizobiaceae suggests that the dmeRF system is a conserved mechanism for metal tolerance in legume endosymbiotic bacteria.

The relationship between soil geochemistry and the bioaccessibility of trace elements in playground soil

A total of 32 samples of surficial soil were collected from 16 playground areas in Madrid (Spain), in order to investigate the importance of the geochemistry of the soil on subsequent bioaccessibility of trace elements. The in vitro bioaccessibility of As, Co, Cr, Cu, Ni, Pb and Zn was evaluated by means of two extraction processes that simulate the gastric environment and one that reproduces a gastric + intestinal digestion sequence. The results of the in vitro bioaccessibility were compared against aqua regia extractions (“total” concentration), and it was found that total concentrations of As, Cu, Pb and Zn were double those of bioaccessible values, whilst that of Cr was ten times higher. Whereas the results of the gastric + intestinal extraction were affected by a high uncertainty, both gastric methods offered very similar and consistent results, with bioaccessibilities following the order: As = Cu = Pb = Zn > Co > Ni > Cr, and ranging from 63 to 7 %. Selected soil properties including pH, organic matter, Fe and CaCO3 content were determined to assess their influence on trace element bioaccessibility, and it was found that Cu, Pb and Zn were predominantly bound to organic matter and, to a lesser extent, Fe oxides. The former fraction was readily accessible in the gastric solution, whereas Fe oxides seemed to recapture negatively charged chloride complexes of these elements in the gastric solution, lowering their bioaccessibility. The homogeneous pH of the playground soils included in the study does not influence trace element bioaccessibility to any significant extent except for Cr, where the very low gastric accessibility seems to be related to the strongly pH-dependent formation of complexes with organic matter. The results for As, which have been previously described and discussed in detail in Mingot et al. (Chemosphere 84: 1386–1391, 2011), indicate a high gastric bioaccessibility for this element as a consequence of its strong association with calcium carbonate and the ease with which these bonds are broken in the gastric solution. The calculation of risk assessments are therefore dependant on the methodology used and the specific environment they address. This has impacts on management strategies formulated to ensure that the most vulnerable of society, children, can live and play without adverse consequences to their health.

Análisis molecular de sistemas génicos implicados en la homeostasis de níquel y eficiencia simbiótica en Rhizobium leguminosarum

Los suelos ultramáficos, que poseen elevadas concentraciones de níquel, cobalto y cromo de manera natural, son fuente de bacterias resistentes a altas concentraciones de metales. Se realizó la caracterización físico-química de seis suelos ultramáficos del suroeste europeo, seleccionándose un suelo de la región de Gorro, Italia, como el más adecuado para aislar bacterias endosimbióticas resistentes a metales. A partir de plantas-trampa de guisante y lenteja inoculados con suspensiones de ese suelo, se obtuvieron 58 aislados de Rhizobium leguminosarum bv. viciae (Rlv) que fueron clasificados en 13 grupos según análisis de PCR-RAPDs. Se determinó la resistencia a cationes metálicos [Ni(II), Co(II), Cu(II), Zn(II)] de una cepa representante de cada grupo, así como la secuencia de los genomas de las cepas que mostraron altos niveles (UPM1137 y UPM1280) y bajos niveles (UPM1131 y UPM1136) de tolerancia a metales. Para identificar mecanismos de resistencia a metales se realizó una mutagénesis al azar en dicha cepa mediante la inserción de un minitransposón. El análisis de 4313 transconjugantes permitió identificar 14 mutantes que mostraron una mayor sensibilidad a Ni(II) que la cepa silvestre. Se determinó el punto de inserción del minitransposón en todos ellos y se analizaron en más detalle dos de los mutantes (D2250 y D4239). En uno de los mutantes (D2250), el gen afectado codifica para una proteína que presenta un 44% de identidad con dmeF (divalent efflux protein) de Cupriavidus metallidurans. Cadena arriba de dmeF se identificó un gen que codifica una proteína con un 39% de identidad con el regulador RcnR de Escherichia coli. Se decidió nombrar a este sistema dmeRF, y se generó un mutante en ambos genes en la cepa Rlv SPF25 (Rlv D15). A partir de experimentos de análisis fenotípico y de regulación se pudo demostrar que el sistema dmeRF tiene un papel relevante en la resistencia a Ni(II) y sobre todo a Co(II) en células en vida libre y en simbiosis con plantas de guisante. Ambos genes forman un operón cuya expresión se induce en respuesta a la presencia de Ni(II) y Co(II). Este sistema se encuentra conservado en distintas especies del género Rhizobium como un mecanismo general de resistencia a níquel y cobalto. Otro de los mutantes identificados (D4239), tiene interrumpido un gen que codifica para un regulador transcripcional de la familia AraC. Aunque inicialmente fue identificado por su sensibilidad a níquel, experimentos posteriores demostraron que su elevada sensibilidad a metales era debida a su sensibilidad al medio TY, y más concretamente a la triptona presente en el medio. En otros medios de cultivo el mutante no está afectado específicamente en su tolerancia a metales. Este mutante presenta un fenotipo simbiótico inusual, siendo inefectivo en guisantes y efectivo en lentejas. Análisis de complementación y de mutagénesis dirigida sugieren que el fenotipo de la mutación podría depender de otros factores distintos del gen portador de la inserción del minitransposón. ABSTRACT Ultramafic soils, having naturally high concentrations of nickel, cobalt and chrome, are potential sources of highly metal-resistant bacteria. A physico-chemical characterization of six ultramafic soils from the European southwest was made. A soil from Gorro, Italy, was chosen as the most appropriated for the isolation of heavy-metal-resistant endosymbiotic bacteria. From pea and lentil trap plants inoculated with soil suspensions, 58 isolates of Rhizobium leguminosarum bv. viciae (Rlv) were obtained and classified into 13 groups based on PCR-RAPDs analysis. The resistance to metallic cations [Ni(II), Co(II), Cu(II), Zn(II)] was analyzed in a representative strain of each group. From the results obtained in the resistance assays, the Rlv UPM1137 strain was selected to identify metal resistance mechanism. A random mutagenesis was made in UPM1137 by using minitransposon insertion. Analysis of 4313 transconjugants allowed to identify 14 mutants with higher sensitivity to Ni(II) than the wild type strain. The insertion point of the minitransposon was determined in all of them, and two mutants (D2250 and D4239) were studied in more detail. In one of the mutants (D2250), the affected gene encodes a protein with 44% identity in compared with DmeF (divalent efflux protein) from Cupriavidus metallidurans. Upstream R. leguminosarum dmeF, a gene encoding a protein with 39% identity with RcnR regulator from E. coli was identified. This protein was named DmeR. A mutant with both genes in the dmeRF deleted was generated and characterized in Rlv SPF25 (Rlv D15). From phenotypic and regulation analysis it was concluded that the dmeRF system is relevant for Ni(II) and specially Co(II) tolerance in both free living and symbiotic forms of the bacteria. This system is conserved in different Rhizobium species like a general mechanism for nickel and cobalt resistance. Other of the identified mutants (D4239) contains the transposon insert on a gene that encodes for an AraC-like transcriptional regulator. Although initially this mutant was identified for its nickel sensitivity, futher experiments demonstrated that its high metal sensitivity is due to its sensitivity to the TY medium, specifically for the tryptone. In other media the mutant is not affected specifically in their tolerance to metals. This mutant showed an unusual symbiotic phenotype, being ineffective in pea and effective in lentil. Complementation analysis and directed mutagenesis suggest that the mutation phenotype could depend of other factors different from the insertion minitransposon gene.

Bioaccessibility of heavy metals and risk assessment in urban gardens of Madrid

Human health problems and solutions. Urban gardening has spread worldwide in recent years as it enhances food security and selfsupply and promotes community integration. However urban soils are significantly enriched in trace elements relative to background levels. Exposure to the soil in urban gardens may therefore result in adverse health effects depending on the degree of contact during gardening, infant recreational activities and ingestion of vegetables grown in them. In order to evaluate this potential risk, 36 composite samples were collected from the top 20 cm of the soil of 6 urban gardens in Madrid. The aqua regia (pseudototal) and glycine-extractable (bioaccessible) concentrations of Co, Cr, Cu, Ni, Pb and Zn were determined by atomic absorption spectrophotometry. Additionally, pH, texture, Fe, Ca, and Mn concentrations, and organic matter and calcium carbonate contents were determined in all urban gardens and their influence on trace element bioaccessibility was analyzed.

Bioaccesibilidad de elementos traza en suelos de parques infantiles. Aplicación al análisis de riesgo

En este trabajo doctoral se evaluó la bioaccesibilidad in vitro para As, Co, Cr, Cu, Ni, Pb y Zn (en la fracción menor de 100μm) por tres procedimientos distintos en 32 muestras de suelo superficial, recogidas en 16 parques infantiles de la ciudad de Madrid.. Dos de los métodos de extracción (SBET y extracción con HCl a pH=1.5) reproducen únicamente la fase gástrica, mientras que otro (RIVM) tiene en cuenta un proceso completo de digestión (gástrico+intestinal). La bioaccesibilidad (%) se definió frente a las concentraciones pseudototales de los elementos traza estudiados (agua regia), utilizando un modelo de regresión lineal pasando por el origen. Los dos métodos gástricos ofrecieron resultados similares y consistentes con datos de otros estudios, siendo el orden de bioaccesibilidad As ≈ Cu ≈ Pb ≈ Zn > Co > Ni > Cr, con rangos entre el 63 y el 7%. Para el procedimiento RIVM (gástrico + intestinal) se obtuvieron valores de un orden similar a los obtenidos en fase gástrica para los elementos As, Cu, Pb y Zn (muy similares para el Zn, algo superiores para Cu y Pb, y algo inferiores para As). Por el contrario, la bioaccesibilidad de Co y Cu es, en este caso, muy superior a la resultante de los ensayos en fase gástrica. El orden de bioaccesibilidad es Co ≈ Cu ≈ Pb > As ≈ Cr ≈ Zn, con rangos entre el 42 y el 69%. Los resultados de los tres procedimientos evaluados correlacionan muy intensamente para los elementos traza As, Cu, Pb y Zn, existiendo intensas correlaciones entre casi todos los elementos estudiados para las dos fases gástricas, no siendo así en el ensayo de digestión completa. Se estudiaron algunas propiedades físico-químicas de los suelos muestreados, así como su composición en algunos elementos mayoritarios con el objeto de evaluar su influencia sobre la bioaccesibilidad. Se observa una dependencia de la bioaccesibilidad (%) de distintos elementos respecto a algunas propiedades de los suelos estudiados, tales como: contenido en Fe, Ca (carbonatos) y P, materia orgánica y pH. El contenido en Fe resulta ser muy relevante en cuanto a la bioaccesibilidad obtenida. En todos los casos correlaciona negativamente con el porcentaje de bioaccesibilidad siendo más significativo este fenómeno en el caso de las extracciones en fase gástrica. Se sugiere que dada la baja solubilización de los óxidos de hierro en los medios extractantes empleados hay una fuerte adsorción de complejos aniónicos (metal-anión cloruro) sobre la superficie de estos óxidos de Fe, con la consiguientes disminución de la bioaccesibilidad. En cuanto al contenido en calcio (carbonatos) este dato parece muy relevante si nos referimos a la bioaccesibilidad del As. Efectivamente el As aparece ligado al Ca del suelo y su solubilización en medios ácidos implicaría un aumento de la bioaccesibilidad del As, mientras que su precipitación al pasar a pH básico (fase intestinal) provocaría una disminución de la bioaccesibilidad. La materia orgánica sólo se ha demostrado relevante respecto a los contenidos pseudototales para el Zn. Para el porcentaje de bioaccesibilidad es significativo para muchos elementos en los ensayos en fase gástrica. La influencia del pH de los suelos estudiados sólo parece ser muy significativo en el caso del Cr. Los valores altamente homogéneos del pH de los suelos estudiados sin duda hacen que este parámetro no resulte significativo para más elementos, tal como se desprende de estudios anteriores. ABSTRACT A total of 32 samples of superficial soil were collected from 16 playground areas in Madrid. The in vitro bioaccessibility of As, Co, Cr, Ni, Pb and Zn (fraction below 100μm) was evaluated by means of three extraction processes. Two of them (SBET and HCl-extraction, pH=1.5) simulate the gastric enviroment, while the other one (RIVM) reproduces a gastric+intestinal digestion sequence. Bioaccessibility (%) was compared against pseudo-total concentrations of trace elements studied (aqua regia) with a linear regression model (forced to intercept the origin) Both gastric methods offered very similar and consistent results with data from other studies, with bioaccessibilities following the order: As ≈ Cu ≈ Pb ≈ Zn > Co > Ni > Cr, and ranging from 63% to 7% The values obtained through RIVM (gastric+ intestinal) method are similar to those obtained in gastric environment for elements: As, Cu, Pb and Zn (very similar to Zn, to a higher extent Cu and Pb, and to a lower extent As). On the contrary the bioaccessibility obtained for elements Co and Cu is considerable higher than in gastric environment sequence. Bioaccessibilities follows the order Co ≈ Cu ≈ Pb >As ≈ Cr ≈ Zn, ranging between 42 and 69%. The three procedures used correlate very intensively to trace elements As,Cu, Pb and Zn, existing strong correlations between almost all elements studied for the two gastric environment, not in the case of the complete digestion sequence. Some soil physical – chemical properties selected were studied, as well as their composition in some main elements in order to assess their influence on bioaccessibility. A dependence was observed between different elements bioaccesibility (%) and some soil properties, such as: Fe, Ca (carbonate) content and P, organic matter and pH. Fe content becomes very relevant regarding the bioaccessibility obtained. In all cases it correlated negatively with bioaccessibility percentage being more significant this phenomenon in gastric environment extractions. It is suggested that given the low solubility of iron oxide in the extractant media used there has to be a strong adsorption of anionic complexes (metal – chloride anion) on these Fe oxides surface, with a consequent decrease of bioaccessibility. Regarding calcium (carbonate) content this data seems very relevant referred to As bioaccessibility. Indeed, As appears to be bound to soil Ca and its solubilisation in acid media would increase As bioaccessibility, while its precipitation at basic pH (intestinal environment) would cause a reduction in bioaccessibility. The influence of organic matter only seemed significant for Zn “total” content, while it is significant in terms of gastric bioaccessibility for many elements. Soil pH only seems to be very significant in case of Cr. The highly homogeneous values for soil pH makes the influence of this parameter negligible for the other elements, unlike what has been observed in several previous studies.