Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

Fibers with integrated mechanochemical switches: minimalistic design principles derived from fibronectin.

Inspired by molecular mechanisms that cells exploit to sense mechanical forces and convert them into biochemical signals, chemists dream of designing mechanochemical switches integrated into materials. Using the adhesion protein fibronectin, whose multiple repeats essentially display distinct molecular recognition motifs, we derived a computational model to explain how minimalistic designs of repeats translate into the mechanical characteristics of their fibrillar assemblies. The hierarchy of repeat-unfolding within fibrils is controlled not only by their relative mechanical stabilities, as found for single molecules, but also by the strength of cryptic interactions between adjacent molecules that become activated by stretching. The force-induced exposure of cryptic sites furthermore regulates the nonlinearity of stress-strain curves, the strain at which such fibers break, and the refolding kinetics and fraction of misfolded repeats. Gaining such computational insights at the mesoscale is important because translating protein-based concepts into novel polymer designs has proven difficult.

Genomic organization and expression analysis of Toll-like receptor 3 in grass carp (Ctenopharyngodon idella)

Toll-like receptor 3 (TLR3) participates in the innate immune response by recognizing viral pathogens. To investigate grass carp immune system responding to GCRV (grass carp reovirus) infection, the full-length cDNA sequence and genomic organization of grass carp TLR3 (CiTLR3) was identified and characterized. The full-length genome sequence of CiTLR3 is composed of 5668 nucleotides, including five exons and four introns. The full-length of CiTLR3 cDNA is 3681 bp in length and encodes a polypeptide of 904 amino acids with an estimated molecular mass of 102,765 Da and a predicted isoelectric point of 8.35. Analysis of the deduced amino acid sequence indicated that CiTLR3 has four main structural domains, including a signal peptide sequence, 14 LRR (leucine-rich repeat) motifs, a transmembrane region and a TIR (Toll/interleukin-1 receptor) domain. It is most similar to the crucian carp (Carassius auratus) TLR3 amino acid sequence with an identity of 99%. Quantitative RT-PCR analysis showed that CiTLR3 transcripts were significantly up-regulated starting at day 1 and continued through day 7 following GCRV infection (P < 0.05). These data implied that CiTLR3 is involved in antiviral defense, provide molecular and functional information for grass carp TLR3, and implicate their role in mediating immune protection against grass carp viral diseases. (C) 2009 Elsevier Ltd. All rights reserved.

Toll-like receptor 4 signaling pathway can be triggered by grass carp reovirus and Aeromonas hydrophila infection in rare minnow Gobiocypris rarus

Toll-like receptor 4 (TLR4) is critical for LPS recognition and cellular responses. It also recognizes some viral envelope proteins. Detection mostly results in the inflammation rather than specific antiviral responses. However, it's unclear in fish. In this report, a TLR4 gene (named as GrTLR4b) was cloned and characterized from rare minnow Gobiocypris rarus. The full length of GrTLR4b cDNA consists of 2766 nucleotides and encodes a polypeptide of 818 amino acids with an estimated molecular mass of 94,518 Da and a predicted isoelectric point of 8.41. The predicted amino acid sequence comprises a signal peptide, six leucine-rich repeat (LRR) motifs, one leucine-rich repeat C-terminal (LRRCT) motif, followed by a transmembrane segment of 23 amino acids, and a cytoplasmic region of 167 amino acids containing one Toll - interleukin 1 - receptor (TIR) motif. It's closely similar to the zebrafish (Danio rerio) TLR4b amino acid sequence with an identity of 77%. Quantitative RT-PCR analysis showed GrTLR4b mRNA was constitutive expression in gill, heart, intestine, kidney, liver, muscle and spleen tissues in healthy animals and up-regulated by viruses and bacteria. After being infected by grass carp reovirus or Aeromonas hydrophila, GrTLR4b expressions were up-regulated from 24 h post-injection and lasted until the fish became moribund (P < 0.05). These data implied that TLR4 signaling pathway could be activated by both viral and bacterial infection in rare minnow. (C) 2009 Elsevier Ltd. All rights reserved.

Cloning, characterization, and gene expression analysis of a novel cadmium metallothionein gene in Tetrahymena pigmentosa

A novel cadmium-inducible metallothionein (MT) gene (Tpig-MT1) was cloned and sequenced from the ciliate Tetrahymena pigmentosa. The number of deduced amino acids is 118. The polypeptide possesses CCC and CC clusters characteristic of typical Tetrahymena Cd-inducible MTs. The structure of Tpig-MT1 is different from the reported Cd-MT in T. pyriformis, T. thermophila and T. pigmentosa. Tpig-MT1 contains two intragenic tandem repeats with 72.9% identity described as Tpig-MT1 (repeat A1) and Tpig-MT1 (repeat A2). The transcriptional response of Tpig-MT1 gene to different heavy metals (Cd, Cu, Zn, Hg, Pb) and oxidative stress (H2O2) was measured using real-time quantitative PCR. The results showed that the gene was quickly induced (1 h) by the five heavy metals and the order of expression level was Hg>Pb>Cd>Cu>Zn. The induction effect of H2O2 was 5-fold after about 15 min, but soon decreased to a non-significant level (30 min). The genetic diversity of Tetrahymena MT genes is discussed in relation to the unique structure of the Tpig-MT1 gene and other reported Cd-MT isoforms. (C) 2008 Elsevier B.V. All rights reserved.

Genetic variation of fish parasite populations in historically connected habitats: undetected habitat fragmentation effect on populations of the nematode Procamallanus fulvidraconis in the catfish Pelteobagrus fulvidraco

Habitat fragmentation may have some significant effects on population genetic structure because geographic distance and physical barriers may impede gene flow between populations. In this study, we investigated whether recent habitat fragmentation affected genetic structure and diversity of populations of the nematode Procamallanus fulvidraconis in the yellowhead catfish, Pelteobagrus fin't4draco. The nematode was collected from 12 localities in 7 floodplain lakes of the Yangtze River. Using I I intersirnple sequence repeat markers, analysis of molecular variance showed that genetic diversity occurred mainly within populations (70.26%). Expected heterozygosity (He) of P. fulvidraconis was barely different between connected (0.2105) and unconnected lakes (0.2083). Population subdivision (Fst) between connected lakes (0.2177) was higher than in unconnected lakes (0. 1676). However, the connected and unconnected lakes did not Cluster into 2 clades. A Mantel test revealed significant positive correlation between genetic and geographic distances (R = 0.5335, P < 0.01). These results suggest that habitat fragmentation did not cause genetic differentiation among populations or a reduction of diversity in isolated populations of P. fulvidraconis. At least 2 factors may increase the dispersal range of the nematode, i.e., flash flooding in summer and other species of fish that may serve as the definitive hosts. Moreover, lake fragmentation is probably a recent process; population size of the nematode in these lakes is large enough to maintain Population structure.

Gene structures and promoter characteristics of interferon regulatory factor 1 (IRF-1), IRF-2 and IRF-7 from snakehead Channa argus

Three interferon regulatory factor (IRF) genes, CaIRF-1, CaIRF-2 and CaIRF-7, and their promoters of snakehead (Channa argus) were cloned and characterized. The CaIRF-1 gene consists of ten exons, spans 4.3 kb and encodes a putative peptide of 299 aa. The CaIRF-2 gene consists of nine exons, spans 8 kb and encodes a putative peptide of 328 aa. The gene organizations of CaIRF-1 and CaIRF-2 are very similar to that of human IRF-1 and IRF-2 except more compact. Comparison of exon-intron organization of the two genes indicated a common evolutionary structure, notably within the exons encoding the DNA binding domain (DBD) of the two factors. The CaIRF-7 gene spans 4.1 kb and encodes a putative peptide of 437 aa. However, the gene organization of CaIRF-7 consisting of ten exons is different to human IRF-7a gene which has an intron in 5' UTR. Three CaIRFs share homology in N-terminal encompassing the DBD that contains a characteristic repeat of tryptophan residues. The promoters of CaIRF-1 and CaIRF-2 genes contain the conserved sites for NF-kappa B and Sp1. The gamma-IFN activation sites (GAS) were found in the promoters of CaIRF-1 and CaIRF-7. The promoter of CaIRF-7 contains conserved interferon stimulating response element (ISRE) which is characteristic of IFN-induced gene promoter, and suggests that there also exist intracellular amplifier circuit in fish IFN signal pathway. Moreover, the element GAAANN oriented in both directions is repeated in CaIRF promoter regions, which confers to further inducibility by IFN. The constitutive expression of CaIRF genes were found to increase obviously in response to induction by the known IFN-inducer poly I:C. (c) 2008 Published by Elsevier Ltd.

Development of novel EST-SSR markers in common carp by data mining from public EST sequences

Expressed sequence tags (ESTs) are a source for microsatellite development. In the present study, EST-derived microsatelltes (EST-SSRs) were generated and characterized in the common carp (Cyprinus carpio) by data mining from updated public EST databases and by subsequent testing for polymorphism. About 5.5% (555) of 10,088 ESTs contain repeat motifs of various types and lengths with CA being the most abundant dinucleotide one. Out of the 60 EST-SSRs for which PCR primers were designed, 25 loci showed polymorphism in a common carp population with the alleles per locus ranging from 3 to 17 (mean 7). The observed (H-O) and expected (HE) heterozygosities of these EST-SSRs were 0.13-1.00 and 0.12-0.91, respectively. Six EST-SSR loci significantly deviated from the Hardy-Weinberg equilibrium (HWE) expectation, and the remaining 19 loci were in HWE. Of the 60 primer sets, the rates of polymorphic EST-SSRs were 42% in common carp, 17% in crucian carp (Carassius auratus), and 5% in silver carp (Hypophthalmichthys molitrix), respectively. These new EST-SSR markers would provide sufficient polymorphism for population genetic studies and genome mapping of the common carp and its closely related fishes. (c) 2007 Published by Elsevier B.V.

Identification of the Angel-related elements in cyprinid fishes and their phylogenetic implications

Angel-related element belongs to the family of miniature inverted-repeat transposable elements (MITEs). In this paper we report the identification of an Angel-related element in the series Leuciscini of cyprinid fishes, which is located in the second intron of the growth hormone (GH) gene. We have also found that this element is absent in orthologous locus in the series Barbini of cyprinid fishes, that provides new evidence for the monophyly of the series Leuciscini. The insertion of Angel-related element into the GH gene might take place in the common ancestor of the series Leuciscini after its divergence from the series Barbini. The high sequence divergence and relatively broad species distribution of Angel-related elements implies that they might be ancient transposons which appeared about 26 million years ago.

Cloning and expression of medaka dazl during ernbryogenesis and gametogenesis

The Deleted in azoospermia family consists of RNA-binding proteins Bottle, Daz, and Daz-like (Dazl) that are expressed in the germline. Here, we report the cloning and expression of the medakafish (Oryzias latipes) dazl gene (odazl). Interestingly, although the predicted medaka Dazl protein (oDazl) contains a RRM motif and a DAZ repeat characteristic of its mammalian homologs, it lacks 80 aa at the C-terminus. By RT-PCR, RNA in situ hybridization, Western blotting and fluorescent immunohistochemistry using a rabbit anti-DazI antibody (alpha Oazl), we analyzed the expression patterns of odazl and its protein. The odazl transcript persists throughout embryogenesis and delineates with primordial germ cells. In adults, the expression of odazl RNA and its protein is restricted to germ cells of both the testis and ovary. We observed differential expression of RNA and protein at critical stages of gametogenesis. In the testis, the odazl RNA is low at premeiotic stages, abundant at meiotic stages, but absent in postmeiotic stages; whereas the oDazl protein is rich in premeiotic stages, reduced at meiotic stages, becomes barely detectable or absent in postmeiotic round spermatids or sperm, respectively. This is in sharp mature spermatozoa. In the ovary, the odazl RNA contrast to the human situation where the Dazl transcript and protein are present in and protein persist throughout oogenesis and also show differential expression at premeiotic, meiotic and postmeiotic stages. Thus, the odazl or its protein is a marker for germ cells during embryogenesis and at critical stages of gametogenesis in both sexes of medaka. (c) 2006 Elsevier B.V. All rights reserved.

Population genetic structure of Carchesium polypinum (Ciliophora : Peritrichia) in four chinese lakes inferred from ISSR fingerprinting: High diversity but low differentiation

Although the peritrichous ciliate Carchesium polypinum is common in freshwater, its population genetic structure is largely unknown. We used inter-simple sequence repeat (ISSR) fingerprinting to analyze the genetic structure of 48 different isolates of the species from four lakes in Wuhan, central China. Using eight polymorphic primers, 81 discernible DNA fragments were detected, among which 76 (93.83%) were polymorphic, indicating high genetic diversity at the isolate level. Further, Nei's gene diversity (h) and Shannon's Information index (I) between the different isolates both revealed a remarkable genetic diversity, higher than previously indicated by their morphology. At the same time, substantial gene flow was found. So the main factors responsible for the high level of diversity within populations are probably due to conjugation (sexual reproduction) and wide distribution of swarmers. Analysis of molecular variance (AMOVA) showed that there was low genetic differentiation among the four populations probably due to common ancestry and flooding events. The cluster analysis and principal component analysis (PCA) suggested that genotypes isolated from the same lake displayed a higher genetic similarity than those from different lakes. Both analyses separated C. polypinum isolates into subgroups according to the geographical locations. However, there is only a weak positive correlation between the genetic distance and geographical distance, suggesting a minor effect of geographical distance on the distribution of genetic diversity between populations of C. polypinum at the local level. In conclusion, our studies clearly demonstrated that a single morphospecies may harbor high levels of genetic diversity, and that the degree of resolution offered by morphology as a marker for measuring distribution patterns of genetically distinct entities is too low.

A miniature insertion element transposable in Microcystis sp FACHB 854

A 186-bp sequence with imperfect terminal inverted repeats and target direct repeats but without any transposase-encoding capacity was found to be transposable in an isolate derived from Microcystis sp. FACHB 854. This miniature insertion element, designated as ISM854-1, and with its homologues present at least 10 copies in the genome of Microcystis FACHB 854, is inserted into the 8-bp long and AT-rich target sequences, but none or few in other Microcystis strains. A variant of ISM854-1, denoted ISM854-1A, has perfect inverted repeat sequences and may transpose in pairs in a structure like a composite transposon. This is the first report of non-autonomous transposition of a mini-IS in a cyanobacterium.

Two cathelicidin genes are present in both rainbow trout (Oncorhynchus mykiss) and Atlantic salmon (Salmo salar)

Further to the previous finding of the rainbow trout rtCATH_1 gene, this paper describes three more cathelicidin genes found in salmonids: two in Atlantic salmon, named asCATH_1 and asCATH_2, and one in rainbow trout, named rtCATH_2. All the three new salmonid cathelicidin genes share the common characteristics of mammalian cathelicidin genes, such as consisting of four exons and possessing a highly conserved preproregion and four invariant cysteines clustered in the C-terminal region of the cathelin-like domain. The asCATH_1 gene is homologous to the rainbow trout rtCATH_1 gene, in that it possesses three repeat motifs of TGGGGGTGGC in exon IV and two cysteine residues in the predicted mature peptide, while the asCATH_2 gene and rtCATH_2 gene are homologues of each other, with 96% nucleotide identity. Salmonid cathelicidins possess the same elastase-sensitive residue, threonine, as hagfish cathelicidins and the rabbit CAP18 molecule. The cleavage site of the four salmonid cathelicidins is within a conserved amino acid motif of QKIRTRR, which is at the beginning of the sequence encoded by exon W. Two 36-residue peptides corresponding to the core part of rtCATH_1 and rtCATH_2 were chemically synthesized and shown to exhibit potent antimicrobial activity. rtCATH_2 was expressed constitutively in gill, head kidney, intestine, skin and spleen, while the expression of rtCATH_1 was inducible in gill, head kidney, and spleen after bacterial challenge. Four cathelicidin genes have now been characterized in salmonids and two were identified in hagfish, confirming that cathelicidin genes evolved early and are likely present in all vertebrates.

Characterization of a highly conserved microsatellite marker with utility potentials in cyprinid fishes

A microsatellite locus, MFW1, originating from common carp is highly conserved in flanking nucleotides but variable in repeat length in some fishes from different families of the Cypriniformes. This orthologous locus is polymorphic in approximately 58% of the species tested in the order and is inherited by Mendelian law. It proved to be a potentially good marker in population genetics and in the cyprinid species-breeding programme in which no microsatellite markers were available.

Complex mutation at a microsatellite locus in sturgeons: Acipenser sinensis, A-schrenckii, A-gueldenstaedtii and A-baerii

Cross-species amplifications of microsatellite locus Spl-106, which was originally screened from the genome of shovelnose sturgeon (Scaphirhynchus platorynchus) with a perfect TAGA repeat motif, were carried out in four other species of the genera Acipenser. A total of 34 polymerase chain reaction (PCR) products representing 16 different alleles of this locus was sequenced. Sequence analysis results showed that besides the number changes of repeat units, many mutational events, such as single-base substitutions and various insertion/deletion (indels) occurred not only at species level but also at individual level, even among the different alleles within the same individual. The repeat motifs varied from perfect (TAGA)n array to perfect compound (TAAA)m (GAAA)n and perfect or imperfect compound (TAAA)m (TAGA)n (TAAA)x arrays in different species and different individuals. The evolution dynamics of this locus in sturgeons was inferred in that it may evolve from a single perfect to different perfect or imperfect compounds.