Reinforcement learning using a continuous time actor-critic framework with spiking neurons.

Animals repeat rewarded behaviors, but the physiological basis of reward-based learning has only been partially elucidated. On one hand, experimental evidence shows that the neuromodulator dopamine carries information about rewards and affects synaptic plasticity. On the other hand, the theory of reinforcement learning provides a framework for reward-based learning. Recent models of reward-modulated spike-timing-dependent plasticity have made first steps towards bridging the gap between the two approaches, but faced two problems. First, reinforcement learning is typically formulated in a discrete framework, ill-adapted to the description of natural situations. Second, biologically plausible models of reward-modulated spike-timing-dependent plasticity require precise calculation of the reward prediction error, yet it remains to be shown how this can be computed by neurons. Here we propose a solution to these problems by extending the continuous temporal difference (TD) learning of Doya (2000) to the case of spiking neurons in an actor-critic network operating in continuous time, and with continuous state and action representations. In our model, the critic learns to predict expected future rewards in real time. Its activity, together with actual rewards, conditions the delivery of a neuromodulatory TD signal to itself and to the actor, which is responsible for action choice. In simulations, we show that such an architecture can solve a Morris water-maze-like navigation task, in a number of trials consistent with reported animal performance. We also use our model to solve the acrobot and the cartpole problems, two complex motor control tasks. Our model provides a plausible way of computing reward prediction error in the brain. Moreover, the analytically derived learning rule is consistent with experimental evidence for dopamine-modulated spike-timing-dependent plasticity.

A PDE viewpoint on basic properties of coordination algorithms with symmetries

Several recent control applications consider the coordination of subsystems through local interaction. Often the interaction has a symmetry in state space, e.g. invariance with respect to a uniform translation of all subsystem values. The present paper shows that in presence of such symmetry, fundamental properties can be highlighted by viewing the distributed system as the discrete approximation of a partial differential equation. An important fact is that the symmetry on the state space differs from the popular spatial invariance property, which is not necessary for the present results. The relevance of the viewpoint is illustrated on two examples: (i) ill-conditioning of interaction matrices in coordination/consensus problems and (ii) the string instability issue. ©2009 IEEE.

Differential geometry and design coupling in MDO

The notion of coupling within a design, particularly within the context of Multidisciplinary Design Optimization (MDO), is much used but ill-defined. There are many different ways of measuring design coupling, but these measures vary in both their conceptions of what design coupling is and how such coupling may be calculated. Within the differential geometry framework which we have previously developed for MDO systems, we put forth our own design coupling metric for consideration. Our metric is not commensurate with similar types of coupling metrics, but we show that it both provides a helpful geo- metric interpretation of coupling (and uncoupledness in particular) and exhibits greater generality and potential for analysis than those similar metrics. Furthermore, we discuss how the metric might be profitably extended to time-varying problems and show how the metric's measure of coupling can be applied to multi-objective optimization problems (in unconstrained optimization and in MDO). © 2013 by the American Institute of Aeronautics and Astronautics, Inc. All rights reserved.

Effects of dietary ascorbic acid supplementation on the growth performance, immune and stress response in juvenile Leiocassis longirostris Gunther exposed to ammonia

This experiment was designed to investigate the effect of dietary supplemental ascorbic acid (AA) on the feed intake, growth, serum lysozyme, hepatic superoxide dismutase (SOD) and handling stress response in Chinese longsnout catfish (Leiocassis longirostris Gunther) exposed to three levels of unionized ammonia nitrogen (UIA-N). Juvenile Chinese longsnout catfish were reared in 54 fibreglass tanks with a 3 x 3 factorial design treatment consisting of three supplemental AA levels in ascorbyl 2-monophosphate (38, 364 and 630 mg AA equivalent kg(-1) diet) and three UIA-N concentrations [0.004 (the control), 0.037 and 0.292 mg L-1]. The fish were sampled on the 11th, 32nd and 60th day. On the 62nd day, the remaining fish were subjected to an acute stress by being held in a dipnet out of water for 60 s, and sampled at 30 min post handling. The results showed that the specific growth rate (SGR) in 32 days significantly decreased with increased water UIA-N (P=0.0476) but was not affected by dietary supplemental AA (P > 0.05). After 60 days, SGR, feeding rate (FR) and feed conversion efficiency (FCE) significantly increased with increased dietary supplemental AA (P < 0.001) while remaining unaffected by water UIA-N (P > 0.05). There was no significant interaction between dietary AA and UIA-N for growth responses (P > 0.05). The serum lysozyme activity on the 11th day and the hepatic SOD activity on the 32nd day were significantly affected at high (0.292 mg L-1) water UIA-N. On the 62nd day, the increase in cortisol resulting from acute stress significantly decreased by higher UIA-N (P=0.038). It is suggested that Chinese longsnout catfish displayed an adaptive response after long-term UIA-N exposure, and AA had beneficial effects on the growth and feed intake of catfish and alleviated the negative effects of chronic ammonia stress. A chronically higher ammonia level shows a tendency to inhibit the cortisol response to another acute stressor.

mHealth and global mental health: still waiting for the mH2 wedding?

BACKGROUND: Two phenomena have become increasingly visible over the past decade: the significant global burden of disease arising from mental illness and the rapid acceleration of mobile phone usage in poorer countries. Mental ill-health accounts for a significant proportion of global disability-adjusted life years (DALYs) and years lived with disability (YLDs), especially in poorer countries where a number of factors combine to exacerbate issues of undertreatment. Yet poorer countries have also witnessed significant investments in, and dramatic expansions of, mobile coverage and usage over the past decade. DEBATE: The conjunction of high levels of mental illness and high levels of mobile phone usage in poorer countries highlights the potential for "mH(2)" interventions--i.e. mHealth (mobile technology-based) mental health interventions--to tackle global mental health challenges. However, global mental health movements and initiatives have yet to engage fully with this potential, partly because of scepticism towards technological solutions in general and partly because existing mH(2) projects in mental health have often taken place in a fragmented, narrowly-focused, and small-scale manner. We argue for a deeper and more sustained engagement with mobile phone technology in the global mental health context, and outline the possible shape of an integrated mH(2) platform for the diagnosis, treatment, and monitoring of mental health. SUMMARY: Existing and developing mH(2) technologies represent an underutilised resource in global mental health. If development, evaluation, and implementation challenges are overcome, an integrated mH2 platform would make significant contributions to mental healthcare in multiple settings and contexts.

Ex situ conservation status of an endangered Yangtze finless porpoise population (Neophocaena phocaenoides asiaeorientalis) as measured from microsatellites and mtDNA diversity

The Yangtze finless porpoise, Neophocaena phocaenoides asiaeorientalis, is all endangered small cetacean that occurs only in the middle and lower reaches of the Yangtze River of China. The establishment of a breeding population of the porpoise in Tian-e-Zhou Baiji National Natural Reserve represents the first attempt at ex situ conservation efforts for a cetacean species. With the goal of effective protection, management, and monitoring of this preserved population, we examined its genetic diversity using 930 bp of mtDNA control region sequences and 13 polymorphic microsatellite loci. A very low level of genetic variation (h = 0.6010 +/- 0.0029 s.d.; pi = 0.0007 +/- 0.0000002 s.d.) in the mtDNA control region sequences and a moderate genetic diversity (Ho = 0.5740 +/- 0.2575 s.d.) in the microsatellites were detected in the population. It is necessary to introduce more individuals with representative genetic variations into the reserve ill order to foml a larger and healthier group structure for long-term survival of the population. Successful establishment of the Yangtze finless porpoise population in the Reserve also provides a useful model for an ex situ conservation programme for other rare and endangered species. (c) 2005 International Council for the Exploration ofthe Sea. Published by Elsevier Ltd. All rights reserved.

Species richness in a riparian plant community along the banks of the Xiangxi River, the Three Gorges region

The distribution of vascular plant species richness along an altitudinal gradient and their relationships with environmental variables, including slope, aspect, bank (flooding) height, and river width of the Xiangxi River, Hubei Province, were examined. Total vascular plant species richness changed with elevation: it increased at lower elevations, reached a maximum in the midreaches and decreased thereafter. In particular, tree and herbaceous species richness were related to altitude. Correlation analysis (Kendall's tau) between species richness and environmental variables indicated that the change in species richness in the riparian zone was determined by riparian environmental factors and characteristics of regional vegetation distribution along the altitudinal gradient. The low species richness at lower elevations resulted from seasonal flooding and human activities - agriculture and fuel collection - and the higher. Species richness ill (he midreaches reflected transitional zones ill natural vegetation types that had had little disturbance. These results oil species distribution in the riparian community could he utilized as a reference for restoration efforts to improve water quality of the emerging reservoir resulting from the Three Gorges Hydroelectric Dam project.

Photoluminescence properties of tensile-strained GaAsP/GaInP single quantum wells grown by metal organic chemical vapor deposition

Tensile-strained GaAsP/GaInP single quantum well (QW) laser diode (I-D) structures have been grown by low-pressure metal organic chemical vapor deposition (LP-MOCVD) and related photoluminescence (PL) properties have been investigated in detail. The samples have the same well thickness of 16 nm but different P compositions in a GaAsP QW. Two peaks in room temperature (RT) PL spectra are observed for samples with a composition larger than 0.10. Temperature and excitation-power-dependent PL spectra have been measured for a sample with it P composition of 0.15. It is found that the two peaks have a 35 meV energy separation independent of temperature and only the low-energy peak exists below 85 K. Additionally, both peak intensities exhibit a monotonous increase as excitation power increases. Analyses indicate that the two peaks arise from the intrinsic-exciton recombination mechanisms of electron-heavy hole (e-hh) and electron-light hole (e-hh). A theoretical calculation based oil model-solid theory, taking, into account the spin-orbit splitting energy, shows good agreement with our experimental results. The temperature dependence of PL intensity ratio is well explained using the spontaneous emission theory for e-hh and e-hh transitions. front which the ratio can be characterized mainly by the energy separation between the fill and Ill states.

Comparison of the properties of GaN grown on complex Si-based structures

With the aim of investigating the possible integration of optoelectronic devices, epitaxial GaN layers have been grown on Si(Ill) semiconductor-on-insulator (SOI) and on Si/CoSi2/Si(111) using metalorganic chemical vapor deposition. The samples are found to possess a highly oriented wurtzite structure, a uniform thickness, and abrupt interfaces. The epitaxial orientation is determined as GaN(0001)//Si(111), GaN[1120]//Si[110], and GaN[1010]//Si[112], and the GaN layer is tensilely strained in the direction parallel to the interface. According to Rutherford backscattering/channeling spectrometry and (0002) rocking curves, the crystalline quality of GaN on Si(111) SOI is better than that of GaN on silicide. Room-temperature photoluminescence of GaN/SOI reveals a strong near-band-edge emission at 368 nm (3.37 eV) with a full width at half-maximum of 59 meV. (c) 2005 American Institute of Physics.

Hydrogen behavior in GaN epilayers grown by NH3-MBE

Hydrogen behavior in unintentionally doped GaN epilayers on sapphire substrates grown by NH3-MBE is investigated. Firstly, we find by using nuclear reaction analysis (NRA) that with increasing hydrogen concentration the background electron concentration increases, which suggests that there exists a hydrogen-related donor in undoped GaN, Secondly, Fourier transform infrared (FTIR) absorption and X-ray photoelectron spectroscopy (XPS) reveal Further that hydrogen atom is bound to nitrogen atom in GaN with a local vibrational mode at about 3211 cm(-1) Hence, it is presumed that the hydrogen-related complex Ga. . .H-N is a hydrogen-related donor candidate partly responsible for high n-type background commonly observed in GaN films. Finally, Raman spectroscopy results of the epilayers show that ill addition to the expected compressive biaxial strain, in some cases GaN films suffer from serious tensile biaxial strain. This anomalous behavior has been well interpreted in terms of interstitial hydrogen lattice dilation. (C) 2001 Elsevier Science B.V. All rights reserved.

A multi-frame image super-resolution method

Multi-frame image super-resolution (SR) aims to utilize information from a set of low-resolution (LR) images to compose a high-resolution (HR) one. As it is desirable or essential in many real applications, recent years have witnessed the growing interest in the problem of multi-frame SR reconstruction. This set of algorithms commonly utilizes a linear observation model to construct the relationship between the recorded LR images to the unknown reconstructed HR image estimates. Recently, regularization-based schemes have been demonstrated to be effective because SR reconstruction is actually an ill-posed problem. Working within this promising framework, this paper first proposes two new regularization items, termed as locally adaptive bilateral total variation and consistency of gradients, to keep edges and flat regions, which are implicitly described in LR images, sharp and smooth, respectively. Thereafter, the combination of the proposed regularization items is superior to existing regularization items because it considers both edges and flat regions while existing ones consider only edges. Thorough experimental results show the effectiveness of the new algorithm for SR reconstruction. (C) 2009 Elsevier B.V. All rights reserved.

天花粉蛋白诱导HIV-1 慢性感染细胞凋亡及机制研究

天花粉蛋白（Trichosanthin,TCS）是一种由247 个氨基酸组成的Ⅰ型核糖 体失活蛋白（Ribosome Inactivating Protein，RIP），从葫芦科植物栝楼 (Trichosanthes Kirilowii)球根中提纯获得。它具有广谱的生物学和药学活性， 包括抗肿瘤、免疫抑制、中期引产以及抗病毒活性。上世纪八十年代末， McGrath 等发现TCS 可以抑制HIV-1 在急性感染的T 淋巴细胞和慢性感染的 巨噬细胞中的复制，从而引起了研究者们极大的兴趣。但迄今为止，其抗HIV-1 的作用机制仍不清楚。 我们实验室对TCS 的免疫毒理作用和抗HIV-1 作用进行了多年的研究， 前期工作显示TCS 对HIV-1 的直接作用并不明显，但对于HIV-1 感染细胞却 具有很强的毒性作用。提示TCS 可能通过作用于宿主细胞来发挥其抗HIV-1 活性。在此基础上，我们从细胞方面着手，对TCS 选择性杀伤HIV-1 感染细 胞的作用及机制进行了探讨。首先，通过MTT 法检测发现，相同条件下，TCS 对于H9/HIV-1IIIB 细胞的毒性远远大于其对正常H9 细胞的毒性。其次，流式 细胞仪检测亚二倍体小峰和琼脂糖凝胶电泳检测片断化DNA 的实验证实了 TCS 对H9/HIV-1IIIB 的细胞杀伤作用是通过诱导细胞凋亡实现的。流式细胞仪 的结果显示TCS 以剂量依赖的方式诱导较多的H9/HIV-1IIIB 细胞凋亡，25μ g/mlTCS 作用24h 时，有8.4%的H9 细胞凋亡，而H9/HIV-1IIIB 细胞的凋亡率 则达到24.5%；随着作用浓度的降低，这种差异也在缩小。阳性对照D-Sorbitol 对两种细胞的凋亡率没有明显差别，约为25%。琼脂糖凝胶电泳的结果进一 步证实了这种推测，相同条件下，TCS 诱导H9/HIV-1IIIB 细胞出现更多的DNA 片断化。 TCS 可以选择性的诱导H9/HIV-1IIIB 细胞凋亡，为了进行更深入的机制研 究，我们建立了另外一株HIV-1 慢性感染细胞系，HIV-1 慢性感染的Jurkat 细胞系（Jurkat/HIV-1ⅢB）来验证TCS的作用。发现相同条件下，TCS可以诱 导同等程度的Jurkat/HIV-1ⅢB和Jurkat 细胞凋亡，25μg/mlTCS作用24h 时， 分别有21.08%的Jurkat 细胞和27.27%的Jurkat/HIV-1IIIB 细胞凋亡。以上的结 果说明HIV-1 感染H9 细胞后增强了感染细胞对TCS 的敏感性，而HIV-1 感 染Jurkat 细胞后并不影响其对TCS 的敏感性。根据细胞凋亡途径中是否依赖 线粒体的参与可以将细胞分成TypeⅠ和TypeⅡ两种类型，H9 细胞采取的是 线粒体非依赖性的TypeⅠ型凋亡途径，而Jurkat 细胞则采取线粒体依赖性的 TypeⅡ型凋亡途径。由于Jurkat 细胞对TCS 诱导的凋亡更加敏感，我们推测 HIV-1 感染H9 细胞后，诱导了细胞凋亡途径由TypeⅠ向TypeⅡ型转变。为 此，我们采用流式细胞仪检测了TCS 对凋亡细胞内的线粒体膜电位及 caspase-8 蛋白酶活性的影响，结果显示，H9/HIV-1IIIB、Jurkat 和Jurkat/HIV-1IIIB 细胞对TCS 诱导的凋亡具有相同程度的敏感性，并且伴随着细胞线粒体膜电 位的耗散和caspase-8 蛋白酶的活化；而相同情况下，TCS 对H9 细胞的影响 则很微弱。由此，进一步证实了我们的推测，即HIV-1 感染H9 细胞后，通过 改变细胞内的某些信号，使H9/HIV-1ⅢB细胞的性状更加倾向于Type Ⅱ型细 胞，从而增强其对于TCS 的敏感性.

Hydrogen behavior in GaN epilayers grown by NH3-MBE

Hydrogen behavior in unintentionally doped GaN epilayers on sapphire substrates grown by NH3-MBE is investigated. Firstly, we find by using nuclear reaction analysis (NRA) that with increasing hydrogen concentration the background electron concentration increases, which suggests that there exists a hydrogen-related donor in undoped GaN, Secondly, Fourier transform infrared (FTIR) absorption and X-ray photoelectron spectroscopy (XPS) reveal Further that hydrogen atom is bound to nitrogen atom in GaN with a local vibrational mode at about 3211 cm(-1) Hence, it is presumed that the hydrogen-related complex Ga. . .H-N is a hydrogen-related donor candidate partly responsible for high n-type background commonly observed in GaN films. Finally, Raman spectroscopy results of the epilayers show that ill addition to the expected compressive biaxial strain, in some cases GaN films suffer from serious tensile biaxial strain. This anomalous behavior has been well interpreted in terms of interstitial hydrogen lattice dilation. (C) 2001 Elsevier Science B.V. All rights reserved.

Modeling the effect of ozone loss on photobleaching of chromophoric dissolved organic matter in the St. Lawrence estuary

Temporal trends in total ozone for the St. Lawrence estuary were estimated from ground-based measurements at the NOAA/CMDL station in Caribou, Maine. Linear regression analysis showed that from 1979 to 1999 total ozone has decreased by about 3.3% per decade on an annual basis and ≤6.2% per decade on a monthly basis relative to unperturbed (pre-CFC) levels. The influence of increased ultraviolet-B (280–320 nm) radiation associated with ozone depletion on water column photochemical processes was evaluated by modeling the photobleaching of chromophoric dissolved organic material (CDOM). Linear regression analysis showed small (<0.5% per decade), but statistically significant upward trends in maximum noontime photobleaching rates. Most notably, positive trends in relative rates for May, June, and July, when maximum absolute rates are expected, were predicted. A global model based on TOMS ozone data revealed increases in photobleaching of ≤3% per decade at high latitudes in the Southern Hemisphere. Radiation amplification factors for increases in photochemically weighted UV (280–400 nm) in response to ozone depletion were estimated at 0.1 and 0.08 for photobleaching of CDOM absorbance at 300 and 350 nm, respectively. Application of the laboratory-based model to conditions that more closely resembled those in situ were variable with both overestimation and underestimation of measured rates. The differences between modeled rates and observed rates under quasi-natural conditions were as large or larger than the predicted increases due to ozone depletion. These comparisons suggest that biological activity and mixing play an important, but as yet ill-defined, role in modifying photochemical processes.

Image reconstruction by an alternating minimisation

This paper focuses on the problem of incomplete data in the applications of the circular cone-beam computed tomography. This problem is frequently encountered in medical imaging sciences and some other industrial imaging systems. For example, it is crucial when the high density region of objects can only be penetrated by X-rays in a limited angular range. As the projection data are only available in an angular range, the above mentioned incomplete data problem can be attributed to the limited angle problem, which is an ill-posed inverse problem. This paper reports a modified total variation minimisation method to reduce the data insufficiency in tomographic imaging. This proposed method is robust and efficient in the task of reconstruction by showing the convergence of the alternating minimisation method. The results demonstrate that this new reconstruction method brings reasonable performance. (C) 2010 Elsevier B.V. All rights reserved.