Gradient-based reinforcement learning techniques for underwater robotics behavior learning

Darrerament, l'interès pel desenvolupament d'aplicacions amb robots submarins autònoms (AUV) ha crescut de forma considerable. Els AUVs són atractius gràcies al seu tamany i el fet que no necessiten un operador humà per pilotar-los. Tot i això, és impossible comparar, en termes d'eficiència i flexibilitat, l'habilitat d'un pilot humà amb les escasses capacitats operatives que ofereixen els AUVs actuals. L'utilització de AUVs per cobrir grans àrees implica resoldre problemes complexos, especialment si es desitja que el nostre robot reaccioni en temps real a canvis sobtats en les condicions de treball. Per aquestes raons, el desenvolupament de sistemes de control autònom amb l'objectiu de millorar aquestes capacitats ha esdevingut una prioritat. Aquesta tesi tracta sobre el problema de la presa de decisions utilizant AUVs. El treball presentat es centra en l'estudi, disseny i aplicació de comportaments per a AUVs utilitzant tècniques d'aprenentatge per reforç (RL). La contribució principal d'aquesta tesi consisteix en l'aplicació de diverses tècniques de RL per tal de millorar l'autonomia dels robots submarins, amb l'objectiu final de demostrar la viabilitat d'aquests algoritmes per aprendre tasques submarines autònomes en temps real. En RL, el robot intenta maximitzar un reforç escalar obtingut com a conseqüència de la seva interacció amb l'entorn. L'objectiu és trobar una política òptima que relaciona tots els estats possibles amb les accions a executar per a cada estat que maximitzen la suma de reforços totals. Així, aquesta tesi investiga principalment dues tipologies d'algoritmes basats en RL: mètodes basats en funcions de valor (VF) i mètodes basats en el gradient (PG). Els resultats experimentals finals mostren el robot submarí Ictineu en una tasca autònoma real de seguiment de cables submarins. Per portar-la a terme, s'ha dissenyat un algoritme anomenat mètode d'Actor i Crític (AC), fruit de la fusió de mètodes VF amb tècniques de PG.

Attention and selection behavior on ‘universal search’ result pages

The ISO norm line 9241 states some criteria for ergonomics of human system interaction. In markets with a huge variety of offers and little possibility of differentiation, providers can gain a decisive competitive advantage by user oriented interfaces. A precondition for this is that relevant information can be obtained for entrepreneurial decisions in this regard. To test how users of universal search result pages use those pages and pay attention to different elements, an eye tracking experiment with a mixed design has been developed. Twenty subjects were confronted with search engine result pages (SERPs) and were instructed to make a decision while conditions “national vs. international city” and “with vs. without miniaturized Google map” were used. Different parameters like fixation count, duration and time to first fixation were computed from the eye tracking raw data and supplemented by click rate data as well as data from questionnaires. Results of this pilot study revealed some remarkable facts like a vampire effect on miniaturized Google maps. Furthermore, Google maps did not shorten the process of decision making, Google ads were not fixated, visual attention on SERPs was influenced by position of the elements on the SERP and by the users’ familiarity with the search target. These results support the theory of Amount of Invested Mental Effort (AIME) and give providers empirical evidence to take users’ expectations into account. Furthermore, the results indicated that the task oriented goal mode of participants was a moderator for the attention spent on ads. Most important, SERPs with images attracted the viewers’ attention much longer than those without images. This unique selling proposition may lead to a distortion of competition on markets.

Organizational effects of fetal testosterone on human corpus callosum size and asymmetry

Previous theory and research in animals has identified the critical role that fetal testosterone (FT) plays in organizing sexually dimorphic brain development. However, to date there are no studies in humans directly testing the organizational effects of FT on structural brain development. In the current study we investigated the effects of FT on corpus callosum size and asymmetry. High-resolution structural magnetic resonance images (MRI) of the brain were obtained on 28 8-11-year-old boys whose exposure to FT had been previously measured in utero via amniocentesis conducted during the second trimester. Although there was no relationship between FT and midsaggital corpus callosum size, increasing FT was significantly related to increasing rightward asymmetry (e.g., Right>Left) of a posterior subsection of the callosum, the isthmus, that projects mainly to parietal and superior temporal areas. This potential organizational effect of FT on rightward callosal asymmetry may be working through enhancing the neuroprotective effects of FT and result in an asymmetric distribution of callosal axons. We suggest that this possible organizational effect of FT on callosal asymmetry may also play a role in shaping sexual dimorphism in functional and structural brain development, cognition, and behavior.

Damage rate is a predictor of fatigue life and creep strain rate in fatigue of tensile human cortical bone samples

We present results on the growth of damage in 29 fatigue tests of human femoral cortical bone from four individuals, aged 53–79. In these tests we examine the interdependency of stress, cycles to failure, rate of creep strain, and rate of modulus loss. The behavior of creep rates has been reported recently for the same donors as an effect of stress and cycles (Cotton, J. R., Zioupos, P., Winwood, K., and Taylor, M., 2003, "Analysis of Creep Strain During Tensile Fatigue of Cortical Bone," J. Biomech. 36, pp. 943–949). In the present paper we first examine how the evolution of damage (drop in modulus per cycle) is associated with the stress level or the "normalized stress" level (stress divided by specimen modulus), and results show the rate of modulus loss fits better as a function of normalized stress. However, we find here that even better correlations can be established between either the cycles to failure or creep rates versus rates of damage than any of these three measures versus normalized stress. The data indicate that damage rates can be excellent predictors of fatigue life and creep strain rates in tensile fatigue of human cortical bone for use in practical problems and computer simulations.

Omega-3 polyunsaturated fatty acids and human health outcomes

Current intakes of very long chain omega-3 fatty acids, eicosapentaenoic acid (EPA), and docosahexaenoic acid (DNA) are low in most individuals living in Western countries. A good natural source of these fatty acids is seafood, especially oily fish. Fish oil capsules contain these fatty acids too. Very long chain w-3 fatty acids are readily incorporated from capsules into transport, functional, and storage pools. This incorporation is dose-dependent and follows a kinetic pattern that is characteristic for each pool. At sufficient levels of incorporation, EPA and DHA influence the physical nature of cell membranes and membrane protein-mediated responses, eicosanoid generation, cell signaling and gene expression in many different cell types. Through these mechanisms, EPA and DHA influence cell and tissue physiology, and the way cells and tissues respond to external signals. In most cases, the effects seen are compatible with improvements in disease biomarker profiles or in health-related outcomes. As a result, very long chain omega-3 fatty acids play a role in achieving optimal health and in protection against disease. Long chain omega-3 fatty acids protect against cardiovascular morbidity and mortality, and might be beneficial in rheumatoid arthritis, inflammatory bowel diseases, childhood learning, and behavior, and adult psychiatric and neurodegenerative illnesses. DHA has an important structural role in the eye and brain, and its supply early in life is known to be of vital importance. On the basis of the recognized health improvements brought about by long chain omega-3 fatty acids, recommendations have been made to increase their intake. (C) 2009 International Union of Biochemistry and Molecular Biology, Inc. Volume 35, Number 3, May/June 2009, Pages 266-272. E-mail: pcc@soton.ac.uk

the implication of social cognition for experimental economics From the issue entitled "Special issue on Experimental economics and the social embedding of economic behavior and cognition"

Can human social cognitive processes and social motives be grasped by the methods of experimental economics? Experimental studies of strategic cognition and social preferences contribute to our understanding of the social aspects of economic decisions making. Yet, papers in this issue argue that the social aspects of decision-making introduce several difficulties for interpreting the results of economic experiments. In particular, the laboratory is itself a social context, and in many respects a rather distinctive one, which raises questions of external validity.

Eight-plex iTRAQ analysis of variant metastatic human prostate cancer cells identifies candidate biomarkers of progression: an exploratory study.

BACKGROUND: Due to the heterogeneity in the biological behavior of prostate cancer, biomarkers that can reliably distinguish indolent from aggressive disease are urgently needed to inform treatment choices. METHODS: We employed 8-plex isobaric Tags for Relative and Absolute Quantitation (iTRAQ), to profile the proteomes of two distinct panels of isogenic prostate cancer cells with varying growth and metastatic potentials, in order to identify novel biomarkers associated with progression. The LNCaP, LNCaP-Pro5, and LNCaP-LN3 panel of cells represent a model of androgen-responsive prostate cancer, while the PC-3, PC-3M, and PC-3M-LN4 panel represent a model of androgen-insensitive disease. RESULTS: Of the 245 unique proteins identified and quantified (>or=95% confidence; >or=2 peptides/protein), 17 showed significant differential expression (>or=+/-1.5), in at least one of the variant LNCaP cells relative to parental cells. Similarly, comparisons within the PC-3 panel identified 45 proteins to show significant differential expression in at least one of the variant PC-3 cells compared with parental cells. Differential expression of selected candidates was verified by Western blotting or immunocytochemistry, and corresponding mRNA expression was determined by quantitative real-time PCR (qRT-PCR). Immunostaining of prostate tissue microarrays for ERp5, one of the candidates identified, showed a significant higher immunoexpression in pre-malignant lesions compared with non-malignant epithelium (P < 0.0001, Mann-Whitney U-test), and in high Gleason grade (4-5) versus low grade (2-3) cancers (P < 0.05). CONCLUSIONS: Our study provides proof of principle for the application of an 8-plex iTRAQ approach to uncover clinically relevant candidate biomarkers for prostate cancer progression.

Eliciting consumer preferences for certified animal-friendly foods: can elements of the theory of planned behavior improve choice experiment analysis?

Models used in neoclassical economics assume human behaviour to be purely rational. On the other hand, models adopted in social and behavioural psychology are founded on the ‘black box’ of human cognition. In view of these observations, this paper aims at bridging this gap by introducing psychological constructs in the well established microeconomic framework of choice behaviour based on random utility theory. In particular, it combines constructs developed employing Ajzen’s theory of planned behaviour with Lancaster’s theory of consumer demand for product characteristics to explain stated preferences over certified animal-friendly foods. To reach this objective a web survey was administered in the largest five EU-25 countries: France, Germany, Italy, Spain and the UK. Findings identify some salient cross-cultural differences between northern and southern Europe and suggest that psychological constructs developed using the Ajzen model are useful in explaining heterogeneity of preferences. Implications for policy makers and marketers involved with certified animal-friendly foods are discussed.

Fetal testosterone influences sexually dimorphic gray matter in the human brain

In nonhuman species, testosterone is known to have permanent organizing effects early in life that predict later expression of sex differences in brain and behavior. However, in humans, it is still unknown whether such mechanisms have organizing effects on neural sexual dimorphism. In human males, we show that variation in fetal testosterone (FT) predicts later local gray matter volume of specific brain regions in a direction that is congruent with sexual dimorphism observed in a large independent sample of age-matched males and females from the NIH Pediatric MRI Data Repository. Right temporoparietal junction/posterior superior temporal sulcus (RTPJ/pSTS), planum temporale/parietal operculum (PT/PO), and posterior lateral orbitofrontal cortex (plOFC) had local gray matter volume that was both sexually dimorphic and predicted in a congruent direction by FT. That is, gray matter volume in RTPJ/pSTS was greater for males compared to females and was positively predicted by FT. Conversely, gray matter volume in PT/PO and plOFC was greater in females compared to males and was negatively predicted by FT. Subregions of both amygdala and hypothalamus were also sexually dimorphic in the direction of Male > Female, but were not predicted by FT. However, FT positively predicted gray matter volume of a non-sexually dimorphic subregion of the amygdala. These results bridge a long-standing gap between human and nonhuman species by showing that FT acts as an organizing mechanism for the development of regional sexual dimorphism in the human brain.

Context relevant persuasive interaction and design: consideration of human factors influencing B2C persuasive interaction

The revolutions of electronic and mobile commerce have opened up unique opportunities for global marketing and business to consumer (B2C) persuasive interaction; with the aim of influencing user behavior. The success or failure of such persuasive interaction is impacted by cultural, social, and personal dimensions. Accordingly, failure to consider such dimensions risks the user forming a negative attitude towards retailer’s websites or mobile applications, which subsequently threatens customer behavior. A better understanding of human factors influencing the way people perceive B2C persuasive interaction is needed. This paper is introduces the concept of a context relevant persuasive interaction and proposes a guideline for designing relevant B2C persuasive interaction.

Intrastriatal transplantation of adult human neural crest-derived stem cells improves functional outcome in parkinsonian rats

Parkinson's disease (PD) is considered the second most frequent and one of the most severe neurodegenerative diseases, with dysfunctions of the motor system and with nonmotor symptoms such as depression and dementia. Compensation for the progressive loss of dopaminergic (DA) neurons during PD using current pharmacological treatment strategies is limited and remains challenging. Pluripotent stem cell-based regenerative medicine may offer a promising therapeutic alternative, although the medical application of human embryonic tissue and pluripotent stem cells is still a matter of ethical and practical debate. Addressing these challenges, the present study investigated the potential of adult human neural crest-derived stem cells derived from the inferior turbinate (ITSCs) transplanted into a parkinsonian rat model. Emphasizing their capability to give rise to nervous tissue, ITSCs isolated from the adult human nose efficiently differentiated into functional mature neurons in vitro. Additional successful dopaminergic differentiation of ITSCs was subsequently followed by their transplantation into a unilaterally lesioned 6-hydroxydopamine rat PD model. Transplantation of predifferentiated or undifferentiated ITSCs led to robust restoration of rotational behavior, accompanied by significant recovery of DA neurons within the substantia nigra. ITSCs were further shown to migrate extensively in loose streams primarily toward the posterior direction as far as to the midbrain region, at which point they were able to differentiate into DA neurons within the locus ceruleus. We demonstrate, for the first time, that adult human ITSCs are capable of functionally recovering a PD rat model.

Female-specific intergenerational transmission patterns of the human corticolimbic circuitry

Parents have large genetic and environmental influences on offspring’s cognition, behavior, and brain. These intergenerational effects are observed in mood disorders, with particularly robust association in depression between mothers and daughters. No studies have thus far examined the neural bases of these intergenerational effects in humans. Corticolimbic circuitry is known to be highly relevant in a wide range of processes including mood regulation and depression. These findings suggest that corticolimbic circuitry may also show matrilineal transmission patterns. We therefore examined human parent-offspring association in this neurocircuitry, and investigated the degree of association in gray matter volume between parent and offspring. We used voxel-wise correlation analysis in a total of 35 healthy families, consisting of parents and their biological offspring. We found positive associations of regional grey matter volume in the corticolimbic circuit including the amygdala, hippocampus, anterior cingulate cortex, and ventromedial prefrontal cortex between biological mothers and daughters. This association was significantly greater than mother-son, father-daughter, and father-son associations. The current study suggests that the corticolimbic circuitry, which has been implicated in mood regulation, shows a matrilineal specific transmission patterns. Our preliminary findings are consistent with what has been found behaviorally in depression, and may have clinical implications for disorders known to have dysfunction in mood regulation such as depression. Studies such as ours will likely bridge animal work examining gene expression in the brains and clinical symptom-based observations, and provide promising ways to investigate intergenerational transmission patterns in the human brain.

Human-animal interaction, stress, and embryo production in Bos indicus embryo donors under tropical conditions

This study investigated the effect of human-animal interaction (HAI) and the stress response on the quality of embryo production in superovulated Nelore (Bos indicus) cattle, under tropical conditions. Thirty-two females underwent a superovulation protocol for 5 days. Cortisol concentrations were determined in blood plasma collected on days 0, 4, and 5. Artificial insemination was performed on days 4 and 5, and nonsurgical embryo flushing on day 11. Embryo production and viability were determined. Human stimulation, animal behaviors, accidents, and handling time were recorded to assess HAI. Cattle age was negatively correlated with accidents, frequency of aversive behaviors, and negative stimuli by stockperson during transit through corral compartments to receive superovulation treatments. The factor analysis revealed two distinct groups. The first group was called stressed and had higher cortisol concentration than the nonstressed group, 16.0 +/- 2.1 and 12.5 +/- 1.0 ng/mL, respectively. Comparisons between these groups showed that the frequency of voice emissions by the stockperson and the number of accidents were higher in the stressed group, and also, the mean handling time was longer in the stressed group than for the nonstressed. As a result, viability rate of the embryos was 19% lower in the stressed group (P < 0.05). This indicates that intensive negative HAI is likely related to stress, which affects embryo production in a superovulation program.

Functional Analysis of saxophone, the Drosophila Gene Encoding the BMP Type I Receptor Ortholog of Human ALK1/ACVRL1 and ACVR1/ALK2

In metazoans, bone morphogenetic proteins (BMPS) direct a myriad of developmental and adult homeostatic evens through their heterotetrameric type I and type II receptor complexes. We examined 3 existing and 12 newly generated mutations in the Drosophila type I receptor gene, saxophone (sax), the ortholog of the human Activin Receptor-Like. Kinasel and -2 (ALK1/ACVR1 and ALK2/ACVR1) genes. Our genetic analyses identified two distinct classes of sax alleles. The first class consists of homozygous viable gain-of-function (GOF) alleles that exhibit (1) synthetic lethality in combination with mutations in BMP pathway components, and (2) significant maternal effect lethality that can be rescued by an increased dosage of the BMP encoding gene, dpp(+). In contrast, the second class consists of alleles that are recessive lethal and do not exhibit lethality in combination with mutations in other BMP pathway components. The alleles in this second class are clearly loss-of-function (LOF) with both complete and partial loss-of-function mutations represented. We find that one allele in the second class of recessive lethals exhibits dominant-negative behavior, albeit distinct from the GOF activity of the first class of viable alleles. On the basis of the fact that the first class of viable alleles can be reverted to lethality and on our ability to independently generate recessive lethal sat mutations, our analysis demonstrates that sax is an essential gene. Consistent with this conclusion, we find that a normal sax transcript is produced by sax(P), a viable allele previously reported to be mill, and that this allele can be reverted to lethality. Interestingly, we determine that two mutations in the first: class of sax alleles show the same amino acid substitutions as mutations in the human receptors ALK1/ACVR1-1 and ACVR1/ALK2, responsible for cases of hereditary hemorrhagic telangiectasia type 2 (HHT2) and fibrodysplasia ossificans progressiva (FOP), respectively. Finally, the data presented here identify different functional requirements for the Sax receptor, support the proposal that Sax participates in a heteromeric receptor complex, and provide a mechanistic framework for future investigations into disease states that arise from defects in BMP/TGF-beta signaling.

Proteome analysis of human dorsolateral prefrontal cortex using shotgun mass spectrometry

The prefrontal cortex executes important functions such as differentiation of conflicting thoughts, correct social behavior and personality expression, and is directly implicated in different neurodegenerative diseases. We performed a shotgun proteome analysis that included IEF fractionation, RP-LC, and MALDI-TOF/TOF mass spectrometric analysis of tryptic digests from a pool of seven human dorsolateral prefrontal cortex protein extracts. In this report, we present a catalog of 387 proteins expressed in these samples, identified by two or more peptides and high confidence search scores. These proteins are involved in different biological processes such as cell growth and/or maintenance, metabolism/energy pathways, cell communication/signal trarisduction, protein metabolism, transport, regulation of nucleobase, nucleoside, nucleotide and nucleic acid metabolism, and immune response. This analysis contributes to the knowledge of the human brain proteome by adding sample diversity and protein expression data from an alternative technical approach. It will also aid comparative studies of different brain areas and medical conditions, with future applications in basic and clinical research.