Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Adaptable Strut-and-Tie Model for Design and Verification of Four-Pile Caps

A large percentage of pile caps support only one column, and the pile caps in turn are supported by only a few piles. These are typically short and deep members with overall span-depth ratios of less than 1.5. Codes of practice do not provide uniform treatment for the design of these types of pile caps. These members have traditionally been designed as beams spanning between piles with the depth selected to avoid shear failures and the amount of longitudinal reinforcement selected to provide sufficient flexural capacity as calculated by the engineering beam theory. More recently, the strut-and-tie method has been used for the design of pile caps (disturbed or D-region) in which the load path is envisaged to be a three-dimensional truss, with compressive forces being supported by concrete compressive struts between the column and piles and tensile forces being carried by reinforcing steel located between piles. Both of these models have not provided uniform factors of safety against failure or been able to predict whether failure will occur by flexure (ductile mode) or shear (fragile mode). In this paper, an analytical model based on the strut-and-tie approach is presented. The proposed model has been calibrated using an extensive experimental database of pile caps subjected to compression and evaluated analytically for more complex loading conditions. It has been proven to be applicable across a broad range of test data and can predict the failures modes, cracking, yielding, and failure loads of four-pile caps with reasonable accuracy.

Role of price and enforcement in water allocation: Insights from Game Theory

As many countries are moving toward water sector reforms, practical issues of how water management institutions can better effect allocation, regulation, and enforcement of water rights have emerged. The problem of nonavailability of water to tailenders on an irrigation system in developing countries, due to unlicensed upstream diversions is well documented. The reliability of access or equivalently the uncertainty associated with water availability at their diversion point becomes a parameter that is likely to influence the application by users for water licenses, as well as their willingness to pay for licensed use. The ability of a water agency to reduce this uncertainty through effective water rights enforcement is related to the fiscal ability of the agency to monitor and enforce licensed use. In this paper, this interplay across the users and the agency is explored, considering the hydraulic structure or sequence of water use and parameters that define the users and the agency`s economics. The potential for free rider behavior by the users, as well as their proposals for licensed use are derived conditional on this setting. The analyses presented are developed in the framework of the theory of ""Law and Economics,`` with user interactions modeled as a game theoretic enterprise. The state of Ceara, Brazil, is used loosely as an example setting, with parameter values for the experiments indexed to be approximately those relevant for current decisions. The potential for using the ideas in participatory decision making is discussed. This paper is an initial attempt to develop a conceptual framework for analyzing such situations but with a focus on the reservoir-canal system water rights enforcement.

Galerkin finite element method for incompressible thermofluid flows framed within the bond graph theory

In this paper a bond graph methodology is used to model incompressible fluid flows with viscous and thermal effects. The distinctive characteristic of these flows is the role of pressure, which does not behave as a state variable but as a function that must act in such a way that the resulting velocity field has divergence zero. Velocity and entropy per unit volume are used as independent variables for a single-phase, single-component flow. Time-dependent nodal values and interpolation functions are introduced to represent the flow field, from which nodal vectors of velocity and entropy are defined as state variables. The system for momentum and continuity equations is coincident with the one obtained by using the Galerkin method for the weak formulation of the problem in finite elements. The integral incompressibility constraint is derived based on the integral conservation of mechanical energy. The weak formulation for thermal energy equation is modeled with true bond graph elements in terms of nodal vectors of temperature and entropy rates, resulting a Petrov-Galerkin method. The resulting bond graph shows the coupling between mechanical and thermal energy domains through the viscous dissipation term. All kind of boundary conditions are handled consistently and can be represented as generalized effort or flow sources. A procedure for causality assignment is derived for the resulting graph, satisfying the Second principle of Thermodynamics. (C) 2007 Elsevier B.V. All rights reserved.

Fast Transforms for Acoustic Imaging-Part I: Theory

The classical approach for acoustic imaging consists of beamforming, and produces the source distribution of interest convolved with the array point spread function. This convolution smears the image of interest, significantly reducing its effective resolution. Deconvolution methods have been proposed to enhance acoustic images and have produced significant improvements. Other proposals involve covariance fitting techniques, which avoid deconvolution altogether. However, in their traditional presentation, these enhanced reconstruction methods have very high computational costs, mostly because they have no means of efficiently transforming back and forth between a hypothetical image and the measured data. In this paper, we propose the Kronecker Array Transform ( KAT), a fast separable transform for array imaging applications. Under the assumption of a separable array, it enables the acceleration of imaging techniques by several orders of magnitude with respect to the fastest previously available methods, and enables the use of state-of-the-art regularized least-squares solvers. Using the KAT, one can reconstruct images with higher resolutions than was previously possible and use more accurate reconstruction techniques, opening new and exciting possibilities for acoustic imaging.

On Building a Memory Evolutive System for Application to Learning and Cognition Modeling

We address here aspects of the implementation of a memory evolutive system (MES), based on the model proposed by A. Ehresmann and J. Vanbremeersch (2007), by means of a simulated network of spiking neurons with time dependent plasticity. We point out the advantages and challenges of applying category theory for the representation of cognition, by using the MES architecture. Then we discuss the issues concerning the minimum requirements that an artificial neural network (ANN) should fulfill in order that it would be capable of expressing the categories and mappings between them, underlying the MES. We conclude that a pulsed ANN based on Izhikevich`s formal neuron with STDP (spike time-dependent plasticity) has sufficient dynamical properties to achieve these requirements, provided it can cope with the topological requirements. Finally, we present some perspectives of future research concerning the proposed ANN topology.

Ethylene action blockade and cold storage affect ripening of `Golden` papaya fruit

The purpose of this work was to evaluate the effects of ethylene action blockade and cold storage on the ripening of `Golden` papaya fruit. Papayas harvested at maturity stage 1 (up to 15% yellow skin) were evaluated. Half of the fruits, whether treated or not treated with 100 nL L(-1) of 1-methylcyclopropene (1-MCP), were stored at 23A degrees C, while the other half were stored at 11A degrees C for 20 days prior to being stored at 23A degrees C. Non-refrigerated fruits receiving 1-MCP application presented a reduction in respiratory activity, ethylene production, skin color development and pectinmethylesterase activity. Even with a gradual increase in ethylene production at 23A degrees C, fruits treated with 1-MCP maintained a high firmness, but presented a loss of green skin color. Cold storage caused a decrease in ethylene production when fruits were transferred to 23A degrees C. The results suggest that pulp softening is more dependent on ethylene than skin color development, and that some processes responsible for loss of firmness do not depend on ethylene.

A Strong Protective Action of Guttiferone-A, a Naturally Occurring Prenylated Benzophenone, Against Iron-Induced Neuronal Cell Damage

Guttiferone-A (GA) is a natural occurring polyisoprenylated benzophenone with several reported pharmacological actions. We have assessed the protective action of GA on iron-induced neuronal cell damage by employing the PC12 cell line and primary culture of rat cortical neurons (PCRCN). A strong protection by GA, assessed by the 2,3-bis(2-methoxy-4-nitro-5-sulfophenyl)-2H-tetrazolium-5-carbox-anilide (XTT) assay, was revealed, with IC(50) values <1 mu M. GA also inhibited Fe(3+)-ascorbate reduction, iron-induced oxidative degradation of 2-deoxiribose, and iron-induced lipid peroxidation in rat brain homogenate, as well as stimulated oxygen consumption by Fe(2+) autoxidation. Absorption spectra and cyclic voltammograms of GA Fe(2+)/Fe(3+) complexes suggest the formation of a transient charge transfer complex between Fe(2+) and GA, accelerating Fe(2+) oxidation. The more stable Fe(3+) complex with GA would be unable to participate in Fenton-Haber Weiss-type reactions and the propagation phase of lipid peroxidation. The results show a potential of GA against neuronal diseases associated with iron-induced oxidative stress.

Mechanisms underlying the vasorelaxant action of the pimarane ent-8(14),15-pimaradien-3 beta-ol in the isolated rat aorta

Pimarane-type diterpenes were described to exert antispasmodic and relaxant activities. Based on this observation we hypothesized that the diterpene ent-8(14),15-pimaradien-3 beta-ol (PA-3 beta-ol) induced vascular relaxation. With this purpose, the present work investigates the mechanisms involved in the vasorelaxant effect of the pimarane-type diterpene PA-3 beta-ol. Vascular reactivity experiments, using standard muscle bath procedures, were performed in isolated aortic rings from male Wistar rats. Cytosolic calcium concentration ([Ca(2+)]c) was measured by confocal microscopy using the fluorescent probe Fluo-3AM. PA-3 beta-ol (10, 50 and 100 mu mol/l) inhibited phenylephrine and KCl-induced contraction in either endothelium-intact or denuded rat aortic rings. PA-3 beta-ol also reduced CaCl(2)-induced contraction in Ca(2+)-free solution containing KCl (30 mmol/l) or phenylephrine (0.1 mu mol/l). PA-3 beta-ol (1-300 mu mol/l) concentration dependently relaxed phenylephrine-pre-contracted rings with intact or denuded endothelium. The diterpene also relaxed KCl-pre-contracted rings with intact or denuded endothelium. Moreover, Ca(2+) mobilization study showed that PA-3 beta-ol (100 mu mol/l) and verapamil (1 mu mol/l) inhibited the increase in Ca(2+)-concentration in smooth muscle and endothelial cells induced by phenylephrine (10 mu mol/l) or KCl (60 mmol/l). Pre-incubation of intact or denuded aortic rings with N(G)-nitro-L-arginine methyl ester (L-NAME, 100 mu mol/l) and 1H-[1,2,4] Oxadiazolo[4,3-a]quinoxalin-1-one (ODQ 1 mu mol/l) produced a rightward displacement of the PA-3 beta-ol concentration-response curves. On the other hand, 7-nitroindazole (100 mu mol/l), 1400 W (1 mu mol/l), indomethacin (10 mu mol/l) and tetraethylammonium (1 mmol/l) did not affect PA-3 beta-ol-induced relaxation. Collectively, our results provide evidence that the effects elicited by PA-3 beta-ol involve extracellular Ca(2+) influx blockade. Its effects are also partly mediated by the activation of NO-cGMP pathway. (C) 2009 Elsevier B.V. All rights reserved.

Suppressive action of melatonin on the TH-2 immune response in rats infected with Trypanosoma cruzi

Control of the acute phase of Trypanosoma cruzi infection is critically dependent on cytokine-mediated macrophage activation to intracellular killing, natural killer (NK) cells, CD4(+) T cells, CD8(+) T cells and B cells. Cell-mediated immunity in T. cruzi infection is also modulated by cytokines, but in addition to parasite-specific responses, autoimmunity can be also triggered. Importantly, cytokines may also play a role in the cell-mediated immunity of infected subjects. Here we studied the role of cytokines in the regulation of innate and adaptive immunity during the acute phase of T. cruzi infection in Wistar rats. Melatonin is an effective regulator of the immune system. Macrophages and T lymphocytes, which have melatonin receptors, are target cells for the immunomodulatory function of melatonin. In this paper melatonin was orally given via two protocols: prior to and concomitant with infection. Both treatments were highly effective against T. cruzi with enhanced action for the concomitant treatment. The data suggest an up-regulation of the TH-1 immune response as all analyzed parameters, interleukin (IL)-4, IL-10, transforming growth factor-beta 1 and splenocyte proliferation, displayed reduced levels as compared with the untreated counterparts. However, the direct effects of melatonin on immune cells have not been fully investigated during T. cruzi infection. We conclude that in light of the current results, melatonin exerted important therapeutic benefits through its immune regulatory effects.

Molecular dynamics, flexible docking, virtual screening, ADMET predictions, and molecular interaction field studies to design novel potential MAO-B inhibitors

Monoamine oxidase is a flavoenzyme bound to the mitochondrial outer membranes of the cells, which is responsible for the oxidative deamination of neurotransmitter and dietary amines. It has two distinct isozymic forms, designated MAO-A and MAO-B, each displaying different substrate and inhibitor specificities. They are the well-known targets for antidepressant, Parkinson`s disease, and neuroprotective drugs. Elucidation of the x-ray crystallographic structure of MAO-B has opened the way for the molecular modeling studies. In this work we have used molecular modeling, density functional theory with correlation, virtual screening, flexible docking, molecular dynamics, ADMET predictions, and molecular interaction field studies in order to design new molecules with potential higher selectivity and enzymatic inhibitory activity over MAO-B.

A Computational Framework for Institutional Agency

This paper provides a computational framework, based on Defeasible Logic, to capture some aspects of institutional agency. Our background is Kanger-Lindahl-P\"orn account of organised interaction, which describes this interaction within a multi-modal logical setting. This work focuses in particular on the notions of counts-as link and on those of attempt and of personal and direct action to realise states of affairs. We show how standard Defeasible Logic can be extended to represent these concepts: the resulting system preserves some basic properties commonly attributed to them. In addition, the framework enjoys nice computational properties, as it turns out that the extension of any theory can be computed in time linear to the size of the theory itself.

Linking Agentive Objects in Mandarin Chinese

This paper is concerned with the problem of argument-function mismatch observed in the apparent subject-object inversion in Chinese consumption verbs, e.g., chi 'eat' and he 'drink', and accommodation verbs, e.g., zhu 'live' and shui 'sleep'. These verbs seem to allow the linking of [agent-SUBJ theme-OBJ] as well as [agent-OBJ theme-SUBJ], but only when the agent is also the semantic role denoting the measure or extent of the action. The account offered is formulated within LFG's lexical mapping theory. Under the simplest and also the strictest interpretation of the one-to-one argument-function mapping principle (or the theta-criterion), a composite role such as ag-ext receives syntactic assignment via one composing role only. One-to-one linking thus entails the suppression of the other composing role. Apparent subject-object inversion occurs when the more prominent agent role is suppressed and thus allows the less prominent extent role to dictate the linking of the entire ag-ext composite role. This LMT account also potentially facilitates a natural explanation of markedness among the competing syntactic structures.

Constraints on the spatiotemporal accuracy of interceptive action: effects of target size on hitting a moving object

Results of two experiments are reported that examined how people respond to rectangular targets of different sizes in simple hitting tasks. If a target moves in a straight line and a person is constrained to move along a linear track oriented perpendicular to the targetrsquos motion, then the length of the target along its direction of motion constrains the temporal accuracy and precision required to make the interception. The dimensions of the target perpendicular to its direction of motion place no constraints on performance in such a task. In contrast, if the person is not constrained to move along a straight track, the targetrsquos dimensions may constrain the spatial as well as the temporal accuracy and precision. The experiments reported here examined how people responded to targets of different vertical extent (height): the task was to strike targets that moved along a straight, horizontal path. In experiment 1 participants were constrained to move along a horizontal linear track to strike targets and so target height did not constrain performance. Target height, length and speed were co-varied. Movement time (MT) was unaffected by target height but was systematically affected by length (briefer movements to smaller targets) and speed (briefer movements to faster targets). Peak movement speed (Vmax) was influenced by all three independent variables: participants struck shorter, narrower and faster targets harder. In experiment 2, participants were constrained to move in a vertical plane normal to the targetrsquos direction of motion. In this task target height constrains the spatial accuracy required to contact the target. Three groups of eight participants struck targets of different height but of constant length and speed, hence constant temporal accuracy demand (different for each group, one group struck stationary targets = no temporal accuracy demand). On average, participants showed little or no systematic response to changes in spatial accuracy demand on any dependent measure (MT, Vmax, spatial variable error). The results are interpreted in relation to previous results on movements aimed at stationary targets in the absence of visual feedback.