Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Investigation of a method for predicting AUV derivatives

The paper addresses the problem of autonomous underwater vehicle (AUV) modelling and parameter estimation as a means to predict the dynamic performance of underwater vehicles and thus provide solid guidelines during their design phase. The use of analytical and semi-empirical (ASE) methods to estimate the hydrodynamic derivatives of a popular class of AUVs is discussed. A comparison is done with the results obtained by using computational fluid dynamics to evaluate the bare hull lift force distribution around a fully submerged body. An application is made to the estimation of the hydrodynamic derivatives of the MAYA AUV, an autonomous underwater vehicle developed under a joint Indian-Portuguese project. The estimates obtained were used to predict the turning diameter of the vehicle during sea trials. (C) 2008 Elsevier Ltd. All rights reserved.

Success of surgery for primary aldosteronism judged by residual autonomous aldosterone production

Since February 1996 we have prospectively assessed residual adrenal autonomy by the fludrocortisone suppression test (FST) in 23 patients 3 months after unilateral adrenalectomy for Conn syndrome and in 45 patients after a longer interval. In regard to blood pressure, 36 (53%) patients were cured of hypertension and the remaining 32 (47%) patients had improved hypertension control at the time of their latest postoperative clinical assessment. In regard to the outcome of surgery, patients who achieved normal suppressibility of aldosterone were regarded as cured, and those who had greater suppressibility after surgery were considered improved. Time since surgery for the whole group averaged 26 months. By these biochemical criteria, 42 patients (62%) were cured by surgery, and the rest improved; 16 (76%) of 21 women were cured, and 26 (55%) of 47 men. The women (mean +/- SD age 47 +/- 11 years) were significantly (p < 0.05) younger than the men (52 +/- 9 Sears). Preoperative aldosterone levels before and after FST were similar in the cured and improved groups and fell significantly (p < 0.01) in both groups following surgery. After surgical reduction of autonomous aldosterone production, mean plasma renin activity levels increased sixfold in the cured group and threefold in the improved group. Surgical mortality in this group of 68 patients with Conn syndrome was zero.

The dimerization and topological specificity functions of MinE reside in a structurally autonomous C-terminal domain

Correct placement of the division septum in Escherichia coli requires the co-ordinated action of three proteins, MinC, MinD and MinE. MinC and MinD interact to form a non-specific division inhibitor that blocks septation at all potential division sites. MinE is able to antagonize MinCD in a topologically sensitive manner, as it restricts MinCD activity to the unwanted division sites at the cell poles, Here, we show that the topological specificity function of MinE residues in a structurally autonomous, trypsin-resistant domain comprising residues 31-88, Nuclear magnetic resonance (NMR) and circular dichroic spectroscopy indicate that this domain includes both alpha and beta secondary structure, while analytical ultracentrifugation reveals that it also contains a region responsible for MinE homodimerization. While trypsin digestion indicates that the anti-MinCD domain of MinE (residues 1-22) does not form a tightly folded structural domain, NMR analysis of a peptide corresponding to MinE(1-22) indicates that this region forms a nascent helix in which the peptide rapidly interconverts between disordered (random coil) and alpha-helical conformations, This suggests that the N-terminal region of MinE may be poised to adopt an alpha-helical conformation when it interacts with the target of its anti-MinCD activity, presumably MinD.

Vehicle and driver attributes affecting distance from the steering wheel in motor vehicles

The current study was designed to confirm that female drivers sit closer to the steering wheel than do male drivers and to investigate whether this expected difference in sitting position is attributable to differences in the physical dimensions of men and women. Driver body dimensions and multiple measures of sitting distance from the steering wheel were collected from a sample of 150 men and 150 women. The results confirmed that on average, women sit closer to the steering wheel than men do and that this difference is accounted for by variations in body dimensions, especially height. This result suggests that driver height may provide a good surrogate for sitting distance from the steering wheel when investigating the role of driver position in real-world crash outcomes. The potential applications of this research include change to vehicle design that allows independent adjustment of the relative distance among the driver's seat, the steering wheel, and the floor pedals.

C(alpha)-HOLDER CLASSICAL SOLUTIONS FOR NON-AUTONOMOUS NEUTRAL DIFFERENTIAL EQUATIONS

In this paper we discuss the existence of alpha-Holder classical solutions for non-autonomous abstract partial neutral functional differential equations. An application is considered.

Identification of novel non-autonomous CemaT transposable elements and evidence of their mobility within the C. elegans genome

We describe here two new transposable elements, CemaT4 and CemaT5, that were identified within the sequenced genome of Caenorhabditis elegans using homology based searches. Five variants of CemaT4 were found, all non-autonomous and sharing 26 bp inverted terminal repeats (ITRs) and segments (152-367 bp) of sequence with similarity to the CemaT1 transposon of C. elegans. Sixteen copies of a short, 30 bp repetitive sequence, comprised entirely of an inverted repeat of the first 15 bp of CemaT4's ITR, were also found, each flanked by TA dinucleotide duplications, which are hallmarks of target site duplications of mariner-Tc transposon transpositions. The CemaT5 transposable element had no similarity to maT elements, except for sharing identical ITR sequences with CemaT3. We provide evidence that CemaT5 and CemaT3 are capable of excising from the C. elegans genome, despite neither transposon being capable of encoding a functional transposase enzyme. Presumably, these two transposons are cross-mobilised by an autonomous transposon that recognises their shared ITRs. The excisions of these and other non-autonomous elements may provide opportunities for abortive gap repair to create internal deletions and/or insert novel sequence within these transposons. The influence of non-autonomous element mobility and structural diversity on genome variation is discussed.

Probing the effect of vehicles on topical delivery: Understanding the basic relationship between solvent and solute penetration using silicone membranes

Purpose. In the present study we examined the relationship between solvent uptake into a model membrane (silicone) with the physical properties of the solvents (e.g., solubility parameter, melting point, molecular weight) and its potential predictability. We then assessed the subsequent topical penetration and retention kinetics of hydrocortisone from various solvents to define whether modifications to either solute diffusivity or partitioning were dominant in increasing permeability through solvent-modified membranes. Methods. Membrane sorption of solvents was determined from weight differences following immersion in individual solvents, corrected for differences in density. Permeability and retention kinetics of H-3-hydrocortisone, applied as saturated solutions in the various solvents, were determined over 48 h in horizontal Franz-type glass diffusion cells. Results. Solvent sorption into the membrane could be related to differences in solubility parameters, MW and hydrogen bonding (r(2) = 0.76). The actual and predicted volume of solvent sorbed into the membrane was also found to be linearly related to Log hydrocortisone flux, with changes in both diffusivity and partitioning of hydrocortisone observed for the different solvent vehicles. Conclusions. A simple structure-based predictive model can be applied to the sorption of solvents into silicone membranes. Changes in solute diffusivity and partitioning appeared to contribute to the increased hydrocortisone flux observed with the various solvent vehicles. The application of this predictive model to the more complex skin membrane remains to be determined.

Um algoritmo PSO híbrido para planejamento de caminhos em navegação de veículos utilizando A*

Utilizar robôs autônomos capazes de planejar o seu caminho é um desafio que atrai vários pesquisadores na área de navegação de robôs. Neste contexto, este trabalho tem como objetivo implementar um algoritmo PSO híbrido para o planejamento de caminhos em ambientes estáticos para veículos holonômicos e não holonômicos. O algoritmo proposto possui duas fases: a primeira utiliza o algoritmo A* para encontrar uma trajetória inicial viável que o algoritmo PSO otimiza na segunda fase. Por fim, uma fase de pós planejamento pode ser aplicada no caminho a fim de adaptá-lo às restrições cinemáticas do veículo não holonômico. O modelo Ackerman foi considerado para os experimentos. O ambiente de simulação de robótica CARMEN (Carnegie Mellon Robot Navigation Toolkit) foi utilizado para realização de todos os experimentos computacionais considerando cinco instâncias de mapas geradas artificialmente com obstáculos. O desempenho do algoritmo desenvolvido, A*PSO, foi comparado com os algoritmos A*, PSO convencional e A* Estado Híbrido. A análise dos resultados indicou que o algoritmo A*PSO híbrido desenvolvido superou em qualidade de solução o PSO convencional. Apesar de ter encontrado melhores soluções em 40% das instâncias quando comparado com o A*, o A*PSO apresentou trajetórias com menos pontos de guinada. Investigando os resultados obtidos para o modelo não holonômico, o A*PSO obteve caminhos maiores entretanto mais suaves e seguros.