A proposal of a behavior-based control architecture with reinforcement learning for an autonomous underwater robot

Aquesta tesi proposa l'ús d'un seguit de tècniques pel control a alt nivell d'un robot autònom i també per l'aprenentatge automàtic de comportaments. L'objectiu principal de la tesis fou el de dotar d'intel·ligència als robots autònoms que han d'acomplir unes missions determinades en entorns desconeguts i no estructurats. Una de les premisses tingudes en compte en tots els passos d'aquesta tesis va ser la selecció d'aquelles tècniques que poguessin ésser aplicades en temps real, i demostrar-ne el seu funcionament amb experiments reals. El camp d'aplicació de tots els experiments es la robòtica submarina. En una primera part, la tesis es centra en el disseny d'una arquitectura de control que ha de permetre l'assoliment d'una missió prèviament definida. En particular, la tesis proposa l'ús de les arquitectures de control basades en comportaments per a l'assoliment de cada una de les tasques que composen la totalitat de la missió. Una arquitectura d'aquest tipus està formada per un conjunt independent de comportaments, els quals representen diferents intencions del robot (ex.: "anar a una posició", "evitar obstacles",...). Es presenta una recerca bibliogràfica sobre aquest camp i alhora es mostren els resultats d'aplicar quatre de les arquitectures basades en comportaments més representatives a una tasca concreta. De l'anàlisi dels resultats se'n deriva que un dels factors que més influeixen en el rendiment d'aquestes arquitectures, és la metodologia emprada per coordinar les respostes dels comportaments. Per una banda, la coordinació competitiva és aquella en que només un dels comportaments controla el robot. Per altra banda, en la coordinació cooperativa el control del robot és realitza a partir d'una fusió de totes les respostes dels comportaments actius. La tesis, proposa un esquema híbrid d'arquitectura capaç de beneficiar-se dels principals avantatges d'ambdues metodologies. En una segona part, la tesis proposa la utilització de l'aprenentatge per reforç per aprendre l'estructura interna dels comportaments. Aquest tipus d'aprenentatge és adequat per entorns desconeguts i el procés d'aprenentatge es realitza al mateix temps que el robot està explorant l'entorn. La tesis presenta també un estat de l'art d'aquest camp, en el que es detallen els principals problemes que apareixen en utilitzar els algoritmes d'aprenentatge per reforç en aplicacions reals, com la robòtica. El problema de la generalització és un dels que més influeix i consisteix en permetre l'ús de variables continues sense augmentar substancialment el temps de convergència. Després de descriure breument les principals metodologies per generalitzar, la tesis proposa l'ús d'una xarxa neural combinada amb l'algoritme d'aprenentatge per reforç Q_learning. Aquesta combinació proporciona una gran capacitat de generalització i una molt bona disposició per aprendre en tasques de robòtica amb exigències de temps real. No obstant, les xarxes neurals són aproximadors de funcions no-locals, el que significa que en treballar amb un conjunt de dades no homogeni es produeix una interferència: aprendre en un subconjunt de l'espai significa desaprendre en la resta de l'espai. El problema de la interferència afecta de manera directa en robòtica, ja que l'exploració de l'espai es realitza sempre localment. L'algoritme proposat en la tesi té en compte aquest problema i manté una base de dades representativa de totes les zones explorades. Així doncs, totes les mostres de la base de dades s'utilitzen per actualitzar la xarxa neural, i per tant, l'aprenentatge és homogeni. Finalment, la tesi presenta els resultats obtinguts amb la arquitectura de control basada en comportaments i l'algoritme d'aprenentatge per reforç. Els experiments es realitzen amb el robot URIS, desenvolupat a la Universitat de Girona, i el comportament après és el seguiment d'un objecte mitjançant visió per computador. La tesi detalla tots els dispositius desenvolupats pels experiments així com les característiques del propi robot submarí. Els resultats obtinguts demostren la idoneïtat de les propostes en permetre l'aprenentatge del comportament en temps real. En un segon apartat de resultats es demostra la capacitat de generalització de l'algoritme d'aprenentatge mitjançant el "benchmark" del "cotxe i la muntanya". Els resultats obtinguts en aquest problema milloren els resultats d'altres metodologies, demostrant la millor capacitat de generalització de les xarxes neurals.

Acceptance of earphones in children 12- to 24-months of age during visual reinforcement audiometry

Insert and circumaural earphones were used during visual reinforcement audiometry with children 12-to 24-months of age. Acceptance of earphones was determined by the number of ear specific thresholds obtained and by audiologist subjective ratings. Results indicate that children in this age range accept both types of earphones; however, significantly more ear specific thresholds were obtained using insert earphones compared to circumaural.

Modelling the movement and survival of the root-feeding clover weevil, Sitona lepidus, in the root-zone of white clover

White clover (Trifolium repens) is an important pasture legume but is often difficult to sustain in a mixed sward because, among other things, of the damage to roots caused by the soil-dwelling larval stages of S. lepidus. Locating the root nodules on the white clover roots is crucial for the survival of the newly hatched larvae. This paper presents a numerical model to simulate the movement of newly hatched S. lepidus larvae towards the root nodules, guided by a chemical signal released by the nodules. The model is based on the diffusion-chemotaxis equation. Experimental observations showed that the average speed of the larvae remained approximately constant, so the diffusion-chernotaxis model was modified so that the larvae respond only to the gradient direction of the chemical signal but not its magnitude. An individual-based lattice Boltzmann method was used to simulate the movement of individual larvae, and the parameters required for the model were estimated from the measurement of larval movement towards nodules in soil scanned using X-ray microtomography. The model was used to investigate the effects of nodule density, the rate of release of chemical signal, the sensitivity of the larvae to the signal, and the random foraging of the larvae on the movement and subsequent survival of the larvae. The simulations showed that the most significant factors for larval survival were nodule density and the sensitivity of the larvae to the signal. The dependence of larval survival rate on nodule density was well fitted by the Michealis-Menten kinetics. (c) 2005 Elsevier B.V All rights reserved.

Influence of sodium chloride on seed germination and seedling root growth of cotton (Gossypium hirsutum L.)

Response of cotton (Gossypium hirsutum L. cv. NIAB-78) to salinity, in terms of seed germination, seedling root growth and root Na+ and K+ content was determined in a laboratory experiment. Cotton seeds were exposed to increasing salinity levels using germination water with Sodium chloride concentrations of 0, 50, 100, 150 and 200 mM, to provide different degrees of salt stress. Germinated seeds were counted and roots were harvested at 24, 48, 72 and 96 h after the start of the experiment. It appeared that seed germination was only slightly affected by an increase in salinity (in most cases the differences between treatment were non-significant), whereas root length, root growth rate, root fresh and dry weights were severely affected, generally highly significant differences in these variables were found for comparisons involving most combinations of salinity levels, in particular with increased incubation period. K+ contents decreased with increasing salinity levels, although differences in K+ content were only significant when comparing the control and the 4 salinity levels. Na+ content of the roots increased with increasing levels of NaCl in the germination water, suggesting an exchange of K+ for Na+. The ratio K+/Na+ strongly decreased with rising levels of salinity from around 4.5 for the control to similar to 1 at 200 mM NaCl.

Measuring root traits in barley (Hordeum vulgare ssp vulgare and ssp spontaneum) seedlings using gel chambers, soil sacs and X-ray microtomography

Root characteristics of seedlings of five different barley genotypes were analysed in 2D using gel chambers, and in 3D using soil sacs that were destructively harvested and pots of soil that were assessed non-invasively using X-ray microtomography. After 5 days, Chime produced the greatest number of root axes (similar to 6) and Mehola significantly less (similar to 4) in all growing methods. Total root length was longest in GSH01915 and shortest in Mehola for all methods, but both total length and average root diameter were significantly larger for plants grown in gel chambers than those grown in soil. The ranking of particular growth traits (root number, root angular spread) of plants grown in gel plates, soil sacs and X-ray pots was similar, but plants grown in the gel chambers had a different order of ranking for root length to the soil-grown plants. Analysis of angles in soil-grown plants showed that Tadmore had the most even spread of individual roots and Chime had a propensity for non-uniform distribution and root clumping. The roots of Mehola were less well spread than the barley cultivars supporting the suggestion that wild and landrace barleys tend to have a narrower angular spread than modern cultivars. The three dimensional analysis of root systems carried out in this study provides insights into the limitations of screening methods for root traits and useful data for modelling root architecture.

Host plant recognition by the root feeding clover weevil, Sitona lepidus (Coleoptera : Curculionidae)

This study investigated the ability of neonatal larvae of the root-feeding weevil, Sitona lepidus Gyllenhal, to locate white clover Trifolium repens L. (Fabaceae) roots growing in soil and to distinguish them from the roots of other species of clover and a co-occurring grass species. Choice experiments used a combination of invasive techniques and the novel technique of high resolution X-ray microtomography to non-invasively track larval movement in the soil towards plant roots. Burrowing distances towards roots of different plant species were also examined. Newly hatched S. lepidus recognized T. repens roots and moved preferentially towards them when given a choice of roots of subterranean clover, Trifolium subterraneum L. (Fabaceae), strawberry clover Trifolium fragiferum L. (Fabaceae), or perennial ryegrass Lolium perenne L. (Poaceae). Larvae recognized T. repens roots, whether released in groups of five or singly, when released 25 mm (meso-scale recognition) or 60 mm (macro-scale recognition) away from plant roots. There was no statistically significant difference in movement rates of larvae.

Effects of carbon dioxide on the searching behaviour of the root-feeding clover weevil Sitona lepidus (Coleoptera : Curculionidae)

The respiratory emission of CO2 from roots is frequently proposed as an attractant that allows soil-dwelling insects to locate host plant roots, but this role has recently become less certain. CO2 is emitted from many sources other than roots, so does not necessarily indicate the presence of host plants, and because of the high density of roots in the upper soil layers, spatial gradients may not always be perceptible by soil-dwelling insects. The role of CO2 in host location was investigated using the clover root weevil Sitona lepidus Gyllenhall and its host plant white clover (Trifolium repens L.) as a model system. Rhizochamber experiments showed that CO2 concentrations were approximately 1000 ppm around the roots of white clover, but significantly decreased with increasing distance from roots. In behavioural experiments, no evidence was found for any attraction by S. lepidus larvae to point emissions of CO2, regardless of emission rates. Fewer than 15% of larvae were attracted to point emissions of CO2, compared with a control response of 17%. However, fractal analysis of movement paths in constant CO2 concentrations demonstrated that searching by S. lepidus larvae significantly intensified when they experienced CO2 concentrations similar to those found around the roots of white clover (i.e. 1000 ppm). It is suggested that respiratory emissions of CO2 may act as a 'search trigger' for S. lepidus, whereby it induces larvae to search a smaller area more intensively, in order to detect location cues that are more specific to their host plant.

Modelling cereal root systems for water and nitrogen capture: Towards an economic optimum

A quantitative model of wheat root systems is developed that links the size and distribution of the root system to the capture of water and nitrogen (which are assumed to be evenly distributed with depth) during grain filling, and allows estimates of the economic consequences of this capture to be assessed. A particular feature of the model is its use of summarizing concepts, and reliance on only the minimum number of parameters (each with a clear biological meaning). The model is then used to provide an economic sensitivity analysis of possible target characteristics for manipulating root systems. These characteristics were: root distribution with depth, proportional dry matter partitioning to roots, resource capture coefficients, shoot dry weight at anthesis, specific root weight and water use efficiency. From the current estimates of parameters it is concluded that a larger investment by the crop in fine roots at depth in the soil, and less proliferation of roots in surface layers, would improve yields by accessing extra resources. The economic return on investment in roots for water capture was twice that of the same amount invested for nitrogen capture. (C) 2003 Annals of Botany Company.

Effect of mycorrhizae and pH change at the root-soil interface on phosphorus uptake by Sorghum using a rhizocylinder technique

Sorghum (Sorghum bicolor) was grown for 40 days in. rhizocylinder (a growth container which permitted access to rh zosphere and nonrhizosphere soil), in two soils of low P status. Soils were fertilized with different rates of ammonium and nitrate and supplemented with 40 mg phosphorus (P) kg(-1) and inoculated with either Glomus mosseae (Nicol. and Gerd.) or nonmycorrhizal root inoculum.. N-serve (2 mg kg(-1)) was added to prevent nitrification. At harvest, soil from around the roots was collected at distances of 0-5, 5-10, and 10-20 mm from the root core which was 35 mm diameter. Sorghum plants, with and without mycorrhiza, grew larger with NH4+ than with NO3- application. After measuring soil pH, 4 3 suspensions of the same sample were titrated against 0.01 M HCl or 0.01 M NaOH until soil pH reached the nonplanted pH level. The acid or base requirement for each sample was calculated as mmol H+ or OFF kg(-1) soil. The magnitude of liberated acid or base depended on the form and rate of nitrogen and soil type. When the plant root was either uninfected or infected with mycorrhiza., soil pH changes extended up to 5 mm from the root core surface. In both soils, ammonium as an N source resulted in lower soil pH than nitrate. Mycorrhizal (VAM) inoculation did not enhance this difference. In mycorrhizal inoculated soil, P depletion extended tip to 20 mm from the root surface. In non-VAM inoculated soil P depletion extended up to 10 mm from the root surface and remained unchanged at greater distances. In the mycorrhizal inoculated soils, the contribution of the 0-5 mm soil zone to P uptake was greater than the core soil, which reflects the hyphal contribution to P supply. Nitrogen (N) applications that caused acidification increased P uptake because of increased demand; there is no direct evidence that the increased uptake was due to acidity increasing the solubility of P although this may have been a minor effect.