Efficient learning of reactive robot behaviors with a Neural-Q_learning approach

The purpose of this paper is to propose a Neural-Q_learning approach designed for online learning of simple and reactive robot behaviors. In this approach, the Q_function is generalized by a multi-layer neural network allowing the use of continuous states and actions. The algorithm uses a database of the most recent learning samples to accelerate and guarantee the convergence. Each Neural-Q_learning function represents an independent, reactive and adaptive behavior which maps sensorial states to robot control actions. A group of these behaviors constitutes a reactive control scheme designed to fulfill simple missions. The paper centers on the description of the Neural-Q_learning based behaviors showing their performance with an underwater robot in a target following task. Real experiments demonstrate the convergence and stability of the learning system, pointing out its suitability for online robot learning. Advantages and limitations are discussed

Numerical model of solid phase transformations governed by nucleation and growth: microstructure development during isothermal crystallization

A simple numerical model which calculates the kinetics of crystallization involving randomly distributed nucleation and isotropic growth is presented. The model can be applied to different thermal histories and no restrictions are imposed on the time and the temperature dependences of the nucleation and growth rates. We also develop an algorithm which evaluates the corresponding emerging grain-size distribution. The algorithm is easy to implement and particularly flexible, making it possible to simulate several experimental conditions. Its simplicity and minimal computer requirements allow high accuracy for two- and three-dimensional growth simulations. The algorithm is applied to explore the grain morphology development during isothermal treatments for several nucleation regimes. In particular, thermal nucleation, preexisting nuclei, and the combination of both nucleation mechanisms are analyzed. For the first two cases, the universal grain-size distribution is obtained. The high accuracy of the model is stated from its comparison to analytical predictions. Finally, the validity of the Kolmogorov-Johnson-Mehl-Avrami model SSSR, is verified for all the cases studied

Selenium cycling in volcanic environments: the role of soils as reactive interfaces

Selenium (Se) is an element with important health implications that is emitted in significant amounts from volcanoes. Attracted by the fertility of volcanic soils, around 10% of the world population lives within 100 km of an active volcano. Nevertheless, the behaviour of Se in volcanic environments is poorly understood. Therefore, the main aim of this thesis is to investigate the role of soils in the Se cycling in volcanic environments. Prior to the geochemical studies, precise and accurate methods for the determination of Se contents, speciation and isotopic signatures were developed. Afterwards, a combination of field studies and lab controlled experiments were performed with soils from two contrasting European volcanic settings: Mount Etna in Sicily (Italy) and Mount Teide in Tenerife (Spain). The results showed a strong link between Se behaviour and soil development, indicating that Se mobility in volcanic soils is controlled by sorption processes and soil mineralogy.