Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

A family of implementation-friendly BN elliptic curves

For the last decade, elliptic curve cryptography has gained increasing interest in industry and in the academic community. This is especially due to the high level of security it provides with relatively small keys and to its ability to create very efficient and multifunctional cryptographic schemes by means of bilinear pairings. Pairings require pairing-friendly elliptic curves and among the possible choices, Barreto-Naehrig (BN) curves arguably constitute one of the most versatile families. In this paper, we further expand the potential of the BN curve family. We describe BN curves that are not only computationally very simple to generate, but also specially suitable for efficient implementation on a very broad range of scenarios. We also present implementation results of the optimal ate pairing using such a curve defined over a 254-bit prime field. (C) 2001 Elsevier Inc. All rights reserved.

Inertia in two simple problems of soil hydrodynamics

In the present paper the dynamic solutions of two non-steady seepage problems are discussed. It is shown that the acceleration term in the equation of motion is important for a correct qualitative description of the flow.

A simple and effective inverse projection scheme for void distribution control in topology optimization

The ability to control both the minimum size of holes and the minimum size of structural members are essential requirements in the topology optimization design process for manufacturing. This paper addresses both requirements by means of a unified approach involving mesh-independent projection techniques. An inverse projection is developed to control the minimum hole size while a standard direct projection scheme is used to control the minimum length of structural members. In addition, a heuristic scheme combining both contrasting requirements simultaneously is discussed. Two topology optimization implementations are contributed: one in which the projection (either inverse or direct) is used at each iteration; and the other in which a two-phase scheme is explored. In the first phase, the compliance minimization is carried out without any projection until convergence. In the second phase, the chosen projection scheme is applied iteratively until a solution is obtained while satisfying either the minimum member size or minimum hole size. Examples demonstrate the various features of the projection-based techniques presented.

Methods of Fourier optics in digital holographic microscopy

In this paper, processing methods of Fourier optics implemented in a digital holographic microscopy system are presented. The proposed methodology is based on the possibility of the digital holography in carrying out the whole reconstruction of the recorded wave front and consequently, the determination of the phase and intensity distribution in any arbitrary plane located between the object and the recording plane. In this way, in digital holographic microscopy the field produced by the objective lens can be reconstructed along its propagation, allowing the reconstruction of the back focal plane of the lens, so that the complex amplitudes of the Fraunhofer diffraction, or equivalently the Fourier transform, of the light distribution across the object can be known. The manipulation of Fourier transform plane makes possible the design of digital methods of optical processing and image analysis. The proposed method has a great practical utility and represents a powerful tool in image analysis and data processing. The theoretical aspects of the method are presented, and its validity has been demonstrated using computer generated holograms and images simulations of microscopic objects. (c) 2007 Elsevier B.V. All rights reserved.

Application of the essential work of fracture method in ranking the performance in service of high-density polyethylene resins employed in pressure pipes

High-density polyethylene resins have increasingly been used in the production of pipes for water- and gas-pressurized distribution systems and are expected to remain in service for several years, but they eventually fail prematurely by creep fracture. Usual standard methods used to rank resins in terms of their resistance to fracture are expensive and non-practical for quality control purposes, justifying the search for alternative methods. Essential work of fracture (EWF) method provides a relatively simple procedure to characterize the fracture behavior of ductile polymers, such as polyethylene resins. In the present work, six resins were analyzed using the EWF methodology. The results show that the plastic work dissipation factor, beta w(p), is the most reliable parameter to evaluate the performance. Attention must be given to specimen preparation that might result in excessive dispersion in the results, especially for the essential work of fracture w(e).

Texture and mechanical properties evolution of a deep drawing medium carbon steel during cold rolling and subsequent recrystallization

Medium carbon steels are mostly used for simple applications; however, new applications have been developed for which good sheet metal formability is required. These types of steels have an inherent low formability. A medium-carbon hot-rolled SAE 1050 steel was selected for this study. It has been cold rolled with thickness reductions varying between 7 and 80%. The samples obtained were used to evaluate the strain hardening curve. For samples with a 50 and 80% thickness reduction, an annealing heat treatment was performed to achieve recrystallization. The material was characterized in the ""as-received"", cold rolled and annealed conditions using several methods: optical metallography, X-ray diffraction (texture), Vickers hardness, and tensile testing. For large thickness reductions, the SAE 1050 steel presented low elongation, less than 2%, and yield strength (YS) and tensile strength (TS) around 1400 MPa. Texture in the ""as-received"" condition showed strong components on the {001} plane, in the < 100 >, < 210 > and (110) directions. After cold rolling, the texture did not present any significant changes for small thickness reductions, however. It changed completely for large ones, where gamma, < 111 >//ND, alpha, < 110 > HRD, and gamma prime, < 223 >//ND, fibres were strengthened. After annealing, the microstructure of the SAE 1050 steel was characterized by recrystallized ferrite and globular cementite. There was little change in the alpha fibre for the 50% reduction, whereas for the 80% reduction, its intensity increased. Both gamma and gamma prime fibres vanished upon annealing for 50 and 80% reductions alike. (c) 2008 Elsevier B.V. All rights reserved.

Medium carbon steel deep drawing: A study on the evolution of mechanical properties, texture and simulations, from cold rolling to the end product

Medium carbon steels are mostly used for simple applications; nevertheless new applications have been developed for which good sheet formability is required. This class of steels has an inherent low formability. A medium carbon hot rolled SAE 1050 steel has been selected for this study. It has been cold rolled with reductions in the 7-80% range. Samples have been used to assess the cold work hardening curve. For samples with a 50 and 80% thickness reduction, an annealing heat treatment has been performed to obtain recrystallization. The material has been characterized in the ""as received"", cold rolled and annealed conditions, using several methods: optical microscopy, X-ray diffraction (texture), Vickers hardness and tensile testing. The 50% cold rolled and recrystallized material has been further studied in terms of sheet metal formability and texture evolution during the actual stamping of a steel toecap that has been used to validate the finite element simulations. (C) 2008 Elsevier B.V. All rights reserved.

Application of Fluorescence to the Study of Crude Petroleum

Crude petroleum oils are complex mixtures of different compounds (mainly organic), which are obtained from an extensive range of different geological sources. The fluorescence of crude petroleum oils derives largely from the aromatic hydrocarbon fraction, and this fluorescence emission is strongly influenced by the chemical composition (e.g., fluorophore and quencher concentrations) and physical characteristics (e.g., viscosity and optical density) of the oil. The fluorescence spectroscopy (FS) is increasingly used in petroleum technology due the availability of better optical detection techniques, because FS offers high sensitivity, good diagnostic potential, and relatively simple instrumentation. In this work we analyzed crude petroleum at different dilution in Nujol, a transparent mineral oil. The main objective of this work was to verify the possibility to measure crude oil emission spectroscopic without use of volatile solvents. The mixtures of nujol with different -crude oil concentrations were measured with a 10 mm optical path cuvette thus simplifying the fluorescence spectroscopy signal detection. The emission spectra were obtained by exciting the samples with a 400 W Xenon lamp at 350 nm, 450 nm and 532 nm. The emissions of the samples were collected perpendicularly with the excitation axis.

Adaptive Carrier Tracking for Mars to Earth Communications During Entry, Descent, and Landing

We propose a robust and low complexity scheme to estimate and track carrier frequency from signals traveling under low signal-to-noise ratio (SNR) conditions in highly nonstationary channels. These scenarios arise in planetary exploration missions subject to high dynamics, such as the Mars exploration rover missions. The method comprises a bank of adaptive linear predictors (ALP) supervised by a convex combiner that dynamically aggregates the individual predictors. The adaptive combination is able to outperform the best individual estimator in the set, which leads to a universal scheme for frequency estimation and tracking. A simple technique for bias compensation considerably improves the ALP performance. It is also shown that retrieval of frequency content by a fast Fourier transform (FFT)-search method, instead of only inspecting the angle of a particular root of the error predictor filter, enhances performance, particularly at very low SNR levels. Simple techniques that enforce frequency continuity improve further the overall performance. In summary we illustrate by extensive simulations that adaptive linear prediction methods render a robust and competitive frequency tracking technique.

Vortex configuration flow cell based on low temperature cofired ceramics as a compact chemiluminescence microsystern

The integration of optical detection methods in continuous flow microsystems can highly extend their range of application, as long as some negative effects derived from their scaling down can be minimized. Downsizing affects to a greater extent the sensitivity of systems based on absorbance measurements than the sensitivity of those based on emission ones. However, a careful design of the instrumental setup is needed to maintain the analytical features in both cases. In this work, we present the construction and evaluation of a simple miniaturized optical system, which integrates a novel flow cell configuration to carry out chemiluminescence (CL) measurements using a simple photodiode. It consists of a micro-mixer based on a vortex structure, which has been constructed by means of the low-temperature cofired ceramics (LTCC) technology. This mixer not only efficiently promotes the CL reaction due to the generated high turbulence but also allows the detection to be carried out in the same area, avoiding intensity signal losses. As a demonstration, a flow injection system has been designed and optimized for the detection of cobalt(H) in water samples. It shows a linear response between 2 and 20 mu M with a correlation of r > 0.993, a limit of detection of 1.1 mu M, a repeatability of RSD = 12.4 %, and an analysis time of 17 s. These results demonstrate the suitability of the proposal to the determination of compounds involved in CL reactions by means of an easily constructed versatile device based on low-cost instrumentation.

Universal Set-top Box: A Simple Design to Provide Accessible Services

This paper presents the design of a low cost accessible digital television set-top box. This set-top box was designed and tested to the International ISDB-T system and considered the adoption of solutions that would provide accessible services in digital television in the simplest digital television receiver. The accessible set-top box was evaluated regarding the processing and memory requirements impacts to provide the features for accessible services. The work presents also the access services bandwidth consumption analysis(1).

Simple Excitation Model for Coaxial Driven Monopole Antennas

A new excitation model for the numerical solution of field integral equation (EFIE) applied to arbitrarily shaped monopole antennas fed by coaxial lines is presented. This model yields a stable solution for the input impedance of such antennas with very low numerical complexity and without the convergence and high parasitic capacitance problems associated with the usual delta gap excitation.

New simple and efficient heuristics for the uncapacitated single allocation hub location problem

Hub-and-spoke networks are widely studied in the area of location theory. They arise in several contexts, including passenger airlines, postal and parcel delivery, and computer and telecommunication networks. Hub location problems usually involve three simultaneous decisions to be made: the optimal number of hub nodes, their locations and the allocation of the non-hub nodes to the hubs. In the uncapacitated single allocation hub location problem (USAHLP) hub nodes have no capacity constraints and non-hub nodes must be assigned to only one hub. In this paper, we propose three variants of a simple and efficient multi-start tabu search heuristic as well as a two-stage integrated tabu search heuristic to solve this problem. With multi-start heuristics, several different initial solutions are constructed and then improved by tabu search, while in the two-stage integrated heuristic tabu search is applied to improve both the locational and allocational part of the problem. Computational experiments using typical benchmark problems (Civil Aeronautics Board (CAB) and Australian Post (AP) data sets) as well as new and modified instances show that our approaches consistently return the optimal or best-known results in very short CPU times, thus allowing the possibility of efficiently solving larger instances of the USAHLP than those found in the literature. We also report the integer optimal solutions for all 80 CAB data set instances and the 12 AP instances up to 100 nodes, as well as for the corresponding new generated AP instances with reduced fixed costs. Published by Elsevier Ltd.

Enhancement of europium emission band of europium tetracycline complex in the presence of cholesterol

We report here the observation, for the first time, of the enhancement of Europium-Tetracycline complex emission in cholesterol solutions. This enhancement was initially observed with the addition of the enzyme cholesterol oxidase, which produces H2O2, the agent driver of the Europium tetracycline complex, to the solution. However, it was found that the enzyme is not needed to enhance the luminescence. A calibration curve was determined, resulting in a simple method to measure the cholesterol quantity in a solution. This method shows that the complex can be used as a sensor to determine cholesterol in biological systems.