Accelerating autonomous learning by using heuristic selection of actions

This paper investigates how to make improved action selection for online policy learning in robotic scenarios using reinforcement learning (RL) algorithms. Since finding control policies using any RL algorithm can be very time consuming, we propose to combine RL algorithms with heuristic functions for selecting promising actions during the learning process. With this aim, we investigate the use of heuristics for increasing the rate of convergence of RL algorithms and contribute with a new learning algorithm, Heuristically Accelerated Q-learning (HAQL), which incorporates heuristics for action selection to the Q-Learning algorithm. Experimental results on robot navigation show that the use of even very simple heuristic functions results in significant performance enhancement of the learning rate.

Hybrid Method to Assess Sensitive Process Interruption Costs Due to Faults in Electric Power Distribution Networks

This paper shows a new hybrid method for risk assessment regarding interruptions in sensitive processes due to faults in electric power distribution systems. This method determines indices related to long duration interruptions and short duration voltage variations (SDVV), such as voltage sags and swells in each customer supplied by the distribution network. Frequency of such occurrences and their impact on customer processes are determined for each bus and classified according to their corresponding magnitude and duration. The method is based on information regarding network configuration, system parameters and protective devices. It randomly generates a number of fault scenarios in order to assess risk areas regarding long duration interruptions and voltage sags and swells in an especially inventive way, including frequency of events according to their magnitude and duration. Based on sensitivity curves, the method determines frequency indices regarding disruption in customer processes that represent equipment malfunction and possible process interruptions due to voltage sags and swells. Such approach allows for the assessment of the annual costs associated with each one of the evaluated power quality indices.

A hybrid-mixed finite element formulation for the geometrically exact analysis of three-dimensional framed structures

This paper addresses the development of a hybrid-mixed finite element formulation for the quasi-static geometrically exact analysis of three-dimensional framed structures with linear elastic behavior. The formulation is based on a modified principle of stationary total complementary energy, involving, as independent variables, the generalized vectors of stress-resultants and displacements and, in addition, a set of Lagrange multipliers defined on the element boundaries. The finite element discretization scheme adopted within the framework of the proposed formulation leads to numerical solutions that strongly satisfy the equilibrium differential equations in the elements, as well as the equilibrium boundary conditions. This formulation consists, therefore, in a true equilibrium formulation for large displacements and rotations in space. Furthermore, this formulation is objective, as it ensures invariance of the strain measures under superposed rigid body rotations, and is not affected by the so-called shear-locking phenomenon. Also, the proposed formulation produces numerical solutions which are independent of the path of deformation. To validate and assess the accuracy of the proposed formulation, some benchmark problems are analyzed and their solutions compared with those obtained using the standard two-node displacement/ rotation-based formulation.

Hybrid and multi-field variational principles for geometrically exact three-dimensional beams

This paper addresses the development of several alternative novel hybrid/multi-field variational formulations of the geometrically exact three-dimensional elastostatic beam boundary-value problem. In the framework of the complementary energy-based formulations, a Legendre transformation is used to introduce the complementary energy density in the variational statements as a function of stresses only. The corresponding variational principles are shown to feature stationarity within the framework of the boundary-value problem. Both weak and linearized weak forms of the principles are presented. The main features of the principles are highlighted, giving special emphasis to their relationships from both theoretical and computational standpoints. (C) 2010 Elsevier Ltd. All rights reserved.

Toward Optimal Design of Piezoelectric Transducers Based on Multifunctional and Smoothly Graded Hybrid Material Systems

This work explores the design of piezoelectric transducers based on functional material gradation, here named functionally graded piezoelectric transducer (FGPT). Depending on the applications, FGPTs must achieve several goals, which are essentially related to the transducer resonance frequency, vibration modes, and excitation strength at specific resonance frequencies. Several approaches can be used to achieve these goals; however, this work focuses on finding the optimal material gradation of FGPTs by means of topology optimization. Three objective functions are proposed: (i) to obtain the FGPT optimal material gradation for maximizing specified resonance frequencies; (ii) to design piezoelectric resonators, thus, the optimal material gradation is found for achieving desirable eigenvalues and eigenmodes; and (iii) to find the optimal material distribution of FGPTs, which maximizes specified excitation strength. To track the desirable vibration mode, a mode-tracking method utilizing the `modal assurance criterion` is applied. The continuous change of piezoelectric, dielectric, and elastic properties is achieved by using the graded finite element concept. The optimization algorithm is constructed based on sequential linear programming, and the concept of continuum approximation of material distribution. To illustrate the method, 2D FGPTs are designed for each objective function. In addition, the FGPT performance is compared with the non-FGPT one.

Single Axis Controlled Hybrid Magnetic Bearing for Left Ventricular Assist Device: Hybrid Core and Closed Magnetic Circuit

In previous studies, we presented main strategies for suspending the rotor of a mixed-flow type (centrifugal and axial) ventricular assist device (VAD), originally presented by the Institute Dante Pazzanese of Cardiology (IDPC), Brazil. Magnetic suspension is achieved by the use of a magnetic bearing architecture in which the active control is executed in only one degree of freedom, in the axial direction of the rotor. Remaining degrees of freedom, excepting the rotation, are restricted only by the attraction force between pairs of permanent magnets. This study is part of a joint project in development by IDPC and Escola Politecnica of Sao Paulo University, Brazil. This article shows advances in that project, presenting two promising solutions for magnetic bearings. One solution uses hybrid cores as electromagnetic actuators, that is, cores that combine iron and permanent magnets. The other solution uses actuators, also of hybrid type, but with the magnetic circuit closed by an iron core. After preliminary analysis, a pump prototype has been developed for each solution and has been tested. For each prototype, a brushless DC motor has been developed as the rotor driver. Each solution was evaluated by in vitro experiments and guidelines are extracted for future improvements. Tests have shown good results and demonstrated that one solution is not isolated from the other. One complements the other for the development of a single-axis-controlled, hybrid-type magnetic bearing for a mixed-flow type VAD.

A hybrid commodity price-forecasting model applied to the sugar-alcohol sector

Accurate price forecasting for agricultural commodities can have significant decision-making implications for suppliers, especially those of biofuels, where the agriculture and energy sectors intersect. Environmental pressures and high oil prices affect demand for biofuels and have reignited the discussion about effects on food prices. Suppliers in the sugar-alcohol sector need to decide the ideal proportion of ethanol and sugar to optimise their financial strategy. Prices can be affected by exogenous factors, such as exchange rates and interest rates, as well as non-observable variables like the convenience yield, which is related to supply shortages. The literature generally uses two approaches: artificial neural networks (ANNs), which are recognised as being in the forefront of exogenous-variable analysis, and stochastic models such as the Kalman filter, which is able to account for non-observable variables. This article proposes a hybrid model for forecasting the prices of agricultural commodities that is built upon both approaches and is applied to forecast the price of sugar. The Kalman filter considers the structure of the stochastic process that describes the evolution of prices. Neural networks allow variables that can impact asset prices in an indirect, nonlinear way, what cannot be incorporated easily into traditional econometric models.

S/MIMO MC-CDMA Heuristic Multiuser Detectors Based on Single-Objective Optimization

This paper analyzes the complexity-performance trade-off of several heuristic near-optimum multiuser detection (MuD) approaches applied to the uplink of synchronous single/multiple-input multiple-output multicarrier code division multiple access (S/MIMO MC-CDMA) systems. Genetic algorithm (GA), short term tabu search (STTS) and reactive tabu search (RTS), simulated annealing (SA), particle swarm optimization (PSO), and 1-opt local search (1-LS) heuristic multiuser detection algorithms (Heur-MuDs) are analyzed in details, using a single-objective antenna-diversity-aided optimization approach. Monte- Carlo simulations show that, after convergence, the performances reached by all near-optimum Heur-MuDs are similar. However, the computational complexities may differ substantially, depending on the system operation conditions. Their complexities are carefully analyzed in order to obtain a general complexity-performance framework comparison and to show that unitary Hamming distance search MuD (uH-ds) approaches (1-LS, SA, RTS and STTS) reach the best convergence rates, and among them, the 1-LS-MuD provides the best trade-off between implementation complexity and bit error rate (BER) performance.

Some heuristic algorithms for total tardiness minimization in a flowshop with blocking

The flowshop scheduling problem with blocking in-process is addressed in this paper. In this environment, there are no buffers between successive machines: therefore intermediate queues of jobs waiting in the system for their next operations are not allowed. Heuristic approaches are proposed to minimize the total tardiness criterion. A constructive heuristic that explores specific characteristics of the problem is presented. Moreover, a GRASP-based heuristic is proposed and Coupled with a path relinking strategy to search for better outcomes. Computational tests are presented and the comparisons made with an adaptation of the NEH algorithm and with a branch-and-bound algorithm indicate that the new approaches are promising. (c) 2007 Elsevier Ltd. All rights reserved.

Inverse aerodynamic design applications using the MGM hybrid formulation

The well-known modified Garabedian-Mcfadden (MGM) method is an attractive alternative for aerodynamic inverse design, for its simplicity and effectiveness (P. Garabedian and G. Mcfadden, Design of supercritical swept wings, AIAA J. 20(3) (1982), 289-291; J.B. Malone, J. Vadyak, and L.N. Sankar, Inverse aerodynamic design method for aircraft components, J. Aircraft 24(2) (1987), 8-9; Santos, A hybrid optimization method for aerodynamic design of lifting surfaces, PhD Thesis, Georgia Institute of Technology, 1993). Owing to these characteristics, the method has been the subject of several authors over the years (G.S. Dulikravich and D.P. Baker, Aerodynamic shape inverse design using a Fourier series method, in AIAA paper 99-0185, AIAA Aerospace Sciences Meeting, Reno, NV, January 1999; D.H. Silva and L.N. Sankar, An inverse method for the design of transonic wings, in 1992 Aerospace Design Conference, No. 92-1025 in proceedings, AIAA, Irvine, CA, February 1992, 1-11; W. Bartelheimer, An Improved Integral Equation Method for the Design of Transonic Airfoils and Wings, AIAA Inc., 1995). More recently, a hybrid formulation and a multi-point algorithm were developed on the basis of the original MGM. This article discusses applications of those latest developments for airfoil and wing design. The test cases focus on wing-body aerodynamic interference and shock wave removal applications. The DLR-F6 geometry is picked as the baseline for the analysis.

Inheritance of resistance to Puccinia psidii G. Winter in a eucalyptus interspecific hybrid progeny evaluated under conditions of natural infection

Inheritance of resistance to Puccinia psidii G. Winter in a eucalyptus interspecific hybrid progeny evaluated under conditions of natural infection Rust caused by the fungus Puccinia psidii is currently the most important disease of eucalyptus. It is widely disseminated in Brazil, and causes serious damage in nurseries and plantation areas. The identification of resistant germplasm along with knowledge of the genetic basis of resistance heredity are the first requirements for the success of breeding programs aiming to develop resistant varieties. Earlier studies carried out under controlled conditions suggested a monogenic control as well as the participation of at least two genes promoting resistance to rust. The goal of this study was to evaluate the resistance to P. psidii under field conditions in fourteen progenies from controlled crosses and self-crosses among four hybrid clones of Eucalyptus grandis Hill ex Maiden x Eucalyptus urophylla ST Blake that contrast for resistance to the fungus. Results indicated that resistance could be explained by one locus with main effects and at least three different alleles. However, loci with minor effects may influence the resistance, since variation on severity classes was observed. Differences in segregation of resistance between reciprocal crosses were not observed, indicating absence of cytoplasmic effects.

Water use and crop coefficient for hybrid DKB 390

Water use and crop coefficient for hybrid DKB 390. This work aims to characterize the water use of maize hybrid DKB 390 under suitable conditions of irrigation for both sufficient and below-optimal situations of nitrogen supply. Crop coefficient values for different stages are also presented as a result, in order to provide the basis for crop water budget and management throughout the cycle. A field experiment was carried Out during the main season, in which biomass, soil moisture, leaf area, climate data and light transmittance were evaluated. These have allowed deriving water balance, use and efficiency. The mentioned genotype requires around 600 nun for high yield targets, being less efficient when led under below-optimal nitrogen fertilization.

Local adaptation and species segregation in two mussel (Mytilus edulis X M. trossulus) hybrid zones

Few marine hybrid zones have been studied extensively, the major exception being the hybrid zone between the mussels Mytilus edulis and M. galloprovincialis in southwestern Europe. Here, we focus on two less studied hybrid zones that also involve Mytilus spp.; M. edulis and M. trossulus are sympatric and hybridize on both western and eastern coasts of the Atlantic Ocean. We review the dynamics of hybridization in these two hybrid zones and evaluate the role of local adaptation for maintaining species boundaries. In Scandinavia, hybridization and gene introgression is so extensive that no individuals with pure M. trossulus genotypes have been found. However, M. trossulus alleles are maintained at high frequencies in the extremely low salinity Baltic Sea for some allozyme genes. A synthesis of reciprocal transplantation experiments between different salinity regimes shows that unlinked Gpi and Pgm alleles change frequency following transplantation, such that post-transplantation allelic composition resembles native populations found in the same salinity. These experiments provide strong evidence for salinity adaptation at Gpi and Pgm (or genes linked to them). In the Canadian Maritimes, pure M. edulis and M. trossulus individuals are abundant, and limited data suggest that M. edulis predominates in low salinity and sheltered conditions, whereas M. trossulus are more abundant on the wave-exposed open coasts. We suggest that these conflicting patterns of species segregation are, in part, caused by local adaptation of Scandinavian M. trossulus to the extremely low salinity Baltic Sea environment.

A hybrid probabilistic criterion for market-based transmission expansion planning

Market-based transmission expansion planning gives information to investors on where is the most cost efficient place to invest and brings benefits to those who invest in this grid. However, both market issue and power system adequacy problems are system planers’ concern. In this paper, a hybrid probabilistic criterion of Expected Economical Loss (EEL) is proposed as an index to evaluate the systems’ overall expected economical losses during system operation in a competitive market. It stands on both investors’ and planner’s point of view and will further improves the traditional reliability cost. By applying EEL, it is possible for system planners to obtain a clear idea regarding the transmission network’s bottleneck and the amount of losses arises from this weak point. Sequentially, it enables planners to assess the worth of providing reliable services. Also, the EEL will contain valuable information for moneymen to undertake their investment. This index could truly reflect the random behaviors of power systems and uncertainties from electricity market. The performance of the EEL index is enhanced by applying Normalized Coefficient of Probability (NCP), so it can be utilized in large real power systems. A numerical example is carried out on IEEE Reliability Test System (RTS), which will show how the EEL can predict the current system bottleneck under future operational conditions and how to use EEL as one of planning objectives to determine future optimal plans. A well-known simulation method, Monte Carlo simulation, is employed to achieve the probabilistic characteristic of electricity market and Genetic Algorithms (GAs) is used as a multi-objective optimization tool.