Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

Generalization of the Y-differential autotransformer for 12- and 18-pulse converters

This work proposes a methodology to generalize the Y-connections for 12- and 18-pulse autotransformers. A single mathematical expression, obtained through simple trigonometric operations, represents all the connections. The proposed methodology allows choosing any ratio between the input and the output voltages. The converters can operate either as step-up or as step-down voltage. To simplify the design of the windings, graphics are generated to calculate the turn-ratio and the polarity of each secondary winding, with respect to the primary winding. A design example, followed by digital simulations, illustrates the presented steps. Experimental results of two prototypes (12 and 18 pulses) are presented. The results also show that high power factor is an inherent characteristic of multi-pulse converters, without any active or passive power factor pre-regulators needs. (c) 2005 Elsevier B.V. All rights reserved.

ALGORITHM FOR PROBABILISTIC ANALYSIS of DISTRIBUTION SYSTEMS WITH DISTRIBUTED GENERATION

In this paper an efficient algorithm for probabilistic analysis of unbalanced three-phase weakly-meshed distribution systems is presented. This algorithm uses the technique of Two-Point Estimate Method for calculating the probabilistic behavior of the system random variables. Additionally, the deterministic analysis of the state variables is performed by means of a Compensation-Based Radial Load Flow (CBRLF). Such load flow efficiently exploits the topological characteristics of the network. To deal with distributed generation, a strategy to incorporate a simplified model of a generator in the CBRLF is proposed. Thus, depending on the type of control and generator operation conditions, the node with distributed generation can be modeled either as a PV or PQ node. To validate the efficiency of the proposed algorithm, the IEEE 37 bus test system is used. The probabilistic results are compared with those obtained using the Monte Carlo method.

Lie symmetry analysis and reductions of a two-dimensional integrable generalization of the Camassa-Holm equation

In this Letter we investigate Lie symmetries of a (2 + 1)-dimensional integrable generalization of the Camassa-Holm (CH) equation. Through the similarity reductions we obtain four different (1 + 1)-dimensional systems of partial differential equations in which one of them turns out to be a (1 + 1)-dimensional CH equation. We establish their integrability by providing the Lax pair for all of them. Further, we present a brief analysis for some types of particular solutions which include the cuspon, peakon and soliton solutions for the two-dimensional generalization of the CH equation. (C) 2000 Published by Elsevier B.V. B.V.

Generalization of the Hamilton-Jacobi approach for higher-order singular systems

We generalize the Hamilton-Jacobi formulation for higher-order singular systems and obtain the equations of motion as total differential equations. To do this we first study the constraints structure present in such systems.

Generalization of the delta-differential autotransformer for 12 and 18-pulse converters

This work proposes a methodology to generalize the A-connections for 12 and 18-pulse autotransformers. A single mathematical expression, obtained through simple trigonometric operations, represents all the connections. The proposed methodology allows choosing any ratio between the input and the output voltages. The converters can operate either as step-up or as step-down voltage. To simplify the design of the windings, graphics are generated to calculate the turn-ratio and the polarity of each secondary winding, with respect to the primary winding. A design example, followed by digital simulations, and experimental results illustrate the presented steps. The results also show that high power factor is an inherent characteristic of multi-pulse converters, without any active or passive power factor pre-regulators needs.

A probabilistic model for elastic-plastic fracture analysis of plane and axisymmetric structures

This work presents a methodology for elastic-plastic fracture reliability analysis of plane and axisymmetric structures. The structural reliability analysis is accomplished by means of the FORM analytical method. The virtual crack extension technique based on a direct minimization of potencial energy is utililized for the calculation of the energy release rate. Results are presented to illustrate the performance of the adopted methodology.

Simple generalization of Wisdom's perturbative method

A simple generalization of Wisdom's perturbative method, as originally proposed by Wisdom (1985), is obtained. Any number of resonant cosines can be handled and the method can also accommodate more involved disturbing functions. Averaged trajectories are easily obtained by drawing level curves of the action. Here, the method is first tested for simple models of 3:1 and 2:1 resonant problems. Comparisons with numerical integration and surface-section curves show very good agreements.

Supersymmetric and shape-invariant generalization for nonresonant and intensity-dependent Jaynes-Cummings systems

A class of shape-invariant bound-state problems which represent transitions in a two-level system introduced earlier are generalized to include arbitrary energy splittings between the two levels as well as intensity-dependent interactions. We show that the coupled-channel Hamiltonians obtained correspond to the generalizations of the nonresonant and intensity-dependent Jaynes-Cummings Hamiltonians, widely used in quantized theories of lasers. In this general context, we determine the eigenstates, eigenvalues, the time evolution matrix and the population inversion matrix factor.

Microscopic generalization of the standard interacting boson model Hamiltonian in the restricted dynamics approach

The evaluation of the microscopic generalized interacting boson model (GIBM) Hamiltonian, deduced from the general microscopic nuclear Hamiltonian via the collective O-A-1 invariant microscopic Hamiltonian of the general restricted dynamics model (RDM) in the case of central multipole and multipole-Gauss type effective NN-potential is briefly discussed. The GIBM version, which includes all sixth-order terms in the expansion of the collective part of the NN-potential, has been obtained. This GIBM Hamiltonian contains additional terms compared with the standard (sd-boson) interacting boson model (IBM). The microscopic expressions for the standard IBM Hamiltonian parameters in terms of the employed effective NN-potential parameters have also been obtained.

A Family of Autoconnected Transformers for 12-and 18-Pulse Converters-Generalization for Delta and Wye Topologies

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Probabilistic neural network to predict cracks in taphole mud used in blast furnaces

One of the major problems facing Blast Furnaces is the occurrence of cracks in taphole mud, as the underlying causes are not easily identifiable. The absence of this knowledge makes it difficult the use of conventional techniques for predictability and mitigation. This paper will address the application of Probabilistic Neural Network using the Matlab software as a means to detect and control such cracks. The most relevant BF operational variables were picked through the statistic tool "Principal Component Analysis - PCA." Based upon the selection of these variables a probabilistic neural network was built. A set of BF operational data, consisting of 30 controlling variables, was divided into 2 groups, one of which for network training, and the other one to validate the neural network. The neural network got 98% of the cases right. The results show the effectiveness of this tool for crack prediction in relation to clay intrinsic properties and as a result of the fluctuation in operational variables.

Two-dimensional integrable generalization of the Camassa-Holm equation

In this letter we discuss the (2 + 1)-dimensional generalization of the Camassa-Holm equation. We require that this generalization be, at the same time, integrable and physically derivable under the same asymptotic analysis as the original Camassa-Holm equation. First, we find the equation in a perturbative calculation in shallow-water theory. We then demonstrate its integrability and find several particular solutions describing (2 + 1) solitary-wave like solutions. © 1999 Published by Elsevier Science B.V. All rights reserved.

Variational studies and replica symmetry breaking in the generalization problem of the binary perceptron

We analyze the average performance of a general class of learning algorithms for the nondeterministic polynomial time complete problem of rule extraction by a binary perceptron. The examples are generated by a rule implemented by a teacher network of similar architecture. A variational approach is used in trying to identify the potential energy that leads to the largest generalization in the thermodynamic limit. We restrict our search to algorithms that always satisfy the binary constraints. A replica symmetric ansatz leads to a learning algorithm which presents a phase transition in violation of an information theoretical bound. Stability analysis shows that this is due to a failure of the replica symmetric ansatz and the first step of replica symmetry breaking (RSB) is studied. The variational method does not determine a unique potential but it allows construction of a class with a unique minimum within each first order valley. Members of this class improve on the performance of Gibbs algorithm but fail to reach the Bayesian limit in the low generalization phase. They even fail to reach the performance of the best binary, an optimal clipping of the barycenter of version space. We find a trade-off between a good low performance and early onset of perfect generalization. Although the RSB may be locally stable we discuss the possibility that it fails to be the correct saddle point globally. ©2000 The American Physical Society.