Experience generalization for multi-agent reinforcement learning

On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.

Excitations and finite-size properties of an integrable supersymmetric electronic model

Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq)

Generalization of the Y-differential autotransformer for 12- and 18-pulse converters

This work proposes a methodology to generalize the Y-connections for 12- and 18-pulse autotransformers. A single mathematical expression, obtained through simple trigonometric operations, represents all the connections. The proposed methodology allows choosing any ratio between the input and the output voltages. The converters can operate either as step-up or as step-down voltage. To simplify the design of the windings, graphics are generated to calculate the turn-ratio and the polarity of each secondary winding, with respect to the primary winding. A design example, followed by digital simulations, illustrates the presented steps. Experimental results of two prototypes (12 and 18 pulses) are presented. The results also show that high power factor is an inherent characteristic of multi-pulse converters, without any active or passive power factor pre-regulators needs. (c) 2005 Elsevier B.V. All rights reserved.

On asymptotic solutions of integrable wave equations

Asymptotic 'soliton train' solutions of integrable wave equations described by inverse scattering transform method with second-order scalar eigenvalue problem are considered. It is shown that if asymptotic solution can be presented as a modulated one-phase nonlinear periodic wavetrain, then the corresponding Baker-Akhiezer function transforms into quasiclassical eigenfunction of the linear spectral problem in weak dispersion limit for initially smooth pulses. In this quasiclassical limit the corresponding eigenvalues can be calculated with the use of the Bohr Sommerfeld quantization rule. The asymptotic distributions of solitons parameters obtained in this way specify the solution of the Whitham equations. (C) 2001 Elsevier B.V. B.V. All rights reserved.

Integrable hierarchy for multidimensional Toda equations and topological-anti-topological fusion

The negative symmetry flows are incorporated into the Riemann-Hilbert problem for the homogeneous A(m)-hierarchy and its (gl) over cap (m + 1, C) extension.A loop group automorphism of order two is used to define a sub-hierarchy of (gl) over cap (m + 1, C) hierarchy containing only the odd symmetry flows. The positive and negative flows of the +/-1 grade coincide with equations of the multidimensional Toda model and of topological-anti-topological fusion. (C) 2002 Elsevier B.V. B.V. All rights reserved.

Integrable aspects of the scaling q-state Potts models I: bound states and bootstrap closure

We discuss the q-state Potts models for q less than or equal to 4, in the scaling regimes close to their critical or tricritical points. Starting from the kink S-matrix elements proposed by Chim and Zamolodchikov, the bootstrap is closed for the scaling regions of all critical points, and for the tricritical points when 4 > q greater than or equal to 2. We also note a curious appearance of the extended last line of Freudenthal's magic square in connection with the Potts models. (C) 2003 Elsevier B.V. B.V. All rights reserved.

T-duality in 2D integrable models

The non-conformal analogue of Abelian T-duality transformations relating pairs of axial and vector integrable models from the non-Abelian affine Toda family is constructed and studied in detail.

Integrable aspects of the scaling q-state Potts models II: finite-size effects

We continue our discussion of the q-state Potts models for q less than or equal to 4, in the scaling regimes close to their critical and tricritical points. In a previous paper, the spectrum and full S-matrix of the models on an infinite line were elucidated; here, we consider finite-size behaviour. TBA equations are proposed for all cases related to phi(21) and phi(12) perturbations of unitary minimal models. These are subjected to a variety of checks in the ultraviolet and infrared limits, and compared with results from a recently-proposed non-linear integral equation. A non-linear integral equation is also used to study the flows from tricritical to critical models, over the full range of q. Our results should also be of relevance to the study of the off-critical dilute A models in regimes 1 and 2. (C) 2003 Elsevier B.V. B.V. All rights reserved.

The structures underlying soliton solutions in integrable hierarchies

We point out that a common feature of integrable hierarchies presenting soliton solutions is the existence of some special ''vacuum solutions'' such that the Lax operators evaluated on them, lie in some abelian subalgebra of the associated Kac-Moody algebra. The soliton solutions are constructed out of those ''vacuum solitons'' by the dressing transformation procedure.

On the relationship between a 2 x 2 matrix and second-order scalar spectral problems for integrable equations

A simple proof is given that a 2 x 2 matrix scheme for an inverse scattering transform method for integrable equations can be converted into the standard form of the second-order scalar spectral problem associated with the same equations. Simple formulae relating these two kinds of representation of integrable equations are established.

Classical integrable super sinh-Gordon equation with defects

The introduction of defects is discussed under the Lagrangian formalism and Backlund transformations for the N = 1 super sinh-Gordon model. Modified conserved momentum and energy are constructed for this case. Some explicit examples of different Backlund soliton solutions are discussed. The Lax formulation within the space split by the defect leads to the integrability of the model and henceforth to the existence of an infinite number of constants of motion.

Symmetry analysis of an integrable reaction-diffusion equation

In this paper, we investigate the invariance and integrability properties of an integrable two-component reaction-diffusion equation. We perform Painleve analysis for both the reaction-diffusion equation modelled by a coupled nonlinear partial differential equations and its general similarity reduced ordinary differential equation and confirm its integrability. Further, we perform Lie symmetry analysis for this model. Interestingly our investigations reveals a rich variety of particular solutions, which have not been reported in the literature, for this model. (C) 2000 Elsevier B.V. Ltd. All rights reserved.

Integrable field theories with defects

The structure of integrable field theories in the presence of defects is discussed in terms of boundary functions under the Lagrangian formalism. Explicit examples of bosonic and fermionic theories are considered. In particular, the boundary functions for the super sinh-Gordon model is constructed and shown to generate the Backlund transformations for its soliton solutions.

Exact static soliton solutions of (3+1)-dimensional integrable theory with nonzero Hopf numbers

In this paper, we explicitly construct an infinite number of Hopfions (static, soliton solutions with nonzero Hopf topological charges) within the recently proposed (3 + 1)-dimensional, integrable, and relativistically invariant field theory. Two integers label the family of Hopfions we have found. Their product is equal to the Hopf charge which provides a lower bound to the soliton's finite energy. The Hopfions are explicitly constructed in terms of the toroidal coordinates and shown to have a form of linked closed vortices.

Generalization of the Hamilton-Jacobi approach for higher-order singular systems

We generalize the Hamilton-Jacobi formulation for higher-order singular systems and obtain the equations of motion as total differential equations. To do this we first study the constraints structure present in such systems.