Diffusion Gradient Temporal Difference for Cooperative Reinforcement Learning with Linear Function Approximation

We introduce a diffusion-based algorithm in which multiple agents cooperate to predict a common and global statevalue function by sharing local estimates and local gradient information among neighbors. Our algorithm is a fully distributed implementation of the gradient temporal difference with linear function approximation, to make it applicable to multiagent settings. Simulations illustrate the benefit of cooperation in learning, as made possible by the proposed algorithm.

Accurate analytical approximation of asteroid deflection with constant tangential thrust

We present analytical formulas to estimate the variation of achieved deflection for an Earth-impacting asteroid following a continuous tangential low-thrust deflection strategy. Relatively simple analytical expressions are obtained with the aid of asymptotic theory and the use of Peláez orbital elements set, an approach that is particularly suitable to the asteroid deflection problem and is not limited to small eccentricities. The accuracy of the proposed formulas is evaluated numerically showing negligible error for both early and late deflection campaigns. The results will be of aid in planning future low-thrust asteroid deflection missions

A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

Quantification of impacts in artistic gymnastics with accelerometry: an approximation

Intensity and volume of training in Artisti Gymnastics are increasing as the sooner athlete's age of incorporation creating some disturbance in them.

Quasi-cylindrical approximation to the swirling flow in an atomizer chamber

A quasi-cylindrical approximation is used to analyse the axisymmetric swirling flow of a liquid with a hollow air core in the chamber of a pressure swirl atomizer. The liquid is injected into the chamber with an azimuthal velocity component through a number of slots at the periphery of one end of the chamber, and flows out as an anular sheet through a central orifice at the other end, following a conical convergence of the chamber wall. An effective inlet condition is used to model the effects of the slots and the boundary layer that develops at the nearby endwall of the chamber. An analysis is presented of the structure of the liquid sheet at the end of the exit orifice, where the flow becomes critical in the sense that upstream propagation of long-wave perturbations ceases to be possible. This nalysis leads to a boundary condition at the end of the orifice that is an extension of the condition of maximum flux used with irrotational models of the flow. As is well known, the radial pressure gradient induced by the swirling flow in the bulk of the chamber causes the overpressure that drives the liquid towards the exit orifice, and also leads to Ekman pumping in the boundary layers of reduced azimuthal velocity at the convergent wall of the chamber and at the wall opposite to the exit orifice. The numerical results confirm the important role played by the boundary layers. They make the thickness of the liquid sheet at the end of the orifice larger than predicted by rrotational models, and at the same time tend to decrease the overpressure required to pass a given flow rate through the chamber, because the large axial velocity in the boundary layers takes care of part of the flow rate. The thickness of the boundary layers increases when the atomizer constant (the inverse of a swirl number, proportional to the flow rate scaled with the radius of the exit orifice and the circulation around the air core) decreases. A minimum value of this parameter is found below which the layer of reduced azimuthal velocity around the air core prevents the pressure from increasing and steadily driving the flow through the exit orifice. The effects of other parameters not accounted for by irrotational models are also analysed in terms of their influence on the boundary layers.

Optimal Piecewise Linear Function Approximation for GPU-based Applications

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of this kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the trade-off between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern Graphics Processing Units (GPUs), where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions, we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

Discrete linear stability and sensitivity analysis of fluid flows

Este trabajo presenta un método discreto para el cálculo de estabilidad hidrodinámica y análisis de sensibilidad a perturbaciones externas para ecuaciones diferenciales y en particular para las ecuaciones de Navier-Stokes compressible. Se utiliza una aproximación con variable compleja para obtener una precisión analítica en la evaluación de la matriz Jacobiana. Además, mapas de sensibilidad para la sensibilidad a las modificaciones del flujo de base y a una fuerza constante permiten identificar las regiones del campo fluido donde una modificacin (ej. fuerza puntual) tiene un efecto estabilizador del flujo. Se presentan cuatro casos de prueba: (1) un caso analítico para comprobar la derivación discreta, (2) una cavidad cerrada a bajo Reynolds para mostrar la mayor precisión en el cálculo de los valores propios con la aproximación de paso complejo, (3) flujo 2D en un cilindro circular para validar la metodología, y (4) flujo en un cavidad abierta, presentado para validar el método en casos de inestabilidades convectivamente inestables. Los tres últimos casos mencionados (2-4) se resolvieron con las ecuaciones de Navier-Stokes compresibles, utilizando un método Discontinuous Galerkin Spectral Element Method. Se obtuvo una buena concordancia para el caso de validación (3), cuando se comparó el nuevo método con resultados de la literatura. Además, este trabajo muestra que para el cálculo de los modos propios directos y adjuntos, así como para los mapas de sensibilidad, el uso de variables complejas es de suprema importancia para obtener una predicción precisa. El método descrito es aplicado al análisis para la estabilización de la estela generada por un disco actuador, que representa un modelo sencillo para hélices, rotores de helicópteros o turbinas eólicas. Se explora la primera bifurcación del flujo para un disco actuador, y se sugiere que está asociada a una inestabilidad de tipo Kelvin-Helmholtz, cuya estabilidad se controla con en el número de Reynolds y en la resistencia del disco actuador (o fuerza resistente). En primer lugar, se verifica que la disminución de la resistencia del disco tiene un efecto estabilizador parecido a una disminución del Reynolds. En segundo lugar, el análisis hidrodinmico discreto identifica dos regiones para la colocación de una fuerza puntual que controle las inestabilidades, una cerca del disco y otra en una zona aguas abajo. En tercer lugar, se muestra que la inclusión de un forzamiento localizado cerca del actuador produce una estabilización más eficiente que al forzar aguas abajo. El análisis de los campos de flujo controlados confirma que modificando el gradiente de velocidad cerca del actuador es más eficiente para estabilizar la estela. Estos resultados podrían proporcionar nuevas directrices para la estabilización de la estela de turbinas de viento o de marea cuando estén instaladas en un parque eólico y minimizar las interacciones no estacionarias entre turbinas. ABSTRACT A discrete framework for computing the global stability and sensitivity analysis to external perturbations for any set of partial differential equations is presented. In particular, a complex-step approximation is used to achieve near analytical accuracy for the evaluation of the Jacobian matrix. Sensitivity maps for the sensitivity to base flow modifications and to a steady force are computed to identify regions of the flow field where an input could have a stabilising effect. Four test cases are presented: (1) an analytical test case to prove the theory of the discrete framework, (2) a lid-driven cavity at low Reynolds case to show the improved accuracy in the calculation of the eigenvalues when using the complex-step approximation, (3) the 2D flow past a circular cylinder at just below the critical Reynolds number is used to validate the methodology, and finally, (4) the flow past an open cavity is presented to give an example of the discrete method applied to a convectively unstable case. The latter three (2–4) of the aforementioned cases were solved with the 2D compressible Navier–Stokes equations using a Discontinuous Galerkin Spectral Element Method. Good agreement was obtained for the validation test case, (3), with appropriate results in the literature. Furthermore, it is shown that for the calculation of the direct and adjoint eigenmodes and their sensitivity maps to external perturbations, the use of complex variables is paramount for obtaining an accurate prediction. An analysis for stabilising the wake past an actuator disc, which represents a simple model for propellers, helicopter rotors or wind turbines is also presented. We explore the first flow bifurcation for an actuator disc and it suggests that it is associated to a Kelvin- Helmholtz type instability whose stability relies on the Reynolds number and the flow resistance applied through the disc (or actuator forcing). First, we report that decreasing the disc resistance has a similar stabilising effect to an decrease in the Reynolds number. Second, a discrete sensitivity analysis identifies two regions for suitable placement of flow control forcing, one close to the disc and one far downstream where the instability originates. Third, we show that adding a localised forcing close to the actuator provides more stabilisation that forcing far downstream. The analysis of the controlled flow fields, confirms that modifying the velocity gradient close to the actuator is more efficient to stabilise the wake than controlling the sheared flow far downstream. An interesting application of these results is to provide guidelines for stabilising the wake of wind or tidal turbines when placed in an energy farm to minimise unsteady interactions.

The segmental approximation in multijunction solar cells

The segmental approach has been considered to analyze dark and light I-V curves. The photovoltaic (PV) dependence of the open-circuit voltage (Voc), the maximum power point voltage (Vm), the efficiency (?) on the photogenerated current (Jg), or on the sunlight concentration ratio (X), are analyzed, as well as other photovoltaic characteristics of multijunction solar cells. The characteristics being analyzed are split into monoexponential (linear in the semilogarithmic scale) portions, each of which is characterized by a definite value of the ideality factor A and preexponential current J0. The monoexponentiality ensures advantages, since at many steps of the analysis, one can use the analytical dependences instead of numerical methods. In this work, an experimental procedure for obtaining the necessary parameters has been proposed, and an analysis of GaInP/GaInAs/Ge triple-junction solar cell characteristics has been carried out. It has been shown that up to the sunlight concentration ratios, at which the efficiency maximum is achieved, the results of calculation of dark and light I-V curves by the segmental method fit well with the experimental data. An important consequence of this work is the feasibility of acquiring the resistanceless dark and light I-V curves, which can be used for obtaining the I-V curves characterizing the losses in the transport part of a solar cell.

A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems

We propose a general procedure for solving incomplete data estimation problems. The procedure can be used to find the maximum likelihood estimate or to solve estimating equations in difficult cases such as estimation with the censored or truncated regression model, the nonlinear structural measurement error model, and the random effects model. The procedure is based on the general principle of stochastic approximation and the Markov chain Monte-Carlo method. Applying the theory on adaptive algorithms, we derive conditions under which the proposed procedure converges. Simulation studies also indicate that the proposed procedure consistently converges to the maximum likelihood estimate for the structural measurement error logistic regression model.

Approximation algorithms

Increasing global competition, rapidly changing markets, and greater consumer awareness have altered the way in which corporations do business. To become more efficient, many industries have sought to model some operational aspects by gigantic optimization problems. It is not atypical to encounter models that capture 106 separate “yes” or “no” decisions to be made. Although one could, in principle, try all 2106 possible solutions to find the optimal one, such a method would be impractically slow. Unfortunately, for most of these models, no algorithms are known that find optimal solutions with reasonable computation times. Typically, industry must rely on solutions of unguaranteed quality that are constructed in an ad hoc manner. Fortunately, for some of these models there are good approximation algorithms: algorithms that produce solutions quickly that are provably close to optimal. Over the past 6 years, there has been a sequence of major breakthroughs in our understanding of the design of approximation algorithms and of limits to obtaining such performance guarantees; this area has been one of the most flourishing areas of discrete mathematics and theoretical computer science.

Motor learning by field approximation.

We investigated how human subjects adapt to forces perturbing the motion of their ams. We found that this kind of learning is based on the capacity of the central nervous system (CNS) to predict and therefore to cancel externally applied perturbing forces. Our experimental results indicate: (i) that the ability of the CNS to compensate for the perturbing forces is restricted to those spatial locations where the perturbations have been experienced by the moving arm. The subjects also are able to compensate for forces experienced at neighboring workspace locations. However, adaptation decays smoothly and quickly with distance from the locations where disturbances had been sensed by the moving limb. (ii) Our experiments also how that the CNS builds an internal model of the external perturbing forces in intrinsic (muscles and / or joints) coordinates.

On the uniform approximation of Cauchy continuous functions

In the context of real-valued functions defined on metric spaces, it is known that the locally Lipschitz functions are uniformly dense in the continuous functions and that the Lipschitz in the small functions - the locally Lipschitz functions where both the local Lipschitz constant and the size of the neighborhood can be chosen independent of the point - are uniformly dense in the uniformly continuous functions. Between these two basic classes of continuous functions lies the class of Cauchy continuous functions, i.e., the functions that map Cauchy sequences in the domain to Cauchy sequences in the target space. Here, we exhibit an intermediate class of Cauchy continuous locally Lipschitz functions that is uniformly dense in the real-valued Cauchy continuous functions. In fact, our result is valid when our target space is an arbitrary Banach space.

Logic-Based Outer Approximation for the Design of Discrete-Continuous Dynamic Systems with Implicit Discontinuities

We address the optimization of discrete-continuous dynamic optimization problems using a disjunctive multistage modeling framework, with implicit discontinuities, which increases the problem complexity since the number of continuous phases and discrete events is not known a-priori. After setting a fixed alternative sequence of modes, we convert the infinite-dimensional continuous mixed-logic dynamic (MLDO) problem into a finite dimensional discretized GDP problem by orthogonal collocation on finite elements. We use the Logic-based Outer Approximation algorithm to fully exploit the structure of the GDP representation of the problem. This modelling framework is illustrated with an optimization problem with implicit discontinuities (diver problem).

Logic-Based Outer-Approximation Algorithm for Solving Discrete-Continuous Dynamic Optimization Problems

We present an extension of the logic outer-approximation algorithm for dealing with disjunctive discrete-continuous optimal control problems whose dynamic behavior is modeled in terms of differential-algebraic equations. Although the proposed algorithm can be applied to a wide variety of discrete-continuous optimal control problems, we are mainly interested in problems where disjunctions are also present. Disjunctions are included to take into account only certain parts of the underlying model which become relevant under some processing conditions. By doing so the numerical robustness of the optimization algorithm improves since those parts of the model that are not active are discarded leading to a reduced size problem and avoiding potential model singularities. We test the proposed algorithm using three examples of different complex dynamic behavior. In all the case studies the number of iterations and the computational effort required to obtain the optimal solutions is modest and the solutions are relatively easy to find.

A Belief System's Organization Based on a Computational Model of the Dynamic Context: First Approximation

In this article we present a model of organization of a belief system based on a set of binary recursive functions that characterize the dynamic context that modifies the beliefs. The initial beliefs are modeled by a set of two-bit words that grow, update, and generate other beliefs as the different experiences of the dynamic context appear. Reason is presented as an emergent effect of the experience on the beliefs. The system presents a layered structure that allows a functional organization of the belief system. Our approach seems suitable to model different ways of thinking and to apply to different realistic scenarios such as ideologies.