Diffusion Gradient Temporal Difference for Cooperative Reinforcement Learning with Linear Function Approximation

We introduce a diffusion-based algorithm in which multiple agents cooperate to predict a common and global statevalue function by sharing local estimates and local gradient information among neighbors. Our algorithm is a fully distributed implementation of the gradient temporal difference with linear function approximation, to make it applicable to multiagent settings. Simulations illustrate the benefit of cooperation in learning, as made possible by the proposed algorithm.

A unified framework for linear function approximation of value functions in stochastic control

This paper contributes with a unified formulation that merges previ- ous analysis on the prediction of the performance ( value function ) of certain sequence of actions ( policy ) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approxi- mated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the pro- posed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.

Optimal Piecewise Linear Function Approximation for GPU-based Applications

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of this kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the trade-off between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern Graphics Processing Units (GPUs), where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions, we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

New methods for the estimation of Takagi-Sugeno model based extended Kalman filter and its applications to optimal control for nonlinear systems

This paper describes new approaches to improve the local and global approximation (matching) and modeling capability of Takagi–Sugeno (T-S) fuzzy model. The main aim is obtaining high function approximation accuracy and fast convergence. The main problem encountered is that T-S identification method cannot be applied when the membership functions are overlapped by pairs. This restricts the application of the T-S method because this type of membership function has been widely used during the last 2 decades in the stability, controller design of fuzzy systems and is popular in industrial control applications. The approach developed here can be considered as a generalized version of T-S identification method with optimized performance in approximating nonlinear functions. We propose a noniterative method through weighting of parameters approach and an iterative algorithm by applying the extended Kalman filter, based on the same idea of parameters’ weighting. We show that the Kalman filter is an effective tool in the identification of T-S fuzzy model. A fuzzy controller based linear quadratic regulator is proposed in order to show the effectiveness of the estimation method developed here in control applications. An illustrative example of an inverted pendulum is chosen to evaluate the robustness and remarkable performance of the proposed method locally and globally in comparison with the original T-S model. Simulation results indicate the potential, simplicity, and generality of the algorithm. An illustrative example is chosen to evaluate the robustness. In this paper, we prove that these algorithms converge very fast, thereby making them very practical to use.

A new approach to fuzzy estimation of Takagi-Sugeno model and its applications to optimal control for nonlinear systems

An efficient approach is presented to improve the local and global approximation and modelling capability of Takagi-Sugeno (T-S) fuzzy model. The main aim is obtaining high function approximation accuracy. The main problem is that T-S identification method cannot be applied when the membership functions are overlapped by pairs. This restricts the use of the T-S method because this type of membership function has been widely used during the last two decades in the stability, controller design and are popular in industrial control applications. The approach developed here can be considered as a generalized version of T-S method with optimized performance in approximating nonlinear functions. A simple approach with few computational effort, based on the well known parameters' weighting method is suggested for tuning T-S parameters to improve the choice of the performance index and minimize it. A global fuzzy controller (FC) based Linear Quadratic Regulator (LQR) is proposed in order to show the effectiveness of the estimation method developed here in control applications. Illustrative examples of an inverted pendulum and Van der Pol system are chosen to evaluate the robustness and remarkable performance of the proposed method and the high accuracy obtained in approximating nonlinear and unstable systems locally and globally in comparison with the original T-S model. Simulation results indicate the potential, simplicity and generality of the algorithm.

Jasmonate-dependent modifications of the pectin matrix during potato development function as a defense mechanism targeted by Dickeya dadantii virulence factors

The plant cell wall constitutes an essential protection barrier against pathogen attack. In addition, cell-wall disruption leads to accumulation of jasmonates (JAs), which are key signaling molecules for activation of plant inducible defense responses. However, whether JAs in return modulate the cell-wall composition to reinforce this defensive barrier remains unknown. The enzyme 13-allene oxide synthase (13-AOS) catalyzes the first committed step towards biosynthesis of JAs. In potato (Solanum tuberosum), there are two putative St13-AOS genes, which we show here to be differentially induced upon wounding. We also determine that both genes complement an Arabidopsis aos null mutant, indicating that they encode functional 13-AOS enzymes. Indeed, transgenic potato plants lacking both St13-AOS genes (CoAOS1/2 lines) exhibited a significant reduction of JAs, a concomitant decrease in wound-responsive gene activation, and an increased severity of soft rot disease symptoms caused by Dickeya dadantii. Intriguingly, a hypovirulent D. dadantii pel strain lacking the five major pectate lyases, which causes limited tissue maceration on wild-type plants, regained infectivity in CoAOS1/2 plants. In line with this, we found differences in pectin methyl esterase activity and cell-wall pectin composition between wild-type and CoAOS1/2 plants. Importantly, wild-type plants had pectins with a lower degree of methyl esterification, which are the substrates of the pectate lyases mutated in the pel strain. These results suggest that, during development of potato plants, JAs mediate modification of the pectin matrix to form a defensive barrier that is counteracted by pectinolytic virulence factors from D. dadantii.

A matrix pencil approach to the existence of compactly supported reconstruction functions in average sampling

The aim of this work is to solve a question raised for average sampling in shift-invariant spaces by using the well-known matrix pencil theory. In many common situations in sampling theory, the available data are samples of some convolution operator acting on the function itself: this leads to the problem of average sampling, also known as generalized sampling. In this paper we deal with the existence of a sampling formula involving these samples and having reconstruction functions with compact support. Thus, low computational complexity is involved and truncation errors are avoided. In practice, it is accomplished by means of a FIR filter bank. An answer is given in the light of the generalized sampling theory by using the oversampling technique: more samples than strictly necessary are used. The original problem reduces to finding a polynomial left inverse of a polynomial matrix intimately related to the sampling problem which, for a suitable choice of the sampling period, becomes a matrix pencil. This matrix pencil approach allows us to obtain a practical method for computing the compactly supported reconstruction functions for the important case where the oversampling rate is minimum. Moreover, the optimality of the obtained solution is established.

Computing the Hessenberg matrix associated with a self-similar measure

We introduce in this paper a method to calculate the Hessenberg matrix of a sum of measures from the Hessenberg matrices of the component measures. Our method extends the spectral techniques used by G. Mantica to calculate the Jacobi matrix associated with a sum of measures from the Jacobi matrices of each of the measures. We apply this method to approximate the Hessenberg matrix associated with a self-similar measure and compare it with the result obtained by a former method for self-similar measures which uses a fixed point theorem for moment matrices. Results are given for a series of classical examples of self-similar measures. Finally, we also apply the method introduced in this paper to some examples of sums of (not self-similar) measures obtaining the exact value of the sections of the Hessenberg matrix.

Structure function and Multifractal spectrum applied to Digital Elevation Model

A Digital Elevation Model (DEM) provides the information basis used for many geographic applications such as topographic and geomorphologic studies, landscape through GIS (Geographic Information Systems) among others. The DEM capacity to represent Earth?s surface depends on the surface roughness and the resolution used. Each DEM pixel depends on the scale used characterized by two variables: resolution and extension of the area studied. DEMs can vary in resolution and accuracy by the production method, although there are statistical characteristics that keep constant or very similar in a wide range of scales. Based on this property, several techniques have been applied to characterize DEM through multiscale analysis directly related to fractal geometry: multifractal spectrum and the structure function. The comparison of the results by both methods is discussed. The study area is represented by a 1024 x 1024 data matrix obtained from a DEM with a resolution of 10 x 10 m each point, which correspond with a region known as ?Monte de El Pardo? a property of Spanish National Heritage (Patrimonio Nacional Español) of 15820 Ha located to a short distance from the center of Madrid. Manzanares River goes through this area from North to South. In the southern area a reservoir is found with a capacity of 43 hm3, with an altitude of 603.3 m till 632 m when it is at the highest capacity. In the middle of the reservoir the minimum altitude of this area is achieved.

Automatic quantification of matrix cracking and fiber rotation by X-ray computed tomography in shear-deformed carbon fiber-reinforced laminates

The deformation and damage mechanisms of carbon fiber-reinforced epoxy laminates deformed in shear were studied by means of X-ray computed tomography. In particular, the evolution of matrix cracking, interply delamination and fiber rotation was ascertained as a function of the applied strain. In order to provide quantitative information, an algorithm was developed to automatically determine the crack density and the fiber orientation from the tomograms. The investigation provided new insights about the complex interaction between the different damage mechanisms (i.e. matrix cracking and interply delamination) as a function of the applied strain, ply thickness and ply location within the laminate as well as quantitative data about the evolution of matrix cracking and fiber rotation during deformation