A tutorial survey of reinforcement learning

This paper gives a compact, self-contained tutorial survey of reinforcement learning, a tool that is increasingly finding application in the development of intelligent dynamic systems. Research on reinforcement learning during the past decade has led to the development of a variety of useful algorithms. This paper surveys the literature and presents the algorithms in a cohesive framework.

Doubly spectral stochastic finite element method (DSSFEM) for random field problems

Uncertainties in complex dynamic systems play an important role in the prediction of a dynamic response in the mid- and high-frequency ranges. For distributed parameter systems, parametric uncertainties can be represented by random fields leading to stochastic partial differential equations. Over the past two decades, the spectral stochastic finite-element method has been developed to discretize the random fields and solve such problems. On the other hand, for deterministic distributed parameter linear dynamic systems, the spectral finite-element method has been developed to efficiently solve the problem in the frequency domain. In spite of the fact that both approaches use spectral decomposition (one for the random fields and the other for the dynamic displacement fields), very little overlap between them has been reported in literature. In this paper, these two spectral techniques are unified with the aim that the unified approach would outperform any of the spectral methods considered on their own. An exponential autocorrelation function for the random fields, a frequency-dependent stochastic element stiffness, and mass matrices are derived for the axial and bending vibration of rods. Closed-form exact expressions are derived by using the Karhunen-Loève expansion. Numerical examples are given to illustrate the unified spectral approach.

Error, Confidence, and Cost in Engineering Computations

The questions that one should answer in engineering computations - deterministic, probabilistic/randomized, as well as heuristic - are (i) how good the computed results/outputs are and (ii) how much the cost in terms of amount of computation and the amount of storage utilized in getting the outputs is. The absolutely errorfree quantities as well as the completely errorless computations done in a natural process can never be captured by any means that we have at our disposal. While the computations including the input real quantities in nature/natural processes are exact, all the computations that we do using a digital computer or are carried out in an embedded form are never exact. The input data for such computations are also never exact because any measuring instrument has inherent error of a fixed order associated with it and this error, as a matter of hypothesis and not as a matter of assumption, is not less than 0.005 per cent. Here by error we imply relative error bounds. The fact that exact error is never known under any circumstances and any context implies that the term error is nothing but error-bounds. Further, in engineering computations, it is the relative error or, equivalently, the relative error-bounds (and not the absolute error) which is supremely important in providing us the information regarding the quality of the results/outputs. Another important fact is that inconsistency and/or near-consistency in nature, i.e., in problems created from nature is completely nonexistent while in our modelling of the natural problems we may introduce inconsistency or near-inconsistency due to human error or due to inherent non-removable error associated with any measuring device or due to assumptions introduced to make the problem solvable or more easily solvable in practice. Thus if we discover any inconsistency or possibly any near-inconsistency in a mathematical model, it is certainly due to any or all of the three foregoing factors. We do, however, go ahead to solve such inconsistent/near-consistent problems and do get results that could be useful in real-world situations. The talk considers several deterministic, probabilistic, and heuristic algorithms in numerical optimisation, other numerical and statistical computations, and in PAC (probably approximately correct) learning models. It highlights the quality of the results/outputs through specifying relative error-bounds along with the associated confidence level, and the cost, viz., amount of computations and that of storage through complexity. It points out the limitation in error-free computations (wherever possible, i.e., where the number of arithmetic operations is finite and is known a priori) as well as in the usage of interval arithmetic. Further, the interdependence among the error, the confidence, and the cost is discussed.

Amplitude death in complex networks induced by environment

We present a mechanism for amplitude death in coupled nonlinear dynamical systems on a complex network having interactions with a common environment like external system. We develop a general stability analysis that is valid for any network topology and obtain the threshold values of coupling constants for the onset of amplitude death. An important outcome of our study is a universal relation between the critical coupling strength and the largest nonzero eigenvalue of the coupling matrix. Our results are fully supported by the detailed numerical analysis for different network topologies.

Generalized State Estimation and Model Predictive Guidance for Spiraling and Ballistic Targets

An extended Kalman filter based generalized state estimation approach is presented in this paper for accurately estimating the states of incoming high-speed targets such as ballistic missiles. A key advantage of this nine-state problem formulation is that it is very much generic and can capture spiraling as well as pure ballistic motion of targets without any change of the target model and the tuning parameters. A new nonlinear model predictive zero-effort-miss based guidance algorithm is also presented in this paper, in which both the zero-effort-miss as well as the time-to-go are predicted more accurately by first propagating the nonlinear target model (with estimated states) and zero-effort interceptor model simultaneously. This information is then used for computing the necessary lateral acceleration. Extensive six-degrees-of-freedom simulation experiments, which include noisy seeker measurements, a nonlinear dynamic inversion based autopilot for the interceptor along with appropriate actuator and sensor models and magnitude and rate saturation limits for the fin deflections, show that near-zero miss distance (i.e., hit-to-kill level performance) can be obtained when these two new techniques are applied together. Comparison studies with an augmented proportional navigation based guidance shows that the proposed model predictive guidance leads to a substantial amount of conservation in the control energy as well.

Fuel-optimal G-MPSP Guidance for Powered Descent Phase of Soft Lunar Landing

A fuel optimal nonlinear sub-optimal guidance scheme is presented in this paper for soft landing of a lunar craft during the powered descent phase. The recently developed Generalized Model Predictive Static Programming (G-MPSP) is used to compute the required magnitude and angle of the thrust vector. Both terminal position and velocity vector are imposed as hard constraints, which ensures high position accuracy and facilitates initiation of vertical descent at the end of the powered descent phase. A key feature of the G-MPSP algorithm is that it converts the nonlinear dynamic programming problem into a low-dimensional static optimization problem (of the same dimension as the output vector). The control history update is done in closed form after computing a time-varying weighting matrix through a backward integration process. This feature makes the algorithm computationally efficient, which makes it suitable for on-board applications. The effectiveness of the proposed guidance algorithm is demonstrated through promising simulation results.

Small-Signal Stability Analysis of an Open-Loop Induction Motor Drive Including the Effect of Inverter Deadtime

This paper demonstrates light-load instability in a 100-kW open-loop induction motor drive on account of inverter deadtime. An improved small-signal model of an inverter-fed induction motor is proposed. This improved model is derived by linearizing the nonlinear dynamic equations of the motor, which include the inverter deadtime effect. Stability analysis is carried out on the 100-kW415-V three-phase induction motor considering no load. The analysis brings out the region of instability of this motor drive on the voltage versus frequency (V-f) plane. This region of light-load instability is found to expand with increase in inverter deadtime. Subharmonic oscillations of significant amplitude are observed in the steady-state simulated and measured current waveforms, at numerous operating points in the unstable region predicted, confirming the validity of the stability analysis. Furthermore, simulation and experimental results demonstrate that the proposed model is more accurate than an existing small-signal model in predicting the region of instability.

Governing Equations and Numerical Solutions of Tension Leg Platform with Finite Amplitude Motion

It is demonstrated that when tension leg platform (TLP) moves with finite amplitude in waves, the inertia force, the drag force and the buoyancy acting on the platform are nonlinear functions of the response of TLP. The tensions of the tethers are also nonlinear functions of the displacement of TLP. Then the displacement, the velocity and the acceleration of TLP should be taken into account when loads are calculated. In addition, equations of motions should be set up on the instantaneous position. A theoretical model for analyzing the nonlinear behavior of a TLP with finite displacement is developed, in which multifold nonlinearities are taken into account, i.e., finite displacement, coupling of the six degrees of freedom, instantaneous position, instantaneous wet surface, free surface effects and viscous drag force. Based on the theoretical model, the comprehensive nonlinear differential equations are deduced. Then the nonlinear dynamic analysis of ISSC TLP in regular waves is performed in the time domain. The degenerative linear solution of the proposed nonlinear model is verified with existing published one. Furthermore, numerical results are presented, which illustrate that nonlinearities exert a significant influence on the dynamic responses of the TLP.

Relationship between strain localization and catastrophic rupture

In order to explore a prior warning to catastrophic rupture of heterogeneous media, like rocks, the present study investigates the relationship between surface strain localization and catastrophic rupture. Instrumented observations on the evolution of surface strain field and the catastrophic rupture of a rock under uniaxial compression were carried out. It is found that the evolution of surface strain field displays two phases: at the early stage, the strain field keeps nearly uniform with weak fluctuations increasing slowly; but at the stage prior to catastrophic rupture, a certain accelerating localization develops and a localized zone emerges. Based on the measurements, an analysis was performed with local mean-field approximation. More importantly, it is found that the scale of localized zone is closely related to the catastrophic rupture strain and the rupture strain can be calculated in accord with the local-mean-field model satisfactorily. This provides a possible clue to the forecast of catastrophic rupture. (c) 2007 Elsevier Ltd. All rights reserved.

Predicting chaotic time-series with wavelet networks

A new technique, wavelet network, is introduced to predict chaotic time series. By using this technique, firstly, we make accurate short-term predictions of the time series from chaotic attractors. Secondly, we make accurate predictions of the values and bifurcation structures of the time series from dynamical systems whose parameter values are changing with time. Finally we predict chaotic attractors by making long-term predictions based on remarkably few data points, where the correlation dimensions of predicted attractors are calculated and are found to be almost identical to those of actual attractors.

Doubts and Equilibria

In real life strategic interactions, decision-makers are likely to entertain doubts about the degree of optimality of their play. To capture this feature of real choice-making, we present here a model based on the doubts felt by an agent about how well is playing a game. The doubts are coupled with (and mutually reinforced by) imperfect discrimination capacity, which we model here by means of similarity relations. We assume that each agent builds procedural preferences de ned on the space of expected payoffs-strategy frequencies attached to his current strategy. These preferences, together with an adaptive learning process lead to doubt-based selection dynamic systems. We introduce the concepts of Mixed Strategy Doubt Equilibria, Mixed Strategy Doubt-Full Equilibria and Mixed Strategy Doubtless Equilibria and show the theoretical and the empirical relevance of these concepts.

海洋平台张力腿非线性动力响应

采用非线性梁模型,充分考虑张力腿轴向与流向的耦合效应及平台运动对张力腿的作用,运用Galerkin法进行分析求解,研究不同的长径比(L/D)情况下,弯曲刚度和非线性耦合效应的影响,以期得到更适用于实际的张力腿简化模型.

有限位移下张力腿平台的非线性动力响应

与一阶无限小位移情况不同,张力腿平台(TLP)发生有限位移时,所受外力与响应耦合,运动方程也必须在瞬时位置建立.建立了有限位移情况下张力腿平台非线性动力响应分析模型,其中考虑了由六自由度有限位移引起的多种非线性因素,如各自由度之间的耦合、瞬时位置、瞬时湿表面等;还包括自由表面效应、粘性力等因素引起的非线性.推导出张力腿平台六自由度有限运动非线性控制方程.对一个名为“ISSC TLP”的典型张力腿平台进行了数值计算,求得该平台在规则波作用下的六自由度运动响应.用退化到线性范围的解与已有解进行了对比,吻合良好.数值结果表明,综合考虑非线性因素后响应有明显改变

Parametric studies of tension leg platform with large amplitude motions

Tension leg platform (TLP) is an important kind of working station for deep water exploration and development in ocean, whose dynamic responses deserve a serious thought. It is shown that for severe sea state, the effects of nonlinearities induced by large displacements of TLP may be noteworthy, and then employment of small displacements model should be restrained. In such situation, large amplitude motion model may be an appropriate alternative. The numerical experiments are performed to study the differences of dynamic responses between the two models. It is shown that for most cases, differences between results of the two models are significant. The variances of the differences vs. the wave period are the most remarkable, and that of the differences vs. wave heading angle are also apparent.