Solution of MDPS using simulation-based value iteration

This article proposes a three-timescale simulation based algorithm for solution of infinite horizon Markov Decision Processes (MDPs). We assume a finite state space and discounted cost criterion and adopt the value iteration approach. An approximation of the Dynamic Programming operator T is applied to the value function iterates. This 'approximate' operator is implemented using three timescales, the slowest of which updates the value function iterates. On the middle timescale we perform a gradient search over the feasible action set of each state using Simultaneous Perturbation Stochastic Approximation (SPSA) gradient estimates, thus finding the minimizing action in T. On the fastest timescale, the 'critic' estimates, over which the gradient search is performed, are obtained. A sketch of convergence explaining the dynamics of the algorithm using associated ODEs is also presented. Numerical experiments on rate based flow control on a bottleneck node using a continuous-time queueing model are performed using the proposed algorithm. The results obtained are verified against classical value iteration where the feasible set is suitably discretized. Over such a discretized setting, a variant of the algorithm of [12] is compared and the proposed algorithm is found to converge faster.

Denumerable state stochastic games with limiting average payoff

We study stochastic games with countable state space, compact action spaces, and limiting average payoff. ForN-person games, the existence of an equilibrium in stationary strategies is established under a certain Liapunov stability condition. For two-person zero-sum games, the existence of a value and optimal strategies for both players are established under the same stability condition.

Analysis and design of split-capacitor resistive-inductive passive damping for LCL filters in grid-connected inverters

Grid-connected inverters require a third-order LCL filter to meet standards such as the IEEE Std. 519-1992 while being compact and cost-effective. LCL filter introduces resonance, which needs to be damped through active or passive methods. Passive damping schemes have less control complexity and are more reliable. This study explores the split-capacitor resistive-inductive (SC-RL) passive damping scheme. The SC-RL damped LCL filter is modelled using state space approach. Using this model, the power loss and damping are analysed. Based on the analysis, the SC-RL scheme is shown to have lower losses than other simpler passive damping methods. This makes the SC-RL scheme suitable for high power applications. A method for component selection that minimises the power loss in the damping resistors while keeping the system well damped is proposed. The design selection takes into account the influence of switching frequency, resonance frequency and the choice of inductance and capacitance values of the filter on the damping component selection. The use of normalised parameters makes it suitable for a wide range of design applications. Analytical results show the losses and quality factor to be in the range of 0.05-0.1% and 2.0-2.5, respectively, which are validated experimentally.

Accurate Steady-State Representation of a Doubly Fed Induction Machine

This letter presents an accurate steady-state phasor model for a doubly fed induction machine. The drawback of existing steady-state phasor model is discussed. In particular, the inconsistency of existing equivalent model with respect to reactive power flows when operated at supersynchronous speeds is highlighted. Relevant mathematical basis for the proposed model is presented and its validity is illustrated on a 2-MW doubly fed induction machine.

Models and algorithms for tracking of maneuvering objects using variable rate particle filters

Standard algorithms in tracking and other state-space models assume identical and synchronous sampling rates for the state and measurement processes. However, real trajectories of objects are typically characterized by prolonged smooth sections, with sharp, but infrequent, changes. Thus, a more parsimonious representation of a target trajectory may be obtained by direct modeling of maneuver times in the state process, independently from the observation times. This is achieved by assuming the state arrival times to follow a random process, typically specified as Markovian, so that state points may be allocated along the trajectory according to the degree of variation observed. The resulting variable dimension state inference problem is solved by developing an efficient variable rate particle filtering algorithm to recursively update the posterior distribution of the state sequence as new data becomes available. The methodology is quite general and can be applied across many models where dynamic model uncertainty occurs on-line. Specific models are proposed for the dynamics of a moving object under internal forcing, expressed in terms of the intrinsic dynamics of the object. The performance of the algorithms with these dynamical models is demonstrated on several challenging maneuvering target tracking problems in clutter. © 2006 IEEE.

Control of dynamical systems with temporal logic specifications

This thesis is motivated by safety-critical applications involving autonomous air, ground, and space vehicles carrying out complex tasks in uncertain and adversarial environments. We use temporal logic as a language to formally specify complex tasks and system properties. Temporal logic specifications generalize the classical notions of stability and reachability that are studied in the control and hybrid systems communities. Given a system model and a formal task specification, the goal is to automatically synthesize a control policy for the system that ensures that the system satisfies the specification. This thesis presents novel control policy synthesis algorithms for optimal and robust control of dynamical systems with temporal logic specifications. Furthermore, it introduces algorithms that are efficient and extend to high-dimensional dynamical systems.

The first contribution of this thesis is the generalization of a classical linear temporal logic (LTL) control synthesis approach to optimal and robust control. We show how we can extend automata-based synthesis techniques for discrete abstractions of dynamical systems to create optimal and robust controllers that are guaranteed to satisfy an LTL specification. Such optimal and robust controllers can be computed at little extra computational cost compared to computing a feasible controller.

The second contribution of this thesis addresses the scalability of control synthesis with LTL specifications. A major limitation of the standard automaton-based approach for control with LTL specifications is that the automaton might be doubly-exponential in the size of the LTL specification. We introduce a fragment of LTL for which one can compute feasible control policies in time polynomial in the size of the system and specification. Additionally, we show how to compute optimal control policies for a variety of cost functions, and identify interesting cases when this can be done in polynomial time. These techniques are particularly relevant for online control, as one can guarantee that a feasible solution can be found quickly, and then iteratively improve on the quality as time permits.

The final contribution of this thesis is a set of algorithms for computing feasible trajectories for high-dimensional, nonlinear systems with LTL specifications. These algorithms avoid a potentially computationally-expensive process of computing a discrete abstraction, and instead compute directly on the system's continuous state space. The first method uses an automaton representing the specification to directly encode a series of constrained-reachability subproblems, which can be solved in a modular fashion by using standard techniques. The second method encodes an LTL formula as mixed-integer linear programming constraints on the dynamical system. We demonstrate these approaches with numerical experiments on temporal logic motion planning problems with high-dimensional (10+ states) continuous systems.

Long-term and seasonal patterns in coastal temperature and salinity along the North American west coast

Numerous integrated time series have been assembled that suggest global temperature has been increasing steadily over the last century. ... However, superimposed on the long-term warming trends of these series are decadal-scale fluctuations, periods of slightly increasing and even decreasing temperature followed by rapid increases in temperature. ... In this pilot study, data for 1931-1990 from eight [western North America] coastal stations are examined to test the utility of a state-space statistical model (developed by Dr. Roy Mendelssohn, PFEG) in separating and describing seasonal patterns and long-term trends.

Dynamic learning with the EM algorithm for neural networks

In this paper, we derive an EM algorithm for nonlinear state space models. We use it to estimate jointly the neural network weights, the model uncertainty and the noise in the data. In the E-step we apply a forwardbackward Rauch-Tung-Striebel smoother to compute the network weights. For the M-step, we derive expressions to compute the model uncertainty and the measurement noise. We find that the method is intrinsically very powerful, simple and stable.

Generalization in Adaptation to Stable and Unstable Dynamics

Humans skillfully manipulate objects and tools despite the inherent instability. In order to succeed at these tasks, the sensorimotor control system must build an internal representation of both the force and mechanical impedance. As it is not practical to either learn or store motor commands for every possible future action, the sensorimotor control system generalizes a control strategy for a range of movements based on learning performed over a set of movements. Here, we introduce a computational model for this learning and generalization, which specifies how to learn feedforward muscle activity in a function of the state space. Specifically, by incorporating co-activation as a function of error into the feedback command, we are able to derive an algorithm from a gradient descent minimization of motion error and effort, subject to maintaining a stability margin. This algorithm can be used to learn to coordinate any of a variety of motor primitives such as force fields, muscle synergies, physical models or artificial neural networks. This model for human learning and generalization is able to adapt to both stable and unstable dynamics, and provides a controller for generating efficient adaptive motor behavior in robots. Simulation results exhibit predictions consistent with all experiments on learning of novel dynamics requiring adaptation of force and impedance, and enable us to re-examine some of the previous interpretations of experiments on generalization. © 2012 Kadiallah et al.

Critical behavior of multifragmentation in finite nuclei and extraction of critical exponents

Using a phenomenological asymmetric nuclear equation of state, we obtained pressure-density isotherms of the finite nucleus Sn-112 simulated in r-space and in p-space and constructed the nuclear fragments by using the coalescence model. After correlatively analysing the fragments, the signal of critical behavior has been found and critical exponents were also extracted.

A new model of phycobilisome in Spirulina platensis

Phycobilisomes (PBS) were isolated from blue-green alga Spirulina platensis. Scanning tunneling microscope was used to investigate the three-dimensional structure of PBS deposited on freshly cleaved highly oriented pyrolytic graphite (HOPG) in ambient condition at room temperature. The results showed that the rods of PBS radiated from the core to different directions in the space other than arrayed in one plane, which was different from the typical hemi-discoidal model structure. The diameter of PBS was up to 70 nm, and the rod was approximately 50 nm in length. Similar results were observed in Langmuir-Blodgett (LB) film of PBS. The dissociated PBS could reaggregate into rod-like structures and easily form two-dimensional membrane while being absorbed on HOPG, however, no intact PBS was observed. The filling-space model structure of PBS in Spirulina platensis with STM from three-dimensional real space at nanometer scale was found, which showed that this new structural model of PBS surely exists in blue-green algae and red algae. The function of this structural model of PBS was also discussed.

基于神经网络的水泥生料配料多目标优化设计方法

为了降低生料成分的不确定性给水泥生料质量控制系统带来的影响,提出了率值补偿的控制策略.分别为三率值创建目标函数,并利用状态空间搜索策略解决多目标优化问题.针对初始样本空间不能覆盖所有样本的问题,提出了基于神经网络的估算模型,对初始样本空间进行拓扑.通过估价函数对状态空间中的状态量进行评价,得到最优的率值状态量;根据率值对原料配比进行调整,最后使率值偏差得到补偿,同时使给配比造成的波动最小.工业实验结果表明,生料的质量合格率由原来的30%提高到50%,该系统能有效地对配料过程进行优化控制.证明了基于神经网络的状态空间搜索策略为水泥生料配料多目标寻优问题提供了一种可行的方法。

多机器人动态编队的强化学习算法研究

在人工智能领域中 ,强化学习理论由于其自学习性和自适应性的优点而得到了广泛关注 随着分布式人工智能中多智能体理论的不断发展 ,分布式强化学习算法逐渐成为研究的重点 首先介绍了强化学习的研究状况 ,然后以多机器人动态编队为研究模型 ,阐述应用分布式强化学习实现多机器人行为控制的方法 应用SOM神经网络对状态空间进行自主划分 ,以加快学习速度 ;应用BP神经网络实现强化学习 ,以增强系统的泛化能力 ;并且采用内、外两个强化信号兼顾机器人的个体利益及整体利益 为了明确控制任务 ,系统使用黑板通信方式进行分层控制 最后由仿真实验证明该方法的有效性

工业机器人的小脑模型控制

本文为动力学控制工业机器人提出了一种综合学习算法,这种学习算法可将以前所学的信息用于新的控制输入.这种控制方法不需要事先知道机器人动力学,它易于应用于特殊的控制问题或修改以适应实际系统中的变化,控制方法在时间上是有效的,且很适合于定点实现.学习控制算法的有效性通过4自由度的直接驱动机器人前两个关节在重复运动中的计算机仿真实验得到了验证.

基于PMAC开放式数控系统软件开发

基于PC和多轴运动控制器的开放式数控系统是理想的开放式数控系统。介绍了基于PMAC的开放式数控系统结构形式,PMAC的差补、位置控制、伺服功能、以PMAC和PC机为硬件平台搭建了数控系统,并对其硬件构成和软件设计结构进行了分析。着重从软件设计的角度,介绍了PTALK控件的功能和作用,对数控系统软件构成进行了详细的阐述。并设计出了友好的用户界面,在实际应用中具有重要意义。