强化学习算法中启发式回报函数的设计及其收敛性分析

回报函数设计的好与坏对学习系统性能有着重要作用,按回报值在状态-动作空间中的分布情况,将回报函数的构建分为两种形式:密集函数和稀疏函数,分析了密集函数和稀疏函数的特点.提出启发式回报函数的基本设计思路,利用基于保守势函数差分形式的附加回报函数,给学习系统提供更多的启发式信息,并对算法的最优策略不变性和迭代收敛性进行了证明.启发式回报函数能够引导学习,加快学习进程,从而可以实现强化学习在实际大型复杂系统应用中的实时控制和调度.

An Extension of 2D Janbu’s Generalized Procedure of Slices for 3D Slope Stability Analysis II—Numerical Method and Applications

This paper provides a numerical approach on achieving the limit equilibrium method for 3D slope stability analysis proposed in the theoretical part of the previous paper. Some programming techniques are presented to ensure the maneuverability of the method. Three examples are introduced to illustrate the use of this method. The results are given in detail such as the local factor of safety and local potential sliding direction for a slope. As the method is an extension of 2D Janbu's generalized procedure of slices (GPS), the results obtained by GPS for the longitudinal sections of a slope are also given for comparison with the 3D results. A practical landslide in Yunyang, the Three Gorges, of China, is also analyzed by the present method. Moreover, the proposed method has the advantages and disadvantages of GPS. The problem frequently encountered in calculation process is still about the convergency, especially in analyzing the stability of a cutting corner. Some advice on discretization is given to ensure convergence when the present method is used. However, the problem about convergency still needs to be further explored based on the rigorous theoretical background.

Finite element beam propagation method for analysis of plasmonic waveguide

The basic idea of the finite element beam propagation method (FE-BPM) is described. It is applied to calculate the fundamental mode of a channel plasmonic polariton (CPP) waveguide to confirm its validity. Both the field distribution and the effective index of the, fundamental mode are given by the method. The convergence speed shows the advantage and stability of this method. Then a plasmonic waveguide with a dielectric strip deposited on a metal substrate is investigated, and the group velocity is negative for the fundamental mode of this kind of waveguide. The numerical result shows that the power flow direction is reverse to that of phase velocity.