强化学习算法中启发式回报函数的设计及其收敛性分析

回报函数设计的好与坏对学习系统性能有着重要作用,按回报值在状态-动作空间中的分布情况,将回报函数的构建分为两种形式:密集函数和稀疏函数,分析了密集函数和稀疏函数的特点.提出启发式回报函数的基本设计思路,利用基于保守势函数差分形式的附加回报函数,给学习系统提供更多的启发式信息,并对算法的最优策略不变性和迭代收敛性进行了证明.启发式回报函数能够引导学习,加快学习进程,从而可以实现强化学习在实际大型复杂系统应用中的实时控制和调度.

An application of the analytic hierarchy process and fuzzy logic inference in a decision support system for forage selection

Forage selection plays a prominent role in the process of returning cultivated lands back into grasslands. The conventional method of selecting forage species can only provide attempts for problem-solving without considering the relationships among the decision factors globally. Therefore, this study is dedicated to developing a decision support system to help farmers correctly select suitable forage species for the target sites. After collecting data through a field study, we developed this decision support system. It consists of three steps: (1) the analytic hierarchy process (AHP), (2) weights determination, and (3) decision making. In the first step, six factors influencing forage growth were selected by reviewing the related references and by interviewing experts. Then a fuzzy matrix was devised to determine the weight of each factor in the second step. Finally, a gradual alternative decision support system was created to help farmers choose suitable forage species for their lands in the third step. The results showed that the AHP and fuzzy logic are useful for forage selection decision making, and the proposed system can provide accurate results in a certain area (Gansu Province) of China.

a flexible negotiation model for an agent-based software process modelling

Our Agent-based Software Process Modelling (ASPM) approach describes a software process as a set of cooperative agents. Negotiation is the way in which the agents construct their cooperative relations, and thus the software process. Currently, most negotiation models use a fixed negotiation protocol and fixed strategies. In order to achieve the flexibility that the negotiation of the agents in ASPM requires, we propose a negotiation model NM-PA. NM-PA mainly includes a generic negotiation protocol and some rules, which possibly change in different negotiation processes. By changing the rules, the model can support multi-protocols and multi-decision-making strategies at a lower cost.

改善三维波动方程叠前深度偏移处理效果的方法研究

3D wave equation prestack depth migration is the effective tool for obtaining the exact imaging result of complex geology structures. It's a part of the 3D seismic data processing. 3D seismic data processing belongs to high dimension signal processing, and there are some difficult problems to do with. They are: How to process high dimension operators? How to improve the focusing? and how to construct the deconvolution operator? The realization of 3D wave equation prestack depth migration, not only realized the leap from poststack to prestack, but also provided the important means to solve the difficult problems in high dimension signal processing. In this thesis, I do a series research especially for the solve of the difficult problems around the 3D wave equation prestack depth migration and using it as a mean. So this thesis service for the realization of 3D wave equation prestack depth migration for one side and improve the migration effect for another side. This thesis expatiates in five departs. Summarizes the main contents as the follows: In the first part, I have completed the projection from 3D data point area to low dimension are using de big matrix transfer and trace rearrangement, and realized the liner processing of high dimension signal. Firstly, I present the mathematics expression of 3D seismic data and the mean according to physics, present the basic ideal of big matrix transfer and describe the realization of five transfer models for example. Secondly, I present the basic ideal and rules for the rearrange and parallel calculate of 3D traces, and give a example. In the conventional DMO focusing method, I recall the history of DM0 process firstly, give the fundamental of DMO process and derive the equation of DMO process and it's impulse response. I also prove the equivalence between DMO and prestack time migration, from the kinematic character of DMO. And derive the relationship between DMO base on wave equation and prestack time migration. Finally, I give the example of DMO process flow and synthetic data of theoretical models. In the wave equation prestak depth migration, I firstly recall the history of migration from time to depth, from poststack to prestack and from 2D to 3D. And conclude the main migration methods, point out their merit and shortcoming. Finally, I obtain the common image point sets using the decomposed migration program code.In the residual moveout, I firstly describe the Viterbi algorithm based on Markov process and compound decision theory and how to solve the shortest path problem using Viterbi algorithm. And based on this ideal, I realized the residual moveout of post 3D wave equation prestack depth migration. Finally, I give the example of residual moveout of real 3D seismic data. In the migration Green function, I firstly give the concept of migration Green function and the 2D Green function migration equation for the approximate of far field. Secondly, I prove the equivalence of wave equation depth extrapolation algorithms. And then I derive the equation of Green function migration. Finally, I present the response and migration result of Green function for point resource, analyze the effect of migration aperture to prestack migration result. This research is benefit for people to realize clearly the effect of migration aperture to migration result, and study on the Green function deconvolution to improve the focusing effect of migration.