强化学习算法中启发式回报函数的设计及其收敛性分析

回报函数设计的好与坏对学习系统性能有着重要作用,按回报值在状态-动作空间中的分布情况,将回报函数的构建分为两种形式:密集函数和稀疏函数,分析了密集函数和稀疏函数的特点.提出启发式回报函数的基本设计思路,利用基于保守势函数差分形式的附加回报函数,给学习系统提供更多的启发式信息,并对算法的最优策略不变性和迭代收敛性进行了证明.启发式回报函数能够引导学习,加快学习进程,从而可以实现强化学习在实际大型复杂系统应用中的实时控制和调度.

成批生产计划调度的集成建模与优化

针对多品种批量生产类型,建立了调度约束的生产计划与调度集成优化模型。模型的目标函数是使总调整费用、库存费用及生产费用之和最小,约束函数包括库存平衡约束和生产能力约束,同时考虑了调度约束,即工序顺序约束和工件在单机上的加工能力约束,保证了计划可行性。该模型为两层混合整数规划模型,对其求解综合运用了遗传算法和启发式规则,提出了混合启发式求解算法。最后,针对某机床厂多品种批量生产类型车间进行了实例应用,对车间零件月份作业计划进行分解,得到各工段单元零件周作业计划,确定了零件各周生产批量与投产顺序。

悬索桥隧道式锚碇与围岩系统的破坏模式研究

The anchorages are unparalleled structures only in a suspension bridge, and as main bearing facilities, play an important role in connecting the superstructures and the ground. The tunnel anchorage, as one alternative type of the anchorages, has more advantages over its counterpart, the gravity anchorage. With the tunnel anchorages adopted, not only can surface excavation be reduced to protect the environment, and natural condition of the rock be utilized and potential bearing capacity of surrounding rock be mobilized to save engineering cost, but also the technological predominance of auxiliary engineering measures, such as prestressed concrete, anchoring piles, rock anchors and collar beam between the two separated anchorages, can be easily cooperated to work together harmoniously under the circumstances of poor rock quality. There are plentiful high mountains and deep canyons in west part of China, and long-span bridge construction is inevitably encountered in order to realize leapfrogging development of the transportation infrastructure. Western mountainous areas usually possess the conditions for constructing tunnel anchorages, and therefore, the tunnel anchorages, which are conformed to the conception of resource conservative and sustainable society, extremely have application and popularization value in western underdeveloped region. The scientific and technological problem about the design, construction and operation of tunnel anchorages should be further investigated. Combining the engineering of western tunnel anchorages for the Balinghe Suspension Bridge, this paper probed into the survey method and in-situ test method for tunnel anchorages, scientific rock quality evaluation of surrounding rock to provide reasonable physical and mechanical parameters for design, construction and operation of tunnel anchorages, bearing capacity estimation for tunnel anchorage, deformation prediction of the anchorage-rockmass system, tunnel-anchorage slope stability analysis and the evaluation of excavation stability and degree of safety of the anchorage tunnel. The following outcomes were obtained: 1. Materials of tunnel anchorages of suspension bridge built (and in progress) at home and abroad were systematically sorted out, with the engineering geological condition and geomechanical property of surrounding rock around the anchorage tunnel, the design size of anchorages and the construction method of anchorage tunnel paid more emphasis on, to unveil the internal relationship between the engineering geological conditions of surrounding rock and the design size and axis angle of anchorages and provide references for future design, construction and study of tunnel anchorages. 2. Physical and mechanical parameters were recommended based on three domestic and foreign methods of rock quality evaluation. 3. In-situ tests, adopting the back-thrust method, of two kinds of reduced scale model, 1/30 and 1/20, for the tunnel anchorages were conducted in the declining exploration drift with rock mass at the test depth being the same as surrounding rock around real anchorages, and reliable field rockmass displacement data were acquired. Attenuation relation between the increment of distance from the anchorage and the decrement of rockmass displacement under maximum test load, and influential scope suffered by anchorage load were obtained. 4. Using similarity theory, the magnitude of real anchorage and rockmass displacement under design load and degree of safety of the anchorage system were deduced. Furthermore, inversion analysis to deformation modulus of slightly weathered dolomite rock, the surrounding rock of anchorage tunnel, was performed by the means of numerical simulation. 5. The influential law of the geometrical size to the limit bearing capacity of tunnel anchorage was studied. 6. Based on engineering geological survey data, accounting for the combination of strata layer and adverse discontinuities, the failure patterns of tunnel anchorage slope were divided into three modes: sliding of splay saddle pier slope, superficial-layer slippage, and deep-layer slippage. Using virtual work principle and taking anchorage load in account, the stability of the three kinds of failure patterns were analyzed in detail. 7. The step-by-step excavation of anchorage tunnel, the numerical overload and the staged decrement of rock strength parameters were numerically simulated to evaluate the excavation stability of surrounding rock around anchorage tunnel, the overload performance of tunnel anchorage, and the safety margin of strength parameters of the surrounding rock.

基于工程地质力学的岩土工程加固理论、技术与实践

In the engineering reinforcement of-rock and soil mass, engineers must consider how to obtain better reinforcing effect at the cost of less reinforcing expense, which, in fact, is the aim of reinforcement design. In order to accomplish the purpose, they require not only researching the material used to reinforce and its structure, but also taking into account of several important geological factors, such as the structure and property of rock and soil mass. How to improve the reinforcing effect according to engineering geomechanical principle at the respect of the reinforcement of engineering soil and rock mass is studied and discussed in this paper. The author studies the theory, technology and practice of geotechnical reinforcement based on engineering geomechanics, taking example for the soil treatment of Zhengzhou Airport, the effect analysis of reinforcement to the slope on the left bank of Wuqiangxi Hydropower Station and the reinforcing design of the No. 102 Landslide and unique sand-slide slope on the Sichuan-Tibet Highway. The paper is comprised of two parts for the convenience of discussion. In the first part, from the first chapter to the fifth chapter, trying to perform the relevant research and application at the viewpoint of soil mass engineering geomechanics, the author mainly discusses the study of reinforcing soft ground soil through dynamical consolidation and its application. Then, in the second part, from the sixth chapter to the eleventh chapter, the study of new technologies in the rock slope reinforcement and their application are discussed. The author finds that not only better reinforcing effect can be gained in the research where the principle and method of rock mass engineering geomechanics is adopted, but also new reinforcing technologies can be put forward. Zhengzhou Airport is an important one in central plains. It lies on Yellow River alluvial deposit and the structure of stratum is complex and heterogeneous. The area of airport is very large, which can result in differential settlement easily, damage of airport and aircraft accident, whereas, there are no similar experiences to dispose the foundation, so the foundation treatment become a principal problem. During the process of treatment, the method of dynamic compaction was adopted after compared with other methods using the theory of synthetic integration. Dynamic compaction is an important method to consolidate foundation, which was successfully used in the foundation of Zhengzhou Airport. For fill foundation, controlling the thickness of fill so as to make the foundation treatment can reach the design demand and optimum thickness of the fill is a difficult problem. Considering this problem, the author proposed a calculation method to evaluate the thickness of fill. The method can consider not only the self-settlement of fill but also the settlement of the ground surface under applied load so as to ensure the settlement occurred during the using period can satisfy the design demand. It is proved that the method is correct after using it to choose reasonable energy of dynamic compaction to treat foundation. At the same time, in order to examine the effect of dynamic compaction, many monitor methods were adopted in the test such as static loading test, modulus of resilience test, deep pore pressure -test, static cone penetration test and the variation of the pore volume measurement. Through the tests, the author summarized the discipline of the accumulation and dissipation of pore pressure in Yellow River alluvial deposit under the action of dynamic compaction, gave a correct division of the property change of silt and clay under dynamic compaction, determined the bearing capacity of foundation after treatment and weighted the reinforcing effect of dynamic consolidation from the variation of the soil particle in microcosmic and the parameter of soil mass' density. It can be considered that the compactness of soil is in proportion to the energy of dynamic compaction. This conclusion provided a reference to the research of the "Problem of Soil Structure-the Central Problem of Soil Mechanics in 21 Century ". It is also important to strengthen rock mass for water conservancy and electric power engineering. Slip-resistance pile and anchoring adit full of reinforced concrete are usually adopted in engineering experience to strengthen rock mass and very important for engineering. But there also some deficiency such as the weakest section can't be highlighted, the monitor is inconvenient and the diameter of pile and adit is very large etc. The author and his supervisor professor Yangzhifa invented prestressed slip-resistance pile and prestressed anchoring adit full of reinforced concrete, utilizing the advantage that the prestressed structure has better anti-tensile characteristic (this invention is to be published). These inventions overcome the disadvantages of general slip-resistance pile and anchoring adit full of reinforced concrete and have the functions of engineering prospecting, strengthening, drainage and monitor simultaneous, so they have better strengthened effect and be more convenient for monitor and more economical than traditional methods. Drainage is an important factor in treatments of rock mass and slop. In view of the traditional drainage method that drainage pore often be clogged so as to resulted in incident, professor Yangzhifa invented the method and setting of guide penetration by fiber bundle. It would take good effect to use it in prestressed slip-resistance pile and anchoring adit full of reinforced concrete. In this paper, the author took example for anchoring adit full of reinforced concrete used to strengthen Wuqiangxi left bank to simulate the strengthened effect after consolidated by prestressed slip-resistance pile, took example for 102 landslide occurred along Sichuan-Tibet highway to simulate the application of slip-resistance pile and the new technology of drainage. At the same time the author proposed the treatment method of flowing sand in Sichuan-Tibet highway, which will benefit the study on strengthening similar engineering. There are five novelties in the paper with the author's theoretical study and engineering practice: 1. Summarizing the role of pore water pressure accumulation and dissipation of the Yellow River alluvial and diluvial soil under the action of dynamical consolidation, which has instructive significance in the engineering construction under the analogical engineering geological conditions in the future. It has not been researched by the predecessors. 2. Putting forward the concept of density D in microcosmic based on the microcosmical structure study of the soil sample. Adopting D to weight the reinforcing effect of dynamic consolidation is considered to be appropriate by the means of comparing the D values of Zhengzhou Airport's ground soil before with after dynamically consolidating reinforcement, so a more convenient balancing method can be provided for engineering practice. 3. According to the deep research into the soil mass engineering geology, engineering rock and soil science, soil mechanics, as well as considerable field experiments, improving the consolidating method in airport construction, from the conventional method, which is dynamically compactmg original ground surface firstly, then filling soil and dynamically layer-consolidating or layer-compacting at last to the upgraded method, which is performing dynamical consolidation after filling soil to place totally at the extent of the certain earth-filling depth. The result of the dynamical consolidation not only complies with the specifications, but also reduces the soil treatment investment by 10 million RMB. 4. Proposing the method for calculating the height of the filled soil by the means of estimating the potential displacement produced in the original ground surface and the filled earth soil under the possible load, selecting the appropriate dynamically-compacting power and determining the virtual height of the filled earth soil. The method is proved to be effective and scientific. 5. According to the thought of Engineering Geomechanics Metal-Synthetic Methodology (EGMS), patenting two inventions (to the stage of roclamation, with Professor Yang Zhi-fa, the cooperative tutor, and etc.) in which multi-functions, engineering geological investigation, reinforcement, drainage and strength remedy, are integrated all over in one body at the viewpoint of the breakage mechanism of the rock slope.

学习不良儿童的元认知偏差及矫正

Metacognitive illusions or metacognitive bias is a concept that is a homologous with metacognitve monitor accuracy. In the dissertation, metacognitive illusions mainly refers to the absolute differences between judgment of learning (JOL) and recall because individuals are misguided by some invalid cues or information. JOL is one kind of metacognitive judgments, which is the prediction about the future performance of learned materials. Its mechanism and accuracy are the key issues in the study of JOL. Cue-utilization framework proposed by Koriat (1997) summarized the previous findings and provided a significant advance in understanding how people make JOL. However, the model is not able to explain individual differences in the accuracy of JOL. From the perspective of people’s cognitive bound, our study use posterior associative word pairs easy to produce metacognitive bias to explore the deeper psychological mechanism of metacontive bias. Moreover, we plan to investigate the cause to result in higher metacognitive illusions of children with LD. Based on these, the study tries to look for the method of mending metacognitive illusions. At the same time, we will summarize the findings of this study and previous literatures, and propose a revesied theory for explaining children’s with LD cue selection and utilization according to Koriat’s cue-utilization model. The results of the present study indicated that: (1) Children showed stable metacognitive illusions for the weak associative and posterior associative word pairs, it was not true for strong associative word pairs. It was higher metacognitive illusions for children with LD than normal children. And it was significant grade differences for metacognitive illusions. A priori associative strength exerted a weaker effect on JOL than it did on recall. (2) Children with LD mainly utilized retrieval fluency to make JOL across immediate and delay conditions. However, for normal children, it showed some distinction between encoding fluency and retrieval fluency as potential cues for JOL across immediate and delay conditions. Obviously, children with LD lacked certain flexibility for cue selection and utilization. (3)When word pairs were new list, it showed higher metacognitve transfer effects for analytic inferential group than heuristic inferential group for normal children in the second block. And metacognitive relative accuracy got increased for both children with and without LD across the experimental conditions. However, it was significantly improved only for normal children in analytic inferential group.

5-13岁儿童对惯性运动的认知发展

Research on naïve physics investigates children’s intuitive understanding of physical objects, phenomena and processes. Children, and also many adults, were found to have a misconception of inertia, called impetus theory. In order to investigate the development of this naïve concept and the mechanism underlying it, four age groups (5-year-olds, 2nd graders, 5th graders, and 8th graders) were included in this research. Modified experimental tasks were used to explore the effects of daily experience, perceptual cues and general information-processing ability on children’s understanding of inertia. The results of this research are: 1) Five- to thirteen-year-olds’ understanding of inertia problems which were constituted by two ogjects moving at the same spped undergoes an L-shaped developmental trend; Children’s performance became worse as they got older, and their performance in the experiment did not necessarily ascend with the improvement of their cognitive abilities. 2) The L-shaped developmental curve suggests that children in different ages used different strategies to solve inertia problems: Five- to eight-year-olds only used heuristic strategy, while eleven- to thirteen-year-olds solved problems by analyzing the details of inertia motion. 3) The different performance between familiar and unfamiliar problems showed that older children were not able to spontaneously transfer their knowledge and experience from daily action and observation of inertia to unfamiliar, abstract inertia problems. 4) Five- to eight-year-olds showed straight and fragmented pattern, while more eleven- to thirteen-year-olds showed standard impetus theory and revised impetus theory pattern, which showed that younger children were influenced by perceptual cues and their understanding of inertia was fragmented, while older children had coherent impetus theory. 5) When the perceptual cues were controlled, even 40 percent 5 years olds showed the information-processing ability to analyze the distance, speed and time of two objects traveling in two different directions at the same time, demonstrating that they have achieved a necessary level to theorize their naïve concept of inertia.

弈棋问题解决启发式的元策略模型

Problem solving is one of the basic processes of human cognition and heuristic strategy is the key to human problem solving, hence, the studies on heuristic strategy is of great importance in cognitive psychology. Current studies on heuristics in problem solving may be summarized as follows: nature and structure of heuristics, problem structure and representation, expert knowledge and expert intuition, nature and role of image, social cognition and social learning. The present study deals with the nature and structure of heuristics. The Solitaire problem was used in our the experiments. Both traditional experimental method and computer simulation were used to study the nature and structure of heuristics. Through a series of experiments, the knowledge of Solitaire problem solving was summed up, its metastrategy is worked out, and then the the metastrategy by computer simulation and experimental verification are tested.

Automatic Recognition of Tractability in Inference Relations

A procedure is given for recognizing sets of inference rules that generate polynomial time decidable inference relations. The procedure can automatically recognize the tractability of the inference rules underlying congruence closure. The recognition of tractability for that particular rule set constitutes mechanical verification of a theorem originally proved independently by Kozen and Shostak. The procedure is algorithmic, rather than heuristic, and the class of automatically recognizable tractable rule sets can be precisely characterized. A series of examples of rule sets whose tractability is non-trivial, yet machine recognizable, is also given. The technical framework developed here is viewed as a first step toward a general theory of tractable inference relations.

Three Cuts for Accelerated Interval Propagation

This paper addresses the problem of nonlinear multivariate root finding. In an earlier paper we described a system called Newton which finds roots of systems of nonlinear equations using refinements of interval methods. The refinements are inspired by AI constraint propagation techniques. Newton is competative with continuation methods on most benchmarks and can handle a variety of cases that are infeasible for continuation methods. This paper presents three "cuts" which we believe capture the essential theoretical ideas behind the success of Newton. This paper describes the cuts in a concise and abstract manner which, we believe, makes the theoretical content of our work more apparent. Any implementation will need to adopt some heuristic control mechanism. Heuristic control of the cuts is only briefly discussed here.

Internet Fish

I have invented "Internet Fish," a novel class of resource-discovery tools designed to help users extract useful information from the Internet. Internet Fish (IFish) are semi-autonomous, persistent information brokers; users deploy individual IFish to gather and refine information related to a particular topic. An IFish will initiate research, continue to discover new sources of information, and keep tabs on new developments in that topic. As part of the information-gathering process the user interacts with his IFish to find out what it has learned, answer questions it has posed, and make suggestions for guidance. Internet Fish differ from other Internet resource discovery systems in that they are persistent, personal and dynamic. As part of the information-gathering process IFish conduct extended, long-term conversations with users as they explore. They incorporate deep structural knowledge of the organization and services of the net, and are also capable of on-the-fly reconfiguration, modification and expansion. Human users may dynamically change the IFish in response to changes in the environment, or IFish may initiate such changes itself. IFish maintain internal state, including models of its own structure, behavior, information environment and its user; these models permit an IFish to perform meta-level reasoning about its own structure. To facilitate rapid assembly of particular IFish I have created the Internet Fish Construction Kit. This system provides enabling technology for the entire class of Internet Fish tools; it facilitates both creation of new IFish as well as additions of new capabilities to existing ones. The Construction Kit includes a collection of encapsulated heuristic knowledge modules that may be combined in mix-and-match fashion to create a particular IFish; interfaces to new services written with the Construction Kit may be immediately added to "live" IFish. Using the Construction Kit I have created a demonstration IFish specialized for finding World-Wide Web documents related to a given group of documents. This "Finder" IFish includes heuristics that describe how to interact with the Web in general, explain how to take advantage of various public indexes and classification schemes, and provide a method for discovering similarity relationships among documents.

Combining Associational and Causal Reasoning to Solve Interpretation and Planning Problems

This report describes a paradigm for combining associational and causal reasoning to achieve efficient and robust problem-solving behavior. The Generate, Test and Debug (GTD) paradigm generates initial hypotheses using associational (heuristic) rules. The tester verifies hypotheses, supplying the debugger with causal explanations for bugs found if the test fails. The debugger uses domain-independent causal reasoning techniques to repair hypotheses, analyzing domain models and the causal explanations produced by the tester to determine how to replace faulty assumptions made by the generator. We analyze the strengths and weaknesses of associational and causal reasoning techniques, and present a theory of debugging plans and interpretations. The GTD paradigm has been implemented and tested in the domains of geologic interpretation, the blocks world, and Tower of Hanoi problems.

Heuristics for Job-Shop Scheduling

Two methods of obtaining approximate solutions to the classic General Job-shop Scheduling Program are investigated. The first method is iterative. A sampling of the solution space is used to decide which of a collection of space pruning constraints are consistent with "good" schedules. The selected space pruning constraints are then used to reduce the search space and the sampling is repeated. This approach can be used either to verify whether some set of space pruning constraints can prune with discrimination or to generate solutions directly. Schedules can be represented as trajectories through a Cartesian space. Under the objective criteria of Minimum maximum Lateness family of "good" schedules (trajectories) are geometric neighbors (reside with some "tube") in this space. This second method of generating solutions takes advantage of this adjacency by pruning the space from the outside in thus converging gradually upon this "tube." One the average this methods significantly outperforms an array of the Priority Dispatch rules when the object criteria is that of Minimum Maximum Lateness. It also compares favorably with a recent relaxation procedure.

Learning by Failing to Explain

Explanation-based Generalization requires that the learner obtain an explanation of why a precedent exemplifies a concept. It is, therefore, useless if the system fails to find this explanation. However, it is not necessary to give up and resort to purely empirical generalization methods. In fact, the system may already know almost everything it needs to explain the precedent. Learning by Failing to Explain is a method which is able to exploit current knowledge to prune complex precedents, isolating the mysterious parts of the precedent. The idea has two parts: the notion of partially analyzing a precedent to get rid of the parts which are already explainable, and the notion of re-analyzing old rules in terms of new ones, so that more general rules are obtained.

Symbolic Integration

SIN and SOLDIER are heuristic programs in LISP which solve symbolic integration problems. SIN (Symbolic INtegrator) solves indefinite integration problems at the difficulty approaching those in the larger integral tables. SIN contains several more methods than are used in the previous symbolic integration program SAINT, and solves most of the problems attempted by SAINT in less than one second. SOLDIER (SOLution of Ordinary Differential Equations Routine) solves first order, first degree ordinary differential equations at the level of a good college sophomore and at an average of about five seconds per problem attempted. The differences in philosophy and operation between SAINT and SIN are described, and suggestions for extending the work presented are made.

Generating Semantic Descriptions From Drawings of Scenes With Shadows

The research reported here concerns the principles used to automatically generate three-dimensional representations from line drawings of scenes. The computer programs involved look at scenes which consist of polyhedra and which may contain shadows and various kinds of coincidentally aligned scene features. Each generated description includes information about edge shape (convex, concave, occluding, shadow, etc.), about the type of illumination for each region (illuminated, projected shadow, or oriented away from the light source), and about the spacial orientation of regions. The methods used are based on the labeling schemes of Huffman and Clowes; this research provides a considerable extension to their work and also gives theoretical explanations to the heuristic scene analysis work of Guzman, Winston, and others.