The misbehavior of value and the discipline of the will.

Most reinforcement learning models of animal conditioning operate under the convenient, though fictive, assumption that Pavlovian conditioning concerns prediction learning whereas instrumental conditioning concerns action learning. However, it is only through Pavlovian responses that Pavlovian prediction learning is evident, and these responses can act against the instrumental interests of the subjects. This can be seen in both experimental and natural circumstances. In this paper we study the consequences of importing this competition into a reinforcement learning context, and demonstrate the resulting effects in an omission schedule and a maze navigation task. The misbehavior created by Pavlovian values can be quite debilitating; we discuss how it may be disciplined.

Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans.

Theories of instrumental learning are centred on understanding how success and failure are used to improve future decisions. These theories highlight a central role for reward prediction errors in updating the values associated with available actions. In animals, substantial evidence indicates that the neurotransmitter dopamine might have a key function in this type of learning, through its ability to modulate cortico-striatal synaptic efficacy. However, no direct evidence links dopamine, striatal activity and behavioural choice in humans. Here we show that, during instrumental learning, the magnitude of reward prediction error expressed in the striatum is modulated by the administration of drugs enhancing (3,4-dihydroxy-L-phenylalanine; L-DOPA) or reducing (haloperidol) dopaminergic function. Accordingly, subjects treated with L-DOPA have a greater propensity to choose the most rewarding action relative to subjects treated with haloperidol. Furthermore, incorporating the magnitude of the prediction errors into a standard action-value learning algorithm accurately reproduced subjects' behavioural choices under the different drug conditions. We conclude that dopamine-dependent modulation of striatal activity can account for how the human brain uses reward prediction errors to improve future decisions.

Lyapunov functions for stable cascades and applications to global stabilization

This paper generalizes recent Lyapunov constructions for a cascade of two nonlinear systems, one of which is stable rather than asymptotically stable. A new cross-term construction in the Lyapunov function allows us to replace earlier growth conditions by a necessary boundedness condition. This method is instrumental in the global stabilization of feedforward systems, and new stabilization results are derived from the generalized construction.

Values and actions in aversion

Model-based and model-free controllers can, in principle, learn arbitrary actions to optimize their behavior, at least those actions that can be expressed and explored. Indeed, these are often referred to as instrumental controllers because their choices are learned to be instrumental for the delivery of desired outcomes. Although this flexibility is very powerful, it comes with an attendant cost of learning. Evolution appears to have endowed everything from the simplest organisms to us with powerful, pre-specified, but inflexible alternatives. These responses are termed Pavlovian, after the famous Russian physiologist and psychologist Pavlov. The responses of the Pavlovian controller are determined by evolutionary (phylogenetic) considerations rather than (ontogenetic) aspects of the contingent development or learning of an individual. These responses directly interact with instrumental choices arising from goal-directed and habitual controllers. This interaction has been studied in a wealth of animal paradigms, and can be helpful, neutral, or harmful, according to circumstance. Although there has been less careful or analytical study of it in humans, it can be interpreted as underpinning a wealth of behavioral aberrations. © 2009 Elsevier Inc. All rights reserved.

Atom probe tomography for advanced metallization

In microelectronics, the increase in complexity and the reduction of devices dimensions make essential the development of new characterization tools and methodologies. Indeed advanced characterization methods with very high spatial resolution are needed to analyze the redistribution at the nanoscale in devices and interconnections. The atom probe tomography has become an essential analysis to study materials at the nanometer scale. This instrument is the only analytical microscope capable to produce 3D maps of the distribution of the chemical species with an atomic resolution inside a material. This technique has benefit from several instrumental improvements during last years. In particular, the use of laser for the analysis of semiconductors and insulating materials offers new perspectives for characterization. The capability of APT to map out elements at the atomic scale with high sensitivity in devices meets the characterization requirements of semiconductor devices such as the determination of elemental distributions for each device region. In this paper, several examples will show how APT can be used to characterize and understand materials and process for advanced metallization. The possibilities and performances of APT (chemical analysis of all the elements, atomic resolution, planes determination, crystallographic information...) will be described as well as some of its limitations (sample preparation, complex evaporation, detection limit, ...). The examples illustrate different aspect of metallization: dopant profiling and clustering, metallic impurities segregation on dislocation, silicide formation and alloying, high K/metal gate optimization, SiGe quantum dots, as well as analysis of transistors and nanowires. © 2013 Elsevier B.V. All rights reserved.

基于度量工具的软件成本估算模型使用方法

输入的主观性以及输入过多是妨碍软件成本估算模型实际应用效果的重要影响因素.针对以上问题,提出了一种基于度量工具的软件成本估算模型使用方法.该方法通过引入统计理论中的工具变量,将度量工具所采集的度量元数据自动转换为软件成本估算模型的输入.这一方面可以避免模型校准和估算过程中输入的主观性与不一致性,提高了估算结果的准确性与可靠性;另一方面能减少估算人员的手工操作,提高工作效率,增加了软件成本估算模型的可用性.结合具体实例说明了所提出方法的可行性与有效性.