The misbehavior of value and the discipline of the will.

Most reinforcement learning models of animal conditioning operate under the convenient, though fictive, assumption that Pavlovian conditioning concerns prediction learning whereas instrumental conditioning concerns action learning. However, it is only through Pavlovian responses that Pavlovian prediction learning is evident, and these responses can act against the instrumental interests of the subjects. This can be seen in both experimental and natural circumstances. In this paper we study the consequences of importing this competition into a reinforcement learning context, and demonstrate the resulting effects in an omission schedule and a maze navigation task. The misbehavior created by Pavlovian values can be quite debilitating; we discuss how it may be disciplined.

When will a flame describing function approach to thermoacoustics work well?

In any thermoacoustic analysis, it is important not only to predict linear frequencies and growth rates, but also the amplitude and frequencies of any limit cycles. The Flame Describing Function (FDF) approach is a quasi-linear analysis which allows the prediction of both the linear and nonlinear behaviour of a thermoacoustic system. This means that one can predict linear growth rates and frequencies, and also the amplitudes and frequencies of any limit cycles. The FDF achieves this by assuming that the acoustics are linear and that the flame, which is the only nonlinear element in the thermoacoustic system, can be adequately described by considering only its response at the frequency at which it is forced. Therefore any harmonics generated by the flame's nonlinear response are not considered. This implies that these nonlinear harmonics are small or that they are sufficiently filtered out by the linear dynamics of the system (the low-pass filter assumption). In this paper, a flame model with a simple saturation nonlinearity is coupled to simple duct acoustics, and the success of the FDF in predicting limit cycles is studied over a range of flame positions and acoustic damping parameters. Although these two parameters affect only the linear acoustics and not the nonlinear flame dynamics, they determine the validity of the low-pass filter assumption made in applying the flame describing function approach. Their importance is highlighted by studying the level of success of an FDF-based analysis as they are varied. This is achieved by comparing the FDF's prediction of limit-cycle amplitudes to the amplitudes seen in time domain simulations.