Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective.

The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.

Finite volume distance field solution applied to medial axis transform

Accurate and efficient computation of the nearest wall distance d (or level set) is important for many areas of computational science/engineering. Differential equation-based distance/ level set algorithms, such as the hyperbolic-natured Eikonal equation, have demonstrated valuable computational efficiency. Here, in the context, as an 'auxiliary' equation to the main flow equations, the Eikonal equation is solved efficiently with two different finite volume approaches (the cell vertex and cell-centered). Application of the distance solution is studied for various geometries. Moreover, a procedure using the differential field to obtain the medial axis transform (MAT) for different geometries is presented. The latter provides a skeleton representation of geometric models that has many useful analysis properties. As an alternative approach to the pure geometric methods (e.g. the Voronoi approach), the current d-MAT procedure bypasses many difficulties that are usually encountered by pure geometric methods, especially in three dimensional space. It is also shown that the d-MAT approach provides the potential to sculpt/control the MAT form for specialized solution purposes. Copyright © 2010 by the American Institute of Aeronautics and Astronautics, Inc.

Eliminating discharge imbalances in predicting shallow water flows with shocks

The shallow water equations are widely used in modelling environmental flows. Being a hyperbolic system of differential equations, they admit shocks that represent hydraulic jumps and bores. Although the water surface can be solved satisfactorily with the modern shock-capturing schemes, the predicted flow rate often suffers from imbalances where shocks occur, eg the mass conservation is violated by failing to maintain a constant discharge rate at every cross-section in a steady open channel flow. A total-variation-diminishing Lax-Wendroff scheme is developed, and used to demonstrate how to achieve an exact flux balance. The performance of the proposed methods is inspected through some test cases, which include 1- and 2-dimensional, flat and irregular bed scenarios. The proposed methods are shown to preserve the mass exactly, and can be easily extended to other shock-capturing models.

Stiffness of sands through a laboratory test database

Deformations of sandy soils around geotechnical structures generally involve strains in the range small (0·01%) to medium (0·5%). In this strain range the soil exhibits non-linear stress-strain behaviour, which should be incorporated in any deformation analysis. In order to capture the possible variability in the non-linear behaviour of various sands, a database was constructed including the secant shear modulus degradation curves of 454 tests from the literature. By obtaining a unique S-shaped curve of shear modulus degradation, a modified hyperbolic relationship was fitted. The three curve-fitting parameters are: an elastic threshold strain γe, up to which the elastic shear modulus is effectively constant at G0; a reference strain γr, defined as the shear strain at which the secant modulus has reduced to 0·5G0; and a curvature parameter a, which controls the rate of modulus reduction. The two characteristic strains γe and γr were found to vary with sand type (i.e. uniformity coefficient), soil state (i.e. void ratio, relative density) and mean effective stress. The new empirical expression for shear modulus reduction G/G0 is shown to make predictions that are accurate within a factor of 1·13 for one standard deviation of random error, as determined from 3860 data points. The initial elastic shear modulus, G0, should always be measured if possible, but a new empirical relation is shown to provide estimates within a factor of 1·6 for one standard deviation of random error, as determined from 379 tests. The new expressions for non-linear deformation are easy to apply in practice, and should be useful in the analysis of geotechnical structures under static loading.

Development of realistic liquid equations of state for multi-phase CFD modelling

Several equations of state (EOS) have been incorporated into a novel algorithm to solve a system of multi-phase equations in which all phases are assumed to be compressible to varying degrees. The EOSs are used to both supply functional relationships to couple the conservative variables to the primitive variables and to calculate accurately thermodynamic quantities of interest, such as the speed of sound. Each EOS has a defined balance of accuracy, robustness and computational speed; selection of an appropriate EOS is generally problem-dependent. This work employs an AUSM+-up method for accurate discretisation of the convective flux terms with modified low-Mach number dissipation for added robustness of the solver. In this paper we show a newly-developed time-marching formulation for temporal discretisation of the governing equations with incorporated time-dependent source terms, as well as considering the system of eigenvalues that render the governing equations hyperbolic.

Encoding of marginal utility across time in the human brain.

Marginal utility theory prescribes the relationship between the objective property of the magnitude of rewards and their subjective value. Despite its pervasive influence, however, there is remarkably little direct empirical evidence for such a theory of value, let alone of its neurobiological basis. We show that human preferences in an intertemporal choice task are best described by a model that integrates marginally diminishing utility with temporal discounting. Using functional magnetic resonance imaging, we show that activity in the dorsal striatum encodes both the marginal utility of rewards, over and above that which can be described by their magnitude alone, and the discounting associated with increasing time. In addition, our data show that dorsal striatum may be involved in integrating subjective valuation systems inherent to time and magnitude, thereby providing an overall metric of value used to guide choice behavior. Furthermore, during choice, we show that anterior cingulate activity correlates with the degree of difficulty associated with dissonance between value and time. Our data support an integrative architecture for decision making, revealing the neural representation of distinct subcomponents of value that may contribute to impulsivity and decisiveness.

Dread and the disvalue of future pain.

Standard theories of decision-making involving delayed outcomes predict that people should defer a punishment, whilst advancing a reward. In some cases, such as pain, people seem to prefer to expedite punishment, implying that its anticipation carries a cost, often conceptualized as 'dread'. Despite empirical support for the existence of dread, whether and how it depends on prospective delay is unknown. Furthermore, it is unclear whether dread represents a stable component of value, or is modulated by biases such as framing effects. Here, we examine choices made between different numbers of painful shocks to be delivered faithfully at different time points up to 15 minutes in the future, as well as choices between hypothetical painful dental appointments at time points of up to approximately eight months in the future, to test alternative models for how future pain is disvalued. We show that future pain initially becomes increasingly aversive with increasing delay, but does so at a decreasing rate. This is consistent with a value model in which moment-by-moment dread increases up to the time of expected pain, such that dread becomes equivalent to the discounted expectation of pain. For a minority of individuals pain has maximum negative value at intermediate delay, suggesting that the dread function may itself be prospectively discounted in time. Framing an outcome as relief reduces the overall preference to expedite pain, which can be parameterized by reducing the rate of the dread-discounting function. Our data support an account of disvaluation for primary punishments such as pain, which differs fundamentally from existing models applied to financial punishments, in which dread exerts a powerful but time-dependent influence over choice.

Differential equation based length scales to improve DES and RANS simulations

Surprisingly expensive to compute wall distances are still used in a range of key turbulence and peripheral physics models. Potentially economical, accuracy improving differential equation based distance algorithms are considered. These involve elliptic Poisson and hyperbolic natured Eikonal equation approaches. Numerical issues relating to non-orthogonal curvilinear grid solution of the latter are addressed. Eikonal extension to a Hamilton-Jacobi (HJ) equation is discussed. Use of this extension to improve turbulence model accuracy and, along with the Eikonal, enhance Detached Eddy Simulation (DES) techniques is considered. Application of the distance approaches is studied for various geometries. These include a plane channel flow with a wire at the centre, a wing-flap system, a jet with co-flow and a supersonic double-delta configuration. Although less accurate than the Eikonal, Poisson method based flow solutions are extremely close to those using a search procedure. For a moving grid case the Poisson method is found especially efficient. Results show the Eikonal equation can be solved on highly stretched, non-orthogonal, curvilinear grids. A key accuracy aspect is that metrics must be upwinded in the propagating front direction. The HJ equation is found to have qualitative turbulence model improving properties. © 2003 by P. G. Tucker.