Does temporal discounting explain unhealthy behavior? A systematic review and reinforcement learning perspective.

The tendency to make unhealthy choices is hypothesized to be related to an individual's temporal discount rate, the theoretical rate at which they devalue delayed rewards. Furthermore, a particular form of temporal discounting, hyperbolic discounting, has been proposed to explain why unhealthy behavior can occur despite healthy intentions. We examine these two hypotheses in turn. We first systematically review studies which investigate whether discount rates can predict unhealthy behavior. These studies reveal that high discount rates for money (and in some instances food or drug rewards) are associated with several unhealthy behaviors and markers of health status, establishing discounting as a promising predictive measure. We secondly examine whether intention-incongruent unhealthy actions are consistent with hyperbolic discounting. We conclude that intention-incongruent actions are often triggered by environmental cues or changes in motivational state, whose effects are not parameterized by hyperbolic discounting. We propose a framework for understanding these state-based effects in terms of the interplay of two distinct reinforcement learning mechanisms: a "model-based" (or goal-directed) system and a "model-free" (or habitual) system. Under this framework, while discounting of delayed health may contribute to the initiation of unhealthy behavior, with repetition, many unhealthy behaviors become habitual; if health goals then change, habitual behavior can still arise in response to environmental cues. We propose that the burgeoning development of computational models of these processes will permit further identification of health decision-making phenotypes.

Using the moving window incorporated neural network to forecast the population behavior of Nostocales spp. in the River Darling, Australia

The paper demonstrates the nonstationarity of algal population behaviors by analyzing the historical populations of Nostocales spp. in the River Darling, Australia. Freshwater ecosystems are more likely to be nonstationary, instead of stationary. Nonstationarity implies that only the near past behaviors could forecast the near future for the system. However, nonstionarity was not considered seriously in previous research efforts for modeling and predicting algal population behaviors. Therefore the moving window technique was incorporated with radial basis function neural network (RBFNN) approach to deal with nonstationarity when modeling and forecasting the population behaviors of Nostocales spp. in the River Darling. The results showed that the RBFNN model could predict the timing and magnitude of algal blooms of Nostocales spp. with high accuracy. Moreover, a combined model based on individual RBFNN models was implemented, which showed superiority over the individual RBFNN models. Hence, the combined model was recommended for the modeling and forecasting of the phytoplankton populations, especially for the forecasting.

Experimental and numerical investigation of the seismic behavior of rectangular tunnels in soft soils

Underground structures constitute crucial components of the transportation networks. Considering their significance for modern societies, their proper seismic design is of great importance. However, this design may become very tricky, accounting of the lack of knowledge regarding their seismic behavior. Several issues that are significantly affecting this behavior (i.e. earth pressures on the structure, seismic shear stresses around the structure, complex deformation modes for rectangular structures during shaking etc.) are still open. The problem is wider for the non-circular (i.e. rectangular) structures, were the soilstructure interaction effects are expected to be maximized. The paper presents representative experimental results from a test case of a series of dynamic centrifuge tests that were performed on rectangular tunnels embedded in dry sand. The tests were carried out at the centrifuge facility of the University of Cambridge, within the Transnational Task of the SERIES EU research program. The presented test case is also numerically simulated and studied. Preliminary full dynamic time history analyses of the coupled soil-tunnel system are performed, using ABAQUS. Soil non-linearity and soil-structure interaction are modeled, following relevant specifications for underground structures and tunnels. Numerical predictions are compared to experimental results and discussed. Based on this comprehensive experimental and numerical study, the seismic behavior of rectangular embedded structures is better understood and modeled, consisting an important step in the development of appropriate specifications for the seismic design of rectangular shallow tunnels.

Diurnal changes of behavior and respiration of Yangtze finless porpoises (Neophocaena phocaenoides asiaeorientalis) in captivity

A preliminary study was carried out to investigate diurnal changes of behavior of three, one adult mate, one adult female, and one juvenile female, Yangtze finless porpoises (Neophocaena phocaenoides asiaeorientalis) in captivity. The respiration and behavior of the porpoises were recorded for 222 hr across 42 days. Behavioral data were recorded for eight general categories: aerial display and fast swimming, begging for fish, playing, nonsexual socializing, sexual behavior, resting, rubbing, and miscellaneous (i.e., other behaviors not included in the above categories). Each behavioral category was scored using one-zero sampling with 10-min intervals. The adult male showed shorter mean respiratory intervals at night (19:00-7:00 h), whereas the mean respiratory intervals of the females were shorter during the day (7:00-19:00 h). Begging for fish of all individuals, playing of the juvenile female, nonsexual socializing, and miscellaneous behavior of the adult female and resting of the male were observed more easily in the day, and aerial display and fast swimming of the adults and resting of the females were observed more easily at night. No significant diurnal difference was found, however, in the remaining categories of each individual. Each of the three porpoises therefore showed a distinct diurnal pattern, but none was obviously more active in the daytime than during the nighttime. Results suggest that daytime-only feeding schedules may be insufficient to meet the energetic needs of marine mammals that show a 24-hr activity cycle, and that nighttime feeding may be a worthwhile addition to husbandry routines.

Self-organization of reflexive behavior from spontaneous motor activity.

In mammals, the development of reflexes is often regarded as an innate process. However, recent findings show that fetuses are endowed with favorable conditions for ontogenetic development. In this article, we hypothesize that the circuitry of at least some mammalian reflexes can be self-organized from the sensory and motor interactions brought forth in a musculoskeletal system. We focus mainly on three reflexes: the myotatic reflex, the reciprocal inhibition reflex, and the reverse myotatic reflex. To test our hypothesis, we conducted a set of experiments on a simulated musculoskeletal system using pairs of agonist and antagonist muscles. The reflex connectivity is obtained by producing spontaneous motor activity in each muscle and by correlating the resulting sensor and motor signals. Our results show that, under biologically plausible conditions, the reflex circuitry thus obtained is consistent with that identified in relation to the analogous mammalian reflexes. In addition, they show that the reflex connectivity obtained depends on the morphology of the musculoskeletal system as well as on the environment that it is embedded in.

Morphological computation for adaptive behavior and cognition

Traditionally, in robotics, artificial intelligence and neuroscience, there has been a focus on the study of the control or the neural system itself. Recently there has been an increasing interest in the notion of embodiment not only in robotics and artificial intelligence, but also in the neurosciences, psychology and philosophy. In this paper, we introduce the notion of morphological computation, and demonstrate how it can be exploited on the one hand for designing intelligent, adaptive robotic systems, and on the other hand for understanding natural systems. While embodiment has often been used in its trivial meaning, i.e. "intelligence requires a body", the concept has deeper and more important implications, concerned with the relation between physical and information (neural, control) processes. Morphological computation is about connecting body, brain and environment. A number of case studies are presented to illustrate the concept. We conclude with some speculations about potential lessons for neuroscience and robotics. © 2006 Elsevier B.V. All rights reserved.

Behavior learning of a face robot based on the characteristics of human instruction

Computer simulation experiments were performed to examine the effectiveness of OR- and comparative-reinforcement learning algorithms. In the simulation, human rewards were given as +1 and -1. Two models of human instruction that determine which reward is to be given in every step of a human instruction were used. Results show that human instruction may have a possibility of including both model-A and model-B characteristics, and it can be expected that the comparative-reinforcement learning algorithm is more effective for learning by human instructions.

Inhibition of the predatory activity of the copepod Mesocyclops notius by Microcystis spp.

The feeding rate of the copepod Mesocyclops notius on two species of cladocerans (Daphnia carinata and Ceriodaphnia cornuta) decreased with increasing environmental concentration of 64-122 mum colonial Microcystis spp. Rate of copepod feeding on Moina micrura was unaffected by the presence of Microcystis spp. Mesocyclops notius rarely preyed on Diaphanosoma brachyurum.

Effects of several organic acids on the feeding behavior of Tilapia nilotica

Automatic recording of the frequency of feeding 'bites' was used to evaluate the effects of several organic acids (citric, metacectonic, lactic, acetic, and oxalic) on the stimulatory feeding behavior of Tilapia nilotica . Some of these acids are added to food stocks to retard spoilage. The results showed that citric acid at a concentration of 10(-2) to 10(-6) m, metacetonic acid at 10(-4) to 10(-6) m, and lactic acid at 10(-2) to 10(-5) m stimulated feeding. Fish tended to avoid metacetonic acid at 10(-3) m and acetic acid at 10(-3) m. Acetic acid at 10(-5) m and oxalic acid at 10(-6) m had no significant effects on fish feeding.

Comparative ontogenetic behavior and migration of kaluga, Huso dauricus, and Amur sturgeon, Acipenser schrenckii, from the Amur River

We conducted laboratory experiments with kaluga, Huso dauricus, and Amur sturgeon, Acipenser schrenckii, to develop a conceptual model of early behavior. We daily observed embryos (first life phase after hatching) and larvae (period initiating exogenous feeding) to day-30 (late larvae) for preference of bright habitat and cover, swimming distance above the bottom, up- and downstream movement, and diel activity. Day-0 embryos of both species strongly preferred bright, open habitat and initiated a strong, downstream migration that lasted 4 days (3 day peak) for kaluga and 3 days (2 day peak) for Amur sturgeon. Kaluga migrants swam far above the bottom (150 cm) on only 1 day and moved day and night; Amur sturgeon migrants swam far above the bottom (median 130 cm) during 3 days and were more nocturnal than kaluga. Post-migrant embryos of both species moved day and night, but Amur sturgeon used dark, cover habitat and swam closer to the bottom than kaluga. The larva period of both species began on day 7 (cumulative temperature degree-days, 192.0 for kaluga and 171.5 for Amur sturgeon). Larvae of both species preferred open habitat. Kaluga larvae strongly preferred bright habitat, initially swam far above the bottom (median 50-105 cm), and migrated downstream at night during days 10-16 (7-day migration). Amur sturgeon larvae strongly avoided illumination, had a mixed response to white substrate, swam 20-30 cm above the bottom during most days, and during days 12-34 (most of the larva period) moved downstream mostly at night (23-day migration). The embryo-larva migration style of the two species likely shows convergence of non-related species for a common style in response to environmental selection in the Amur River. The embryo-larva migration style of Amur sturgeon is unique among Acipenser yet studied.

Observations on behavior and ecology of the Yangtze finless porpoise (Neophocaena phocaenoides asiaeorientalis) group at Tian-e-Zhou Oxbow of the Yangtze River

The authors made 39 surveys (a total of 161 days) in the Tian-e-Zhou Oxbow of the Yangtze River, China, for observing 13 Yangtze finless porpoises (Neophocaena phocaenoides asiaeorientalis) captured from the main stream of the Yangtze River and 7 juveniles born in the oxbow from January 1997 to July 2000. The animals were usually divided into several "core" groups and moved around in shallow, muddy-bottom areas with the largest individual in the lead. Each core group was composed of 2-3 animals (either 2 adults, 1 adult and 1 juvenile, 2 adults and 1 juvenile, or 2 adults and 1 calf). Newly-released animals joined the other animals first, and then reorganized their own groups one or two days later. Average breath interval was 34.4 s (+/- s.d. 4.39) for individuals in the group. The animals mated from May through June and gave birth during the second and last ten days of April of the next year. The gestation period was estimated as 310 - 320 days. Calves over 5 months old began to eat small fish. The distance of calves swimming apart from their suspected mothers increased each month. These findings will help in the management of the reserve to protect this unique freshwater porpoise.

Comparative investigation on spindle behavior and MPF activity changes during oocyte maturation between gynogenetic and amphimictic crucian carp

The spindle behavior and MPF activity changes in the progression of oocyte maturation were investigated and compared with cytological observation and kinase assay between gynogenetic silver crucian carp and amphimictic colored crucian carp. MPF activity was measured by using histone I-Il as phosphorylation substrate. There were two similar oscillatory MPF kinase activity changes during oocyte maturation in two kinds of fishes with different reproductive modes, but there existed some subtle difference between them. The subtle difference was that the first peak of MPF kinase activity was kept to a longer-lasting time in the gynogenetic silver crucian carp than in the amphimictic colored crucian carp. It was suggested that the difference may be related to the spindle behavior changes, such as tripolar spindle formation and spindle rearrangement in the gynogenetic crucian carp.

INFLUENCE OF LINEAR ALKYLBENZENE SULFONATE (LAS) AS ORGANIC COSOLVENT ON LEACHING BEHAVIOR OF PCDD/FS FROM FLY-ASH AND SOIL

The leaching of polychlorinated dibenzo-p-dioxins and dibenzofurans (PCDD/Fs) was measured in soil and standard fly ash column eluted with pure water and linear alkylbenzene sulfonate (LAS)- water. The data obtained were used to evaluate the leachability of PCDD/Fs from waste dump like incineration residual slag and fly ash deposition. The leaching rate was shown to be increased significantly by using LAS water. The leachate contents of PCDD/Fs were above their known water solubility. Concentration of PCDD/Fs in the leachates as well as the relative leaching (calculated on the fly ash content) increased with increasing chlorinating degree and decreasing water solubility. LAS above the critical micelle concentration (CMC) probably enhances PCDD/Fs solubility.