Cortical substrates for exploratory decisions in humans.

Decision making in an uncertain environment poses a conflict between the opposing demands of gathering and exploiting information. In a classic illustration of this 'exploration-exploitation' dilemma, a gambler choosing between multiple slot machines balances the desire to select what seems, on the basis of accumulated experience, the richest option, against the desire to choose a less familiar option that might turn out more advantageous (and thereby provide information for improving future decisions). Far from representing idle curiosity, such exploration is often critical for organisms to discover how best to harvest resources such as food and water. In appetitive choice, substantial experimental evidence, underpinned by computational reinforcement learning (RL) theory, indicates that a dopaminergic, striatal and medial prefrontal network mediates learning to exploit. In contrast, although exploration has been well studied from both theoretical and ethological perspectives, its neural substrates are much less clear. Here we show, in a gambling task, that human subjects' choices can be characterized by a computationally well-regarded strategy for addressing the explore/exploit dilemma. Furthermore, using this characterization to classify decisions as exploratory or exploitative, we employ functional magnetic resonance imaging to show that the frontopolar cortex and intraparietal sulcus are preferentially active during exploratory decisions. In contrast, regions of striatum and ventromedial prefrontal cortex exhibit activity characteristic of an involvement in value-based exploitative decision making. The results suggest a model of action selection under uncertainty that involves switching between exploratory and exploitative behavioural modes, and provide a computationally precise characterization of the contribution of key decision-related brain systems to each of these functions.

Temporal difference models describe higher-order learning in humans.

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Effects of ethanol on vehicle energy efficiency and implications on ethanol life-cycle greenhouse gas analysis.

Bioethanol is the world's largest-produced alternative to petroleum-derived transportation fuels due to its compatibility within existing spark-ignition engines and its relatively mature production technology. Despite its success, questions remain over the greenhouse gas (GHG) implications of fuel ethanol use with many studies showing significant impacts of differences in land use, feedstock, and refinery operation. While most efforts to quantify life-cycle GHG impacts have focused on the production stage, a few recent studies have acknowledged the effect of ethanol on engine performance and incorporated these effects into the fuel life cycle. These studies have broadly asserted that vehicle efficiency increases with ethanol use to justify reducing the GHG impact of ethanol. These results seem to conflict with the general notion that ethanol decreases the fuel efficiency (or increases the fuel consumption) of vehicles due to the lower volumetric energy content of ethanol when compared to gasoline. Here we argue that due to the increased emphasis on alternative fuels with drastically differing energy densities, vehicle efficiency should be evaluated based on energy rather than volume. When done so, we show that efficiency of existing vehicles can be affected by ethanol content, but these impacts can serve to have both positive and negative effects and are highly uncertain (ranging from -15% to +24%). As a result, uncertainties in the net GHG effect of ethanol, particularly when used in a low-level blend with gasoline, are considerably larger than previously estimated (standard deviations increase by >10% and >200% when used in high and low blends, respectively). Technical options exist to improve vehicle efficiency through smarter use of ethanol though changes to the vehicle fleets and fuel infrastructure would be required. Future biofuel policies should promote synergies between the vehicle and fuel industries in order to maximize the society-wise benefits or minimize the risks of adverse impacts of ethanol.

Idea Competitions under scrutiny: Acquisition, intelligence or public relations mechanism?

Idea Competitions (ICs) are becoming a popular mechanism chosen by firms to perform Open Innovation. They are a way to engage with external sources of knowledge such as individual entrepreneurs and small firms who are asked to submit ideas and compete for a prize. However, little is known about the success of ICs as acquisition mechanisms. The researchers conducted interviews in five multinational companies to evaluate the effects of using ICs as an acquisition mechanism. Although still preliminary, the results of this study show that the success of ICs as an acquisition mechanism remains uncertain because their output (i.e. the number of ideas acquired) is often low compared to the input (i.e. the number of ideas submitted) and effort required to run them (e.g. to vet ideas). Across the cases observed, ICs appear to be more successful at identifying and acquiring early-stage ideas, particularly those outside the current business focus. The study shows that ICs deliver other functional benefits such as improved intelligence and public relations and that these need to be considered as part of the evaluation of the IC's success. The paper concludes by discussing the conditions in which ICs are implemented and the implications for Open Innovation theory. © 2013 Elsevier Inc.

Coupling model uncertainty for coupled rainfall/runoff and surface water quality models in river problems

Coupled hydrology and water quality models are an important tool today, used in the understanding and management of surface water and watershed areas. Such problems are generally subject to substantial uncertainty in parameters, process understanding, and data. Component models, drawing on different data, concepts, and structures, are affected differently by each of these uncertain elements. This paper proposes a framework wherein the response of component models to their respective uncertain elements can be quantified and assessed, using a hydrological model and water quality model as two exemplars. The resulting assessments can be used to identify model coupling strategies that permit more appropriate use and calibration of individual models, and a better overall coupled model response. One key finding was that an approximate balance of water quality and hydrological model responses can be obtained using both the QUAL2E and Mike11 water quality models. The balance point, however, does not support a particularly narrow surface response (or stringent calibration criteria) with respect to the water quality calibration data, at least in the case examined here. Additionally, it is clear from the results presented that the structural source of uncertainty is at least as significant as parameter-based uncertainties in areal models. © 2012 John Wiley & Sons, Ltd.

Predicting the response of structures to transient shock loading.

This work concerns the prediction of the response of an uncertain structure to a load of short duration. Assuming an ensemble of structures with small random variations about a nominal form, a mean impulse response can be found using only the modal density of the structure. The mean impulse response turns out to be the same as the response of an infinite structure: the response is calculated by taking into account the direct field only, without reflections. Considering the short duration of an impulsive loading, the approach is reasonable before the effect of the reverberant field becomes important. The convolution between the mean impulse response and the shock loading is solved in discrete time to calculate the response at the driving point and at remote points. Experimental and numerical examples are presented to validate the theory presented for simple structures such as beams, plates, and cylinders.

Efficient statistical electromagnetic analysis in reverberant environments

This paper presents a statistical approach to the electromagnetic analysis of a system that lies within a reverberant cavity that has random or uncertain properties. The need to solve Maxwell's equations within the cavity is avoided by employing a relation known as the diffuse field reciprocity principle, which leads directly to the ensemble mean squared response of the system; all that is required is the impedance matrix of the system associated with radiation into infinite space. The general theoretical approach is presented, and the analysis is then applied to a five-cable bundle in a reverberation room © 2013 EMC Europe Foundation.

Efficient parametric uncertainty analysis within the hybrid Finite Element/Statistical Energy Analysis method

This paper is concerned with the development of efficient algorithms for propagating parametric uncertainty within the context of the hybrid Finite Element/Statistical Energy Analysis (FE/SEA) approach to the analysis of complex vibro-acoustic systems. This approach models the system as a combination of SEA subsystems and FE components; it is assumed that the FE components have fully deterministic properties, while the SEA subsystems have a high degree of randomness. The method has been recently generalised by allowing the FE components to possess parametric uncertainty, leading to two ensembles of uncertainty: a non-parametric one (SEA subsystems) and a parametric one (FE components). The SEA subsystems ensemble is dealt with analytically, while the effect of the additional FE components ensemble can be dealt with by Monte Carlo Simulations. However, this approach can be computationally intensive when applied to complex engineering systems having many uncertain parameters. Two different strategies are proposed: (i) the combination of the hybrid FE/SEA method with the First Order Reliability Method which allows the probability of the non-parametric ensemble average of a response variable exceeding a barrier to be calculated and (ii) the combination of the hybrid FE/SEA method with Laplace's method which allows the evaluation of the probability of a response variable exceeding a limit value. The proposed approaches are illustrated using two built-up plate systems with uncertain properties and the results are validated against direct integration, Monte Carlo simulations of the FE and of the hybrid FE/SEA models. © 2013 Elsevier Ltd.

Bayesian parameter identification for vibro-acoustic models with nonparametric uncertainty

Vibration and acoustic analysis at higher frequencies faces two challenges: computing the response without using an excessive number of degrees of freedom, and quantifying its uncertainty due to small spatial variations in geometry, material properties and boundary conditions. Efficient models make use of the observation that when the response of a decoupled vibro-acoustic subsystem is sufficiently sensitive to uncertainty in such spatial variations, the local statistics of its natural frequencies and mode shapes saturate to universal probability distributions. This holds irrespective of the causes that underly these spatial variations and thus leads to a nonparametric description of uncertainty. This work deals with the identification of uncertain parameters in such models by using experimental data. One of the difficulties is that both experimental errors and modeling errors, due to the nonparametric uncertainty that is inherent to the model type, are present. This is tackled by employing a Bayesian inference strategy. The prior probability distribution of the uncertain parameters is constructed using the maximum entropy principle. The likelihood function that is subsequently computed takes the experimental information, the experimental errors and the modeling errors into account. The posterior probability distribution, which is computed with the Markov Chain Monte Carlo method, provides a full uncertainty quantification of the identified parameters, and indicates how well their uncertainty is reduced, with respect to the prior information, by the experimental data. © 2013 Taylor & Francis Group, London.

Molecular systematics of medusae in the genus Craspedacusta (Cnidaria: Hydrozoa: Limnomedusae) in China with the reference to the identity of species

The objective of this study was to illustrate the phylogenetic relationship of the species in the genus Craspedacusta in China. The medusae samples were collected at 28 localities in China representing seven described species with their entire ITS region (the contiguous sequences of ITS-1, 5.8S and ITS-2 rDNA) rDNA sequences cloned. Among the 28 samples, the range of sequence variation in the complete ITS and 5.8S region was between 0 and 36.2%. Three main clades were revealed by both maximum likelihood and neighbour-joining trees, with sequence difference of 0-0.9, 0-3.7 and 0.1-1.5% in the three clades. The nesting of C. xinyangensis representatives within C. sowerbii, C. brevinema within C. sinensis and C. sichuanensis within C. kiatingi is strongly supported, with interspecific sequence divergence of 0-0.9, 0.1-1.4 and 0.0-0.4%, respectively. Thus, it is suggested that C. xinyangensis should be the synonym of C. sowerbii, C. sichuanensis the synonym of C. kiatingi and C. brevinema the synonym of C. sinensis. However, the taxonomic status of C. ziguiensis is still uncertain. According to the tree topology, C. kiatingi was closer to C. sowerbii than to C. sinensis. Craspedacusta sinensis was the most genetically distinct from distance matrix values, and located at the base of the phylogenetic trees, so it can be speculated that the C. sinensis may be the ancestral form in the genus Craspedacusta.

Aberration correction for free space optical communications using rectangular zernike modal wavefront sensing

A time multiplexed rectangular Zernike modal wavefront sensor based on a nematic phase-only liquid crystal spatial light modulator and specially designed for a high power two-electrode tapered laser diode which is a compact and novel free space optical communication source is used in an adaptive beam steering free space optical communication system, enabling the system to have 1.25 GHz modulation bandwidth, 4.6° angular coverage and the capability of sensing aberrations within the system and caused by atmosphere turbulence up to absolute value of 0.15 waves amplitude and correcting them in one correction cycle. Closed-loop aberration correction algorithm can be implemented to provide convergence for larger and time varying aberrations. Improvement of the system signal-to-noise-ratio performance is achieved by aberration correction. To our knowledge, it is first time to use rectangular orthonormal Zernike polynomials to represent balanced aberrations for high power rectangular laser beam in practice. © 2014 IEEE.

Active sensing system with in situ adjustable sensor morphology.

BACKGROUND: Despite the widespread use of sensors in engineering systems like robots and automation systems, the common paradigm is to have fixed sensor morphology tailored to fulfill a specific application. On the other hand, robotic systems are expected to operate in ever more uncertain environments. In order to cope with the challenge, it is worthy of note that biological systems show the importance of suitable sensor morphology and active sensing capability to handle different kinds of sensing tasks with particular requirements. METHODOLOGY: This paper presents a robotics active sensing system which is able to adjust its sensor morphology in situ in order to sense different physical quantities with desirable sensing characteristics. The approach taken is to use thermoplastic adhesive material, i.e. Hot Melt Adhesive (HMA). It will be shown that the thermoplastic and thermoadhesive nature of HMA enables the system to repeatedly fabricate, attach and detach mechanical structures with a variety of shape and size to the robot end effector for sensing purposes. Via active sensing capability, the robotic system utilizes the structure to physically probe an unknown target object with suitable motion and transduce the arising physical stimuli into information usable by a camera as its only built-in sensor. CONCLUSIONS/SIGNIFICANCE: The efficacy of the proposed system is verified based on two results. Firstly, it is confirmed that suitable sensor morphology and active sensing capability enables the system to sense different physical quantities, i.e. softness and temperature, with desirable sensing characteristics. Secondly, given tasks of discriminating two visually indistinguishable objects with respect to softness and temperature, it is confirmed that the proposed robotic system is able to autonomously accomplish them. The way the results motivate new research directions which focus on in situ adjustment of sensor morphology will also be discussed.

Linear multimodal actuation through discrete coupling

Due to technological limitations, robot actuators are often designed for specific tasks with narrow performance goals, whereas a wide range of behaviors is necessary for autonomous robots in uncertain complex environments. In an effort to increase the versatility of actuators, we introduce a new concept of multimodal actuation (MMA) that employs dynamic coupling in the form of clutches and brakes to change its mode of operation. The dynamic coupling allows motors and passive elements such as springs to be engaged and disengaged within a single actuator. We apply the concept to a linear series elastic actuator which uses friction brakes controlled online for the dynamic coupling. With this prototype, we are able to demonstrate several modes of operation including stiff position control, series elastic actuation as well as the possibility to store and release energy in a controlled manner for explosive tasks such as jumping. In this paper, we model the proposed concept of actuation and show a systematic performance analysis of the physical prototype that we developed in our laboratory. © 1996-2012 IEEE.

Linear multi-modal actuation through discrete coupling

Due to technological limitations robot actuators are often designed for specific tasks with narrow performance goals, whereas a wide range of output and behaviours is necessary for robots to operate autonomously in uncertain complex environments. We present a design framework that employs dynamic couplings in the form of brakes and clutches to increase the performance and diversity of linear actuators. The couplings are used to switch between a diverse range of discrete modes of operation within a single actuator. We also provide a design solution for miniaturized couplings that use dry friction to produce rapid switching and high braking forces. The couplings are designed so that once engaged or disengaged no extra energy is consumed. We apply the design framework and coupling design to a linear series elastic actuator (SEA) and show that this relatively simple implementation increases the performance and adds new behaviours to the standard design. Through a number of performance tests we are able to show rapid switching between a high and a low impedance output mode; that the actuator's spring can be charged to produce short bursts of high output power; and that the actuator has additional passive and rigid modes that consume no power once activated. Robots using actuators from this design framework would see a vast increase in their behavioural diversity and improvements in their performance not yet possible with conventional actuator design. © 2012 IEEE.

AI in the 21st century - With historical reflections

The discipline of Artificial Intelligence (AI) was born in the summer of 1956 at Dartmouth College in Hanover, New Hampshire. Half of a century has passed, and AI has turned into an important field whose influence on our daily lives can hardly be overestimated. The original view of intelligence as a computer program - a set of algorithms to process symbols - has led to many useful applications now found in internet search engines, voice recognition software, cars, home appliances, and consumer electronics, but it has not yet contributed significantly to our understanding of natural forms of intelligence. Since the 1980s, AI has expanded into a broader study of the interaction between the body, brain, and environment, and how intelligence emerges from such interaction. This advent of embodiment has provided an entirely new way of thinking that goes well beyond artificial intelligence proper, to include the study of intelligent action in agents other than organisms or robots. For example, it supplies powerful metaphors for viewing corporations, groups of agents, and networked embedded devices as intelligent and adaptive systems acting in highly uncertain and unpredictable environments. In addition to giving us a novel outlook on information technology in general, this broader view of AI also offers unexpected perspectives into how to think about ourselves and the world around us. In this chapter, we briefly review the turbulent history of AI research, point to some of its current trends, and to challenges that the AI of the 21st century will have to face. © Springer-Verlag Berlin Heidelberg 2007.