Learning optimal adaptation strategies in unpredictable motor tasks.

Picking up an empty milk carton that we believe to be full is a familiar example of adaptive control, because the adaptation process of estimating the carton's weight must proceed simultaneously with the control process of moving the carton to a desired location. Here we show that the motor system initially generates highly variable behavior in such unpredictable tasks but eventually converges to stereotyped patterns of adaptive responses predicted by a simple optimality principle. These results suggest that adaptation can become specifically tuned to identify task-specific parameters in an optimal manner.

Investigation of anti-lock braking strategies for heavy goods vehicles

An articulated lorry was instrumented in order to measure its performance in straight-line braking. The trailer was fitted with two interchangeable tandem axle sub-chassis, one with an air suspension and the other with a steel monoleaf four-spring suspension. The brakes were only applied to the trailer axles, which were fitted with anti-lock braking systems (ABS), with the brake torque controlled in response to anticipated locking of the leading axle of the tandem. The vehicle with the air suspension was observed to have significantly better braking performance than the steel suspension, and to generate smaller inter-axle load transfer and smaller vertical dynamic tyre forces. Computer models of the two suspensions were developed, including their brakes and anti-lock systems. The models were found to reproduce most of the important features of the experimental results. It was concluded that the poor braking performance of the steel four-spring suspension was mainly due to interaction between the ABS and inter-axle load transfer effects. The effect of road roughness was investigated and it was found that vehicle stopping distances can increase significantly with increasing road roughness. Two alternative anti-lock braking control strategies were simulated. It was found that independent sensing and actuation of the ABS system on each wheel greatly reduced the difference in stopping distances between the air and steel suspensions. A control strategy based on limiting wheel slip was least susceptible to the effects of road roughness.

Effects of the user model on simulation-based learning of dialogue strategies

Over the past decade, a variety of user models have been proposed for user simulation-based reinforcement-learning of dialogue strategies. However, the strategies learned with these models are rarely evaluated in actual user trials and it remains unclear how the choice of user model affects the quality of the learned strategy. In particular, the degree to which strategies learned with a user model generalise to real user populations has not be investigated. This paper presents a series of experiments that qualitatively and quantitatively examine the effect of the user model on the learned strategy. Our results show that the performance and characteristics of the strategy are in fact highly dependent on the user model. Furthermore, a policy trained with a poor user model may appear to perform well when tested with the same model, but fail when tested with a more sophisticated user model. This raises significant doubts about the current practice of learning and evaluating strategies with the same user model. The paper further investigates a new technique for testing and comparing strategies directly on real human-machine dialogues, thereby avoiding any evaluation bias introduced by the user model. © 2005 IEEE.