Learning to act in uncertain environments : technical perspective

The problem of decision making in an uncertain environment arises in many diverse contexts: deciding whether to keep a hard drive spinning in a net-book; choosing which advertisement to post to a Web site visitor; choosing how many newspapers to order so as to maximize profits; or choosing a route to recommend to a driver given limited and possibly out-of-date information about traffic conditions. All are sequential decision problems, since earlier decisions affect subsequent performance; all require adaptive approaches, since they involve significant uncertainty. The key issue in effectively solving problems like these is known as the exploration/exploitation trade-off: If I am at a cross-roads, when should I go in the most advantageous direction among those that I have already explored, and when should I strike out in a new direction, in the hopes I will discover something better?

Learning the Kernel Matrix with Semi-Definite Programming

Kernel-based learning algorithms work by embedding the data into a Euclidean space, and then searching for linear relations among the embedded data points. The embedding is performed implicitly, by specifying the inner products between each pair of points in the embedding space. This information is contained in the so-called kernel matrix, a symmetric and positive definite matrix that encodes the relative positions of all points. Specifying this matrix amounts to specifying the geometry of the embedding space and inducing a notion of similarity in the input space -- classical model selection problems in machine learning. In this paper we show how the kernel matrix can be learned from data via semi-definite programming (SDP) techniques. When applied to a kernel matrix associated with both training and test data this gives a powerful transductive algorithm -- using the labelled part of the data one can learn an embedding also for the unlabelled part. The similarity between test points is inferred from training points and their labels. Importantly, these learning problems are convex, so we obtain a method for learning both the model class and the function without local minima. Furthermore, this approach leads directly to a convex method to learn the 2-norm soft margin parameter in support vector machines, solving another important open problem. Finally, the novel approach presented in the paper is supported by positive empirical results.

The psychosocial factors influencing aggressive driving behaviour

Many drivers in highly motorised countries believe that aggressive driving is increasing. While the prevalence of the behaviour is difficult to reliably identify, the consequences of on-road aggression can be severe, with extreme cases resulting in property damage, injury and even death. This research program was undertaken to explore the nature of aggressive driving from within the framework of relevant psychological theory in order to enhance our understanding of the behaviour and to inform the development of relevant interventions. To guide the research a provisional ‘working’ definition of aggressive driving was proposed encapsulating the recurrent characteristics of the behaviour cited in the literature. The definition was: “aggressive driving is any on-road behaviour adopted by a driver that is intended to cause physical or psychological harm to another road user and is associated with feelings of frustration, anger or threat”. Two main theoretical perspectives informed the program of research. The first was Shinar’s (1998) frustration-aggression model, which identifies both the person-related and situational characteristics that contribute to aggressive driving, as well as proposing that aggressive behaviours can serve either an ‘instrumental’ or ‘hostile’ function. The second main perspective was Anderson and Bushman’s (2002) General Aggression Model. In contrast to Shinar’s model, the General Aggression Model reflects a broader perspective on human aggression that facilitates a more comprehensive examination of the emotional and cognitive aspects of aggressive behaviour. Study One (n = 48) examined aggressive driving behaviour from the perspective of young drivers as an at-risk group and involved conducting six focus groups, with eight participants in each. Qualitative analyses identified multiple situational and person-related factors that contribute to on-road aggression. Consistent with human aggression theory, examination of self-reported experiences of aggressive driving identified key psychological elements and processes that are experienced during on-road aggression. Participants cited several emotions experienced during an on-road incident: annoyance, frustration, anger, threat and excitement. Findings also suggest that off-road generated stress may transfer to the on-road environment, at times having severe consequences including crash involvement. Young drivers also appeared quick to experience negative attributions about the other driver, some having additional thoughts of taking action. Additionally, the results showed little difference between males and females in the severity of behavioural responses they were prepared to adopt, although females appeared more likely to displace their negative emotions. Following the self-reported on-road incident, evidence was also found of a post-event influence, with females being more likely to experience ongoing emotional effects after the event. This finding was evidenced by ruminating thoughts or distraction from tasks. However, the impact of such a post-event influence on later behaviours or interpersonal interactions appears to be minimal. Study Two involved the quantitative analysis of n = 926 surveys completed by a wide age range of drivers from across Queensland. The study aimed to explore the relationships between the theoretical components of aggressive driving that were identified in the literature review, and refined based on the findings of Study One. Regression analyses were used to examine participant emotional, cognitive and behavioural responses to two differing on-road scenarios whilst exploring the proposed theoretical framework. A number of socio-demographic, state and trait person-related variables such as age, pre-study emotions, trait aggression and problem-solving style were found to predict the likelihood of a negative emotional response such as frustration, anger, perceived threat, negative attributions and the likelihood of adopting either an instrumental or hostile behaviour in response to Scenarios One and Two. Complex relationships were found to exist between the variables, however, they were interpretable based on the literature review findings. Factor analysis revealed evidence supporting Shinar’s (1998) dichotomous description of on-road aggressive behaviours as being instrumental or hostile. The second stage of Study Two used logistic regression to examine the factors that predicted the potentially hostile aggressive drivers (n = 88) within the sample. These drivers were those who indicated a preparedness to engage in direct acts of interpersonal aggression on the road. Young, male drivers 17–24 years of age were more likely to be classified as potentially hostile aggressive drivers. Young drivers (17–24 years) also scored significantly higher than other drivers on all subscales of the Aggression Questionnaire (Buss & Perry, 1992) and on the ‘negative problem orientation’ and ‘impulsive careless style’ subscales of the Social Problem Solving Inventory – Revised (D’Zurilla, Nezu & Maydeu-Olivares, 2002). The potentially hostile aggressive drivers were also significantly more likely to engage in speeding and drink/drug driving behaviour. With regard to the emotional, cognitive and behavioural variables examined, the potentially hostile aggressive driver group also scored significantly higher than the ‘other driver’ group on most variables examined in the proposed theoretical framework. The variables contained in the framework of aggressive driving reliably distinguished potentially hostile aggressive drivers from other drivers (Nagalkerke R2 = .39). Study Three used a case study approach to conduct an in-depth examination of the psychosocial characteristics of n = 10 (9 males and 1 female) self-confessed hostile aggressive drivers. The self-confessed hostile aggressive drivers were aged 24–55 years of age. A large proportion of these drivers reported a Year 10 education or better and average–above average incomes. As a group, the drivers reported committing a number of speeding and unlicensed driving offences in the past three years and extensive histories of violations outside of this period. Considerable evidence was also found of exposure to a range of developmental risk factors for aggression that may have contributed to the driver’s on-road expression of aggression. These drivers scored significantly higher on the Aggression Questionnaire subscales and Social Problem Solving Inventory Revised subscales, ‘negative problem orientation’ and ‘impulsive/careless style’, than the general sample of drivers included in Study Two. The hostile aggressive driver also scored significantly higher on the Barrett Impulsivity Scale – 11 (Patton, Stanford & Barratt, 1995) measure of impulsivity than a male ‘inmate’, or female ‘general psychiatric’ comparison group. Using the Carlson Psychological Survey (Carlson, 1982), the self-confessed hostile aggressive drivers scored equal or higher scores than the comparison group of incarcerated individuals on the subscale measures of chemical abuse, thought disturbance, anti-social tendencies and self-depreciation. Using the Carlson Psychological Survey personality profiles, seven participants were profiled ‘markedly anti-social’, two were profiled ‘negative-explosive’ and one was profiled as ‘self-centred’. Qualitative analysis of the ten case study self-reports of on-road hostile aggression revealed a similar range of on-road situational factors to those identified in the literature review and Study One. Six of the case studies reported off-road generated stress that they believed contributed to the episodes of aggressive driving they recalled. Intense ‘anger’ or ‘rage’ were most frequently used to describe the emotions experienced in response to the perceived provocation. Less frequently ‘excitement’ and ‘fear’ were cited as relevant emotions. Notably, five of the case studies experienced difficulty articulating their emotions, suggesting emotional difficulties. Consistent with Study Two, these drivers reported negative attributions and most had thoughts of aggressive actions they would like to take. Similarly, these drivers adopted both instrumental and hostile aggressive behaviours during the self-reported incident. Nine participants showed little or no remorse for their behaviour and these drivers also appeared to exhibit low levels of personal insight. Interestingly, few incidents were brought to the attention of the authorities. Further, examination of the person-related characteristics of these drivers indicated that they may be more likely to have come from difficult or dysfunctional backgrounds and to have a history of anti-social behaviours on and off the road. The research program has several key theoretical implications. While many of the findings supported Shinar’s (1998) frustration-aggression model, two key areas of difference emerged. Firstly, aggressive driving behaviour does not always appear to be frustration driven, but can also be driven by feelings of excitation (consistent with the tenets of the General Aggression Model). Secondly, while the findings supported a distinction being made between instrumental and hostile aggressive behaviours, the characteristics of these two types of behaviours require more examination. For example, Shinar (1998) proposes that a driver will adopt an instrumental aggressive behaviour when their progress is impeded if it allows them to achieve their immediate goals (e.g. reaching their destination as quickly as possible); whereas they will engage in hostile aggressive behaviour if their path to their goal is blocked. However, the current results question this assertion, since many of the hostile aggressive drivers studied appeared prepared to engage in hostile acts irrespective of whether their goal was blocked or not. In fact, their behaviour appeared to be characterised by a preparedness to abandon their immediate goals (even if for a short period of time) in order to express their aggression. The use of the General Aggression Model enabled an examination of the three components of the ‘present internal state’ comprising emotions, cognitions and arousal and how these influence the likelihood of a person responding aggressively to an on-road situation. This provided a detailed insight into both the cognitive and emotional aspects of aggressive driving that have important implications for the design of relevant countermeasures. For example, the findings highlighted the potential value of utilising Cognitive Behavioural Therapy with aggressive drivers, particularly the more hostile offenders. Similarly, educational efforts need to be mindful of the way that person-related factors appear to influence one’s perception of another driver’s behaviour as aggressive or benign. Those drivers with a predisposition for aggression were more likely to perceive aggression or ‘wrong doing’ in an ambiguous on-road situation and respond with instrumental and/or hostile behaviour, highlighting the importance of perceptual processes in aggressive driving behaviour.

Inexact uniformization method for computing transient distributions of Markov chains

The uniformization method (also known as randomization) is a numerically stable algorithm for computing transient distributions of a continuous time Markov chain. When the solution is needed after a long run or when the convergence is slow, the uniformization method involves a large number of matrix-vector products. Despite this, the method remains very popular due to its ease of implementation and its reliability in many practical circumstances. Because calculating the matrix-vector product is the most time-consuming part of the method, overall efficiency in solving large-scale problems can be significantly enhanced if the matrix-vector product is made more economical. In this paper, we incorporate a new relaxation strategy into the uniformization method to compute the matrix-vector products only approximately. We analyze the error introduced by these inexact matrix-vector products and discuss strategies for refining the accuracy of the relaxation while reducing the execution cost. Numerical experiments drawn from computer systems and biological systems are given to show that significant computational savings are achieved in practical applications.

Towards a transdisciplinary approach to integrating dance in a middle school curriculum

Within Queensland middle schools the implementation of an integrated curriculum has been challenging for many practitioners. In working to enhance dance learning in a Queensland middle school this research has focused on how dance can be integrated using a transdisciplinary approach. The research has investigated and reflected on the teaching and learning strategies used to integrate dance and has identified the key issues and challenges associated with the complex nature of an integrated curriculum in this context. Action research was used to review, plan for and implement integrated curriculum approaches and give insight into the external and internal challenges within the practice. This research has identified challenges associated with sustaining the integrity of dance as a subject area when integration requires designing curricula that go across key learning area boundaries. It has also revealed working within an integrated curriculum requires using common planning principles that focus on the students’ problem solving skills, making connections with the concepts, topics or ideas from the unit of work. The discussion of ways of working highlights a set of values, strategies or attributes a dance teacher can use while working within this middle school context. These include making collaborative partnerships and showing a willingness to work outside your area of expertise. For the school community, it outlines issues for attention and recommendations to assist in implementing dance while using a transdisciplinary approach. These recommended steps include embedding opportunities for teachers to partake in common planning, and time for professional development around transdisciplinary learning and Arts education.

Optimistic linear programming gives logarithmic regret for irreducible MDPs

We present an algorithm called Optimistic Linear Programming (OLP) for learning to optimize average reward in an irreducible but otherwise unknown Markov decision process (MDP). OLP uses its experience so far to estimate the MDP. It chooses actions by optimistically maximizing estimated future rewards over a set of next-state transition probabilities that are close to the estimates, a computation that corresponds to solving linear programs. We show that the total expected reward obtained by OLP up to time T is within C(P) log T of the reward obtained by the optimal policy, where C(P) is an explicit, MDP-dependent constant. OLP is closely related to an algorithm proposed by Burnetas and Katehakis with four key differences: OLP is simpler, it does not require knowledge of the supports of transition probabilities, the proof of the regret bound is simpler, but our regret bound is a constant factor larger than the regret of their algorithm. OLP is also similar in flavor to an algorithm recently proposed by Auer and Ortner. But OLP is simpler and its regret bound has a better dependence on the size of the MDP.

Scaffolding techniques : a teacher training for cooperative learning in Thailand primary education

In general, the benefits of using cooperative learning include academic achievement, communication skills, problem-solving, social skills and student motivation. Yet cooperative learning as a Western educational concept may be ineffective in a different learning system. The study aims to investigate scaffolding techniques for cooperative learning in Thailand primary education. The program was designed to foster Thai primary school teachers’ cooperative learning implementation that includes the basic tenets of cooperative learning and socio-cognitive based learning. Two teachers were invited to participate in this experimental teacher training program for one and a half weeks. Then the teachers implemented a cooperative learning in their mathematics class for six weeks. The data from teacher interview and classroom observation indicated that the both teachers are able to utilise questions to scaffold their students’ engagement in cooperative learning. This initiative study showed that difficulty or failure of implementing cooperative learning in Thailand education may not be derived from cultural difference. The paper discussed the techniques the participant teachers applied with proactive scaffolding, reactive scaffolding and scaffolding questions that can be used to facilitate the implementation of cooperative learning in Thai school.

Can audio-visual speech recognition outperform acoustically enhanced speech recognition in automotive environment?

The use of visual features in the form of lip movements to improve the performance of acoustic speech recognition has been shown to work well, particularly in noisy acoustic conditions. However, whether this technique can outperform speech recognition incorporating well-known acoustic enhancement techniques, such as spectral subtraction, or multi-channel beamforming is not known. This is an important question to be answered especially in an automotive environment, for the design of an efficient human-vehicle computer interface. We perform a variety of speech recognition experiments on a challenging automotive speech dataset and results show that synchronous HMM-based audio-visual fusion can outperform traditional single as well as multi-channel acoustic speech enhancement techniques. We also show that further improvement in recognition performance can be obtained by fusing speech-enhanced audio with the visual modality, demonstrating the complementary nature of the two robust speech recognition approaches.

Stochastic modelling of T cell homeostasis for two competing clonotypes via the master equation

Stochastic models for competing clonotypes of T cells by multivariate, continuous-time, discrete state, Markov processes have been proposed in the literature by Stirk, Molina-París and van den Berg (2008). A stochastic modelling framework is important because of rare events associated with small populations of some critical cell types. Usually, computational methods for these problems employ a trajectory-based approach, based on Monte Carlo simulation. This is partly because the complementary, probability density function (PDF) approaches can be expensive but here we describe some efficient PDF approaches by directly solving the governing equations, known as the Master Equation. These computations are made very efficient through an approximation of the state space by the Finite State Projection and through the use of Krylov subspace methods when evolving the matrix exponential. These computational methods allow us to explore the evolution of the PDFs associated with these stochastic models, and bimodal distributions arise in some parameter regimes. Time-dependent propensities naturally arise in immunological processes due to, for example, age-dependent effects. Incorporating time-dependent propensities into the framework of the Master Equation significantly complicates the corresponding computational methods but here we describe an efficient approach via Magnus formulas. Although this contribution focuses on the example of competing clonotypes, the general principles are relevant to multivariate Markov processes and provide fundamental techniques for computational immunology.

Stochastic regularization method for backward Cauchy problem in Banach spaces

We consider a stochastic regularization method for solving the backward Cauchy problem in Banach spaces. An order of convergence is obtained on sourcewise representative elements.

Entry into Valhalla : contextualising the learning of legal ethics through the use of Second Life machinima

An ability to recognise and resolve ethical dilemmas was identified by the Australian Law Reform Commission as one of the ten fundamental lawyering skills. While the ‘Priestley 11’ list of areas of law required to qualify for legal practice includes ethics and professional responsibility, the commitment to ethics learning in Australian law schools has been far from uniform. The obligation imposed by the Priestley 11 is frequently discharged by a traditional teaching and learning approach involving lectures and/or tutorials and focusing on the content of the formal rules of professional responsibility. However, the effectiveness of such an approach is open to question. Instead, a practical rather than a theoretical approach to the teaching of legal ethics is required. Effective final-year student learning of ethics may be achieved by an approach which engages students, enabling them to appreciate the relevance of what they are learning to the real world and facilitating their transition from study to their working lives. Entry into Valhalla comprises a suite of modules featuring ‘machinima’ (computer-generated imagery) created using the Second Life virtual environment to contextualise otherwise abstract concepts. It provides an engaging learning environment which enables students to obtain an appreciation of ethical responsibility in a real-world context and facilitates understanding and problem-solving ability.

Unusual Event Detection in Crowded Scenes Using Bag of LBPs in spatio-temporal patches

Modelling events in densely crowded environments remains challenging, due to the diversity of events and the noise in the scene. We propose a novel approach for anomalous event detection in crowded scenes using dynamic textures described by the Local Binary Patterns from Three Orthogonal Planes (LBP-TOP) descriptor. The scene is divided into spatio-temporal patches where LBP-TOP based dynamic textures are extracted. We apply hierarchical Bayesian models to detect the patches containing unusual events. Our method is an unsupervised approach, and it does not rely on object tracking or background subtraction. We show that our approach outperforms existing state of the art algorithms for anomalous event detection in UCSD dataset.

Coupling hybrid-game strategies with particle swarm optimisation for multi-objective high lift systems design optimisation

This paper investigates the High Lift System (HLS) application of complex aerodynamic design problem using Particle Swarm Optimisation (PSO) coupled to Game strategies. Two types of optimization methods are used; the first method is a standard PSO based on Pareto dominance and the second method hybridises PSO with a well-known Nash Game strategies named Hybrid-PSO. These optimization techniques are coupled to a pre/post processor GiD providing unstructured meshes during the optimisation procedure and a transonic analysis software PUMI. The computational efficiency and quality design obtained by PSO and Hybrid-PSO are compared. The numerical results for the multi-objective HLS design optimisation clearly shows the benefits of hybridising a PSO with the Nash game and makes promising the above methodology for solving other more complex multi-physics optimisation problems in Aeronautics.

Advanced Computational Intelligence System for inverse aeronautical design optimisation

Abstract—Computational Intelligence Systems (CIS) is one of advanced softwares. CIS has been important position for solving single-objective / reverse / inverse and multi-objective design problems in engineering. The paper hybridise a CIS for optimisation with the concept of Nash-Equilibrium as an optimisation pre-conditioner to accelerate the optimisation process. The hybridised CIS (Hybrid Intelligence System) coupled to the Finite Element Analysis (FEA) tool and one type of Computer Aided Design(CAD) system; GiD is applied to solve an inverse engineering design problem; reconstruction of High Lift Systems (HLS). Numerical results obtained by the hybridised CIS are compared to the results obtained by the original CIS. The benefits of using the concept of Nash-Equilibrium are clearly demonstrated in terms of solution accuracy and optimisation efficiency.

Efficient hybrid-game strategies coupled to evolutionary algorithms for robust multidisciplinary design optimization in aerospace engineering

A number of game strategies have been developed in past decades and used in the fields of economics, engineering, computer science, and biology due to their efficiency in solving design optimization problems. In addition, research in multiobjective and multidisciplinary design optimization has focused on developing a robust and efficient optimization method so it can produce a set of high quality solutions with less computational time. In this paper, two optimization techniques are considered; the first optimization method uses multifidelity hierarchical Pareto-optimality. The second optimization method uses the combination of game strategies Nash-equilibrium and Pareto-optimality. This paper shows how game strategies can be coupled to multiobjective evolutionary algorithms and robust design techniques to produce a set of high quality solutions. Numerical results obtained from both optimization methods are compared in terms of computational expense and model quality. The benefits of using Hybrid and non-Hybrid-Game strategies are demonstrated.