Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

The moving targets algorithm for difficult temporal credit assignment problems

The 'moving targets' algorithm for training recurrent networks is reviewed and applied to a task which demonstrates the ability of this algorithm to use distant contextual information. Some practical difficulties are discussed, especially with regard to the minimization process. Results on performance and computational requirements of several different 2nd-order minimization algorithms are presented for moving target problems.

Task profiling : A task-Based approach to measuring the integration of ICT in the classroom

The measurement of ICT (information and communication technology) integration is emerging as an area of research interest with such systems as Education Queensland including it in their recently released list of research priorities. Studies to trial differing integration measurement instruments have taken place within Australia in the last few years, particularly Western Australia (Trinidad, Clarkson, & Newhouse, 2004; Trinidad, Newhouse & Clarkson, 2005), Tasmania (Fitzallen 2005) and Queensland (Finger, Proctor, & Watson, 2005). This paper will add to these investigations by describing an alternate and original methodological approach which was trialled in a small-scale pilot study conducted jointly by Queensland Catholic Education Commission (QCEC) and the Centre of Learning Innovation, Queensland University of Technology (QUT) in late 2005. The methodology described is based on tasks which, through a process of profiling, can be seen to be artefacts which embody the internal and external factors enabling and constraining ICT integration.

Finding order in chaos : a problem based collaborative assignment for 1300 students across 19 disciplines

The use of collaborative assignments for assessment is a risky undertaking for students and course designers. Yet the benefits, in terms of core learning outcomes, competencies, collaborative sense making and student involvement, suggest that the effort is worthwhile. Formal descriptions and rules do little to ameliorate the perception of risk and increased anxiety by students. (Ryan, 2007). BEB100 Introducing Professional Learning is a faculty-wide foundation unit with over 1300 students from 19 disciplines across the Faculty of the Built Environment and Engineering (“BEE”) at the Queensland University of Technology (“QUT”), Brisbane, Australia. Finding order in chaos outlines the approach and justification, assessment criteria, learning resources, teamwork tools, tutorial management, communication strategies, 2007-09 Student Learning Experience Survey results, annual improvements, findings and outcomes.

Attention and memory bias for body image and health related information using an Emotional Stroop task in a non-clinical sample

It has been proposed that body image disturbance is a form of cognitive bias wherein schemas for self-relevant information guide the selective processing of appearancerelated information in the environment. This threatening information receives disproportionately more attention and memory, as measured by an Emotional Stroop and incidental recall task. The aim of this thesis was to expand the literature on cognitive processing biases in non-clinical males and females by incorporating a number of significant methodological refinements. To achieve this aim, three phases of research were conducted. The initial two phases of research provided preliminary data to inform the development of the main study. Phase One was a qualitative exploration of body image concerns amongst males and females recruited through the general community and from a university. Seventeen participants (eight male; nine female) provided information on their body image and what factors they saw as positively and negatively impacting on their self evaluations. The importance of self esteem, mood, health and fitness, and recognition of the social ideal were identified as key themes. These themes were incorporated as psycho-social measures and Stroop word stimuli in subsequent phases of the research. Phase Two involved the selection and testing of stimuli to be used in the Emotional Stroop task. Six experimental categories of words were developed that reflected a broad range of health and body image concerns for males and females. These categories were high and low calorie food words, positive and negative appearance words, negative emotion words, and physical activity words. Phase Three addressed the central aim of the project by examining cognitive biases for body image information in empirically defined sub-groups. A National sample of males (N = 55) and females (N = 144), recruited from the general community and universities, completed an Emotional Stroop task, incidental memory test, and a collection of psycho-social questionnaires. Sub-groups of body image disturbance were sought using a cluster analysis, which identified three sub-groups in males (Normal, Dissatisfied, and Athletic) and four sub-groups in females (Normal, Health Conscious, Dissatisfied, and Symptomatic). No differences were noted between the groups in selective attention, although time taken to colour name the words was associated with some of the psycho-social variables. Memory biases found across the whole sample for negative emotion, low calorie food, and negative appearance words were interpreted as reflecting the current focus on health and stigma against being unattractive. Collectively these results have expanded our understanding of processing biases in the general community by demonstrating that the processing biases are found within non-clinical samples and that not all processing biases are associated with negative functionality

Final report : task 2 : design guidelines for delivering high quality indoor environments

This report is for one of the four Tasks of the CRC project ‘Regenerating Construction to Enhance Sustainability’. The report specifically addresses Task 2 ‘Design guidelines for delivering high quality indoor environments’.

Final report : task 3: occupant health, wellbeing and productivity

This report presents the results of a study on indoor environment quality (IEQ) and occupant productivity in two buildings that are owned and Occupied by City of Melbourne, and located next to each other in Central Melbourne, Council House 1 (CH1) and Council House 2 (CH2). The impact of a range of relevant IEQ and other parameters on health, wellbeing and productivity of occupants is assessed. The before-and-after case study has demonstrated that the productivity of office building occupants can potentially be enhanced through good building design, and provision of a high quality, healthy, comfortable and functional interior environment, that takes account of basic occupant needs. It has shown that good indoor environment quality is a necessary pre-requisite for enhanced productivity in office buildings, but that broader aspects of overall building and interior design are also important.

Final report : task 4 : waste minimisation in construction

The Regenerating Construction Project for the CRC for Construction Innovation aims to assist in the delivery of demonstrably superior ‘green’ buildings. Components of the project address eco-efficient redesign, achieving a smaller ecological footprint, enhancing indoor environment and minimising waste in design and construction. The refurbishment of Council House 1 for Melbourne City Council provides an opportunity to develop and demonstrate tools that will be of use for commercial building refurbishment generally. It is hoped that the refurbishment will act as an exemplar project to demonstrate environmentally friendly possibilities for office building refurbishment.

Authentic and conventional assessment in Singapore schools : an empirical study of teacher assignments and student work

This is an empirical examination of the quality of teacher assignments and student work in Singapore schools. Using a theoretical framework based on principles of authentic assessment and intellectual quality, two sets of criteria and scoring rubrics were developed for the training of expert teachers to judge the quality of assignments and student work. Following rigorous training, the inter-rater reliability of expert teacher scoring was high. Samples of teacher assignments and student work were collected in English, social studies, mathematics, and science subject areas from a random stratified sample of 30 elementary schools and 29 high schools. For both grade levels, there were significant differences for the authentic intellectual quality of teachers’ assignments by subject area. Likewise, the differences of authentic intellectual quality for student work were significant and varied by subject area. Subject area effect was large. The correlations between the quality of teachers’ assignment tasks and student work were strong and significant at both grade levels. Where teachers set more intellectually demanding tasks, students were more likely to generate work or artefacts judged to be of higher quality. The findings suggest that teacher professional development in authentic intellectual assessment task design can contribute to the improvement of student learning and performance. It is argued that this will be a key requisite of educational systems like Singapore that are seeking to expand pedagogy and student outcomes beyond a focus on factual and rote knowledge.