Stochastic control strategies and adaptive critic methods

Adaptive critic methods have common roots as generalizations of dynamic programming for neural reinforcement learning approaches. Since they approximate the dynamic programming solutions, they are potentially suitable for learning in noisy, nonlinear and nonstationary environments. In this study, a novel probabilistic dual heuristic programming (DHP) based adaptive critic controller is proposed. Distinct to current approaches, the proposed probabilistic (DHP) adaptive critic method takes uncertainties of forward model and inverse controller into consideration. Therefore, it is suitable for deterministic and stochastic control problems characterized by functional uncertainty. Theoretical development of the proposed method is validated by analytically evaluating the correct value of the cost function which satisfies the Bellman equation in a linear quadratic control problem. The target value of the critic network is then calculated and shown to be equal to the analytically derived correct value.

Static, dynamic, and adaptive heterogeneity in distributed smart camera networks

We study heterogeneity among nodes in self-organizing smart camera networks, which use strategies based on social and economic knowledge to target communication activity efficiently. We compare homogeneous configurations, when cameras use the same strategy, with heterogeneous configurations, when cameras use different strategies. Our first contribution is to establish that static heterogeneity leads to new outcomes that are more efficient than those possible with homogeneity. Next, two forms of dynamic heterogeneity are investigated: nonadaptive mixed strategies and adaptive strategies, which learn online. Our second contribution is to show that mixed strategies offer Pareto efficiency consistently comparable with the most efficient static heterogeneous configurations. Since the particular configuration required for high Pareto efficiency in a scenario will not be known in advance, our third contribution is to show how decentralized online learning can lead to more efficient outcomes than the homogeneous case. In some cases, outcomes from online learning were more efficient than all other evaluated configuration types. Our fourth contribution is to show that online learning typically leads to outcomes more evenly spread over the objective space. Our results provide insight into the relationship between static, dynamic, and adaptive heterogeneity, suggesting that all have a key role in achieving efficient self-organization.

Automatized Generation of Metadata and Calibrated Test Item Banks for E-learning

Report published in the Proceedings of the National Conference on "Education in the Information Society", Plovdiv, May, 2013

Automatized Generation of Personalized Learning Paths through Aspects in Multidimensional Spaces

Report published in the Proceedings of the National Conference on "Education and Research in the Information Society", Plovdiv, May, 2016

The development, application, and implications of a strategy for reflective learning from experience

The problem on which this study focused was individuals' reduced capacity to respond to change and to engage in innovative learning when their reflective learning skills are limited. In this study, the preceding problem was addressed by two primary questions: To what degree can mastery of a strategy for reflective learning be facilitated as a part of an academic curriculum for professional practitioners? What impact will mastery of this strategy have on the learning style and adaptive flexibility of adult learners? The focus of the study was a direct application of human resource development technology in the professional preparation of teachers. The background of the problem in light of changing global paradigms and educational action orientations was outlined and a review of the literature was provided. Roots of thought for two key concepts (i.e., learning to learn from experience and meaningful reflection in learning) were traced. Reflective perspectives from the work of eight researchers were compared. A meta-model of learning from experience drawn from the literature served as a conceptual framework for the study. A strategy for reflective learning developed from this meta-model was taught to 109 teachers-in-training at Florida International University in Miami, Florida. Kolb's Adaptive Style Inventory and Learning Style Inventory were administered to the treatment group and to two control groups taught by the same professor. Three research questions and fourteen hypotheses guided data analysis. Qualitative review of 1565 personal documents generated by the treatment group indicated that 77 students demonstrated "double-loop" learning, going beyond previously established limits to perception, understanding, or action. The mean score for depth of reflection indicated "single-loop" learning with "reflection-in-action" present. The change in the mean score for depth of reflection from the beginning to end of the study was statistically significant (p $<$.05). On quantitative measures of adaptive flexibility and learning style, with two exceptions, there were no significant differences noted between treatment and control groups on pre-test to post-test differences and on post-test mean scores adjusted for pre-test responses and demographic variables. Conclusions were drawn regarding treatment, instrumentation, and application of the strategy and the meta-model. Implications of the strategy and the meta-model for research, for education, for human resource development, for professional practice, and for personal growth were suggested. Qualitative training materials and Kolb's instruments were provided in the appendices.

Pavement performance prediction using adaptive logic networks

Pavement performance is one of the most important components of the pavement management system. Prediction of the future performance of a pavement section is important in programming maintenance and rehabilitation needs. Models for predicting pavement performance have been developed on the basis of traffic and age. The purpose of this research is to extend the use of a relatively new approach to performance prediction in pavement performance modeling using adaptive logic networks (ALN). Adaptive logic networks have recently emerged as an effective alternative to artificial neural networks for machine learning tasks. ^ The ALN predictive methodology is applicable to a wide variety of contexts including prediction of roughness based indices, composite rating indices and/or individual pavement distresses. The ALN program requires key information about a pavement section, including the current distress indexes, pavement age, climate region, traffic and other variables to predict yearly performance values into the future. ^ This research investigates the effect of different learning rates of the ALN in pavement performance modeling. It can be used at both the network and project level for predicting the long term performance of a road network. Results indicate that the ALN approach is well suited for pavement performance prediction modeling and shows a significant improvement over the results obtained from other artificial intelligence approaches. ^

Noradrenergic modulatoins of odor learning and odor representation in the rat

How experience alters neuronal ensemble dynamics and how locus coeruleus-mediated norepinephrine release facilitates memory formation in the brain are the topics of this thesis. Here we employed a visualization technique, cellular compartment analysis of temporal activity by fluorescence in situ hybridization (catFISH), to assess activation patterns of neuronal ensembles in the olfactory bulb (OB) and anterior piriform cortex (aPC) to repeated odor inputs. Two associative learning models were used, early odor preference learning in rat pups and adult rat go-no-go odor discrimination learning. With catFISH of an immediate early gene, Arc, we showed that odor representation in the OB and aPC was sparse (~5-10%) and widely distributed. Odor associative learning enhanced the stability of the rewarded odor representation in the OB and aPC. The stable component, indexed by the overlap between the two ensembles activated by the rewarded odor at two time points, increased from ~25% to ~50% (p = 0.004-1.43E⁻4; Chapter 3 and 4). Adult odor discrimination learning promoted pattern separation between rewarded and unrewarded odor representations in the aPC. The overlap between rewarded and unrewarded odor representations reduced from ~25% to ~14% (p = 2.28E⁻⁵). However, learning an odor mixture as a rewarded odor increased the overlap of the component odor representations in the aPC from ~23% to ~44% (p = 0.010; Chapter 4). Blocking both α- and β-adrenoreceptors in the aPC prevented highly similar odor discrimination learning in adult rats, and reduced OB mitral and granule ensemble stability to the rewarded odor. Similar treatment in the OB only slowed odor discrimination learning. However, OB adrenoceptor blockade disrupted pattern separation and ensemble stability in the aPC when the rats demonstrated deficiency in discrimination (Chapter 5). In another project, the role of α₂-adrenoreceptors in the OB during early odor preference learning was studied. OB α2-adrenoceptor activation was necessary for odor learning in rat pups. α₂-adrenoceptor activation was additive with β-adrenoceptor mediated signalling to promote learning (Chapter 2). Together, these experiments suggest that odor representations are highly adaptive at the early stages of odor processing. The OB and aPC work in concert to support odor learning and top-down adrenergic input exerts a powerful modulation on both learning and odor representation.

Designing an adaptive salutogenic care environment

Humans are profoundly affected by the surroundings which they inhabit. Environmental psychologists have produced numerous credible theories describing optimal human environments, based on the concept of congruence or “fit” (1, 2). Lack of person/environment fit can lead to stress-related illness and lack of psychosocial well-being (3). Conversely, appropriately designed environments can promote wellness (4) or “salutogenesis” (5). Increasingly, research in the area of Evidence-Based Design, largely concentrated in the area of healthcare architecture, has tended to bear out these theories (6). Patients and long-term care residents, because of injury, illness or physical/ cognitive impairment, are less likely to be able to intervene to modify their immediate environment, unless this is designed specifically to facilitate their particular needs. In the context of care settings, detailed design of personal space therefore takes on enormous significance. MyRoom conceptualises a personalisable room, utilising sensoring and networked computing to enable the environment to respond directly and continuously to the occupant. Bio-signals collected and relayed to the system will actuate application(s) intended to positively influence user well-being. Drawing on the evidence base in relation to therapeutic design interventions (7), real-time changes in ambient lighting, colour, image, etc. respond continuously to the user’s physiological state, optimising congruence. Based on research evidence, consideration is also given to development of an application which uses natural images (8). It is envisaged that actuation will require machine-learning based on interpretation of data gathered by sensors; sensoring arrangements may vary depending on context and end-user. Such interventions aim to reduce inappropriate stress/ provide stimulation, supporting both instrumental and cognitive tasks.

Is Our Students Learning - QuestionMark: Extending E-Assessment Systems for Adapative Marking and Feedback

Our key contribution is a flexible, automated marking system that adds desirable functionality to existing E-Assessment systems. In our approach, any given E-Assessment system is relegated to a data-collection mechanism, whereas marking and the generation and distribution of personalised per-student feedback is handled separately by our own system. This allows content-rich Microsoft Word feedback documents to be generated and distributed to every student simultaneously according to a per-assessment schedule.

The feedback is adaptive in that it corresponds to the answers given by the student and provides guidance on where they may have gone wrong. It is not limited to simple multiple choice which are the most prescriptive question type offered by most E-Assessment Systems and as such most straightforward to mark consistently and provide individual per-alternative feedback strings. It is also better equipped to handle the use of mathematical symbols and images within the feedback documents which is more flexible than existing E-Assessment systems, which can only handle simple text strings.

As well as MCQs the system reliably and robustly handles Multiple Response, Text Matching and Numeric style questions in a more flexible manner than Questionmark: Perception and other E-Assessment Systems. It can also reliably handle multi-part questions where the response to an earlier question influences the answer to a later one and can adjust both scoring and feedback appropriately.

New question formats can be added at any time provided a corresponding marking method conforming to certain templates can also be programmed. Indeed, any question type for which a programmatic method of marking can be devised may be supported by our system. Furthermore, since the student’s response to each is question is marked programmatically, our system can be set to allow for minor deviations from the correct answer, and if appropriate award partial marks.

Directed Selection using Reinforcement Learning for the Examination Timetabling Problem

Traditional heuristic approaches to the Examination Timetabling Problem normally utilize a stochastic method during Optimization for the selection of the next examination to be considered for timetabling within the neighbourhood search process. This paper presents a technique whereby the stochastic method has been augmented with information from a weighted list gathered during the initial adaptive construction phase, with the purpose of intelligently directing examination selection. In addition, a Reinforcement Learning technique has been adapted to identify the most effective portions of the weighted list in terms of facilitating the greatest potential for overall solution improvement. The technique is tested against the 2007 International Timetabling Competition datasets with solutions generated within a time frame specified by the competition organizers. The results generated are better than those of the competition winner in seven of the twelve examinations, while being competitive for the remaining five examinations. This paper also shows experimentally how using reinforcement learning has improved upon our previous technique.

A Biocultural Examination of Student Learning Behaviors in Large Undergraduate Lectures

Thesis (Ph.D.)--University of Washington, 2016-08

Learning from Text: Examining Teacher Thinking and Practice in AP Environmental Science

Thesis (Ph.D.)--University of Washington, 2016-08

Optimising plant bed performance through adaptive research with the Australian sweetpotato industry

A key driver of Australian sweetpotato productivity improvements and consumer demand has been industry adoption of disease-free planting material systems. On a farm isolated from main Australian sweetpotato areas, virus-free germplasm is annually multiplied, with subsequent 'pathogen-tested' (PT) sweetpotato roots shipped to commercial Australian sweetpotato growers. They in turn plant their PT roots into specially designated plant beds, commencing in late winter. From these beds, they cut sprouts as the basis for their commercial fields. Along with other intense agronomic practices, this system enables Australian producers to achieve worldRSQUOs highest commercial yields (per hectare) of premium sweetpotatoes. Their industry organisation, ASPG (Australian Sweetpotato Growers Inc.), has identified productivity of mother plant beds as a key driver of crop performance. Growers and scientists are currently collaborating to investigate issues such as catastrophic plant beds losses; optimisation of irrigation and nutrient addition; rapidity and uniformity of initial plant bed harvests; optimal plant bed harvest techniques; virus re-infection of plant beds; and practical longevity of plant beds. A survey of 50 sweetpotato growers in Queensland and New South Wales identified a substantial diversity in current plant bed systems, apparently influenced by growing district, scale of operation, time of planting, and machinery/labour availability. Growers identified key areas for plant bed research as: optimising the size and grading specifications of PT roots supplied for the plant beds; change in sprout density, vigour and performance through sequential cuttings of the plant bed; optimal height above ground level to cut sprouts to maximise commercial crop and plant bed performance; and use of structures and soil amendments in plant bed systems. Our ongoing multi-disciplinary research program integrates detailed agronomic experiments, grower adaptive learning sites, product quality and consumer research, to enhance industry capacity for inspired innovation and commercial, sustainable practice change.

Adaptive regulation or governmentality: patient safety and the changing regulation of medicine

Studies from across the world have shown that clinical mistakes are a major threat to the safety of patient care (World Health Organisation 2004). For the National Health Service (NHS) of England and Wales it is estimated that one in ten hospital patients experience some form of error, and each year these cost the service over £2billion in remedial care (Department of Health 2000). Unsurprisingly, ‘patient safety’ is now a major international health policy priority, questioning the efficacy of existing regulatory practices and proposing a new ethos of learning. Within England and Wales, the National Patient Safety Agency (NPSA) has been created to lead policy development and champion service-wide learning, whilst throughout the NHS the National Reporting and Learning System (NRLS) has been introduced to enable this learning (NPSA 2003). This paper investigates the extent to which, in seeking to better manage the threats to patient safety, this policy agenda represents a transition in medical regulation.

An adaptive neighborhood search algorithm for optimizing stochastic mining complexes

Les métaheuristiques sont très utilisées dans le domaine de l'optimisation discrète. Elles permettent d’obtenir une solution de bonne qualité en un temps raisonnable, pour des problèmes qui sont de grande taille, complexes, et difficiles à résoudre. Souvent, les métaheuristiques ont beaucoup de paramètres que l’utilisateur doit ajuster manuellement pour un problème donné. L'objectif d'une métaheuristique adaptative est de permettre l'ajustement automatique de certains paramètres par la méthode, en se basant sur l’instance à résoudre. La métaheuristique adaptative, en utilisant les connaissances préalables dans la compréhension du problème, des notions de l'apprentissage machine et des domaines associés, crée une méthode plus générale et automatique pour résoudre des problèmes. L’optimisation globale des complexes miniers vise à établir les mouvements des matériaux dans les mines et les flux de traitement afin de maximiser la valeur économique du système. Souvent, en raison du grand nombre de variables entières dans le modèle, de la présence de contraintes complexes et de contraintes non-linéaires, il devient prohibitif de résoudre ces modèles en utilisant les optimiseurs disponibles dans l’industrie. Par conséquent, les métaheuristiques sont souvent utilisées pour l’optimisation de complexes miniers. Ce mémoire améliore un procédé de recuit simulé développé par Goodfellow & Dimitrakopoulos (2016) pour l’optimisation stochastique des complexes miniers stochastiques. La méthode développée par les auteurs nécessite beaucoup de paramètres pour fonctionner. Un de ceux-ci est de savoir comment la méthode de recuit simulé cherche dans le voisinage local de solutions. Ce mémoire implémente une méthode adaptative de recherche dans le voisinage pour améliorer la qualité d'une solution. Les résultats numériques montrent une augmentation jusqu'à 10% de la valeur de la fonction économique.