Local Goals Driven Hierarchical Reinforcement Learning

* This research was partially supported by the Latvian Science Foundation under grant No.02-86d.

Flocking behaviour: Agent-based simulation and hierarchical leadership.

We have studied how leaders emerge in a group as a consequence of interactions among its members. We propose that leaders can emerge as a consequence of a self-organized process based on local rules of dyadic interactions among individuals. Flocks are an example of self-organized behaviour in a group and properties similar to those observed in flocks might also explain some of the dynamics and organization of human groups. We developed an agent-based model that generated flocks in a virtual world and implemented it in a multi-agent simulation computer program that computed indices at each time step of the simulation to quantify the degree to which a group moved in a coordinated way (index of flocking behaviour) and the degree to which specific individuals led the group (index of hierarchical leadership). We ran several series of simulations in order to test our model and determine how these indices behaved under specific agent and world conditions. We identified the agent, world property, and model parameters that made stable, compact flocks emerge, and explored possible environmental properties that predicted the probability of becoming a leader.

Hierarchical Task Network Planning with Common-Sense Reasoning for Multiple-People Behaviour Analysis

Safety on public transport is a major concern for the relevant authorities. We
address this issue by proposing an automated surveillance platform which combines data from video, infrared and pressure sensors. Data homogenisation and integration is achieved by a distributed architecture based on communication middleware that resolves interconnection issues, thereby enabling data modelling. A common-sense knowledge base models and encodes knowledge about public-transport platforms and the actions and activities of passengers. Trajectory data from passengers is modelled as a time-series of human activities. Common-sense knowledge and rules are then applied to detect inconsistencies or errors in the data interpretation. Lastly, the rationality that characterises human behaviour is also captured here through a bottom-up Hierarchical Task Network planner that, along with common-sense, corrects misinterpretations to explain passenger behaviour. The system is validated using a simulated bus saloon scenario as a case-study. Eighteen video sequences were recorded with up to six passengers. Four metrics were used to evaluate performance. The system, with an accuracy greater than 90% for each of the four metrics, was found to outperform a rule-base system and a system containing planning alone.

Characterizing the timing behaviour of power-line communication by means of simulation

Although power-line communication (PLC) is not a new technology, its use to support communication with timing requirements is still the focus of ongoing research. Recently, a new infrastructure was presented, intended for communication using power lines from a central location to geographically dispersed nodes using inexpensive devices. This new infrastructure uses a two-level hierarchical power-line system, together with an IP-based network. Within this infrastructure, in order to provide end-toend communication through the two levels of the powerline system, it is necessary to fully understand the behaviour of the underlying network layers. The masterslave behaviour of the PLC MAC, together with the inherent dynamic topology of power-line networks are important issues that must be fully characterised. Therefore, in this paper we present a simulation model which is being used to study and characterise the behaviour of power-line communication.

Queuing and routing in a hierarchical powerline communication system

Although power-line communication (PLC) is not a new technology, its use to support data communication with timing requirements is still the focus of ongoing research. A new infrastructure intended for communication using power lines from a central location to dispersed nodes using inexpensive devices was presented recently. This new infrastructure uses a two-level hierarchical power-line system, together with an IP-based network. Due to the master-slave behaviour of the PLC medium access, together with the inherent dynamic topology of power-line networks, a mechanism to provide end-to-end communication through the two levels of the power-line system must be provided. In this paper we introduce the architecture of the PLC protocol layer that is being implemented for this end.

Qualitative and Hierarchical Analysis of Protective Factors against Illicit Use of Doping Substances in Athletes Calling a National Anti-Doping Phone-Help Service

Evidence of a sport-specific hierarchy of protective factors against doping would thus be a powerful aid in adapting information and prevention campaigns to target the characteristics of specific athlete groups, and especially those athletes most vulnerable for doping control. The contents of phone calls to a free and anonymous national anti-doping service called 'ecoute dopage' were analysed (192 bodybuilders, 124 cyclists and 44 footballers). The results showed that the protective factors that emerged from analysis could be categorised into two groups. The first comprised 'Health concerns', 'Respect for the law' and 'Doping controls from the environment' and the second comprised 'Doubts about the effectiveness of illicit products, 'Thinking skills' and 'Doubts about doctors'. The ranking of the factors for the cyclists differed from that of the other athletes. The ordering of factors was 1) respect for the law, 2) doping controls from the environment, 3) health concerns 4) doubts about doctors, and 5) doubts about the effectiveness illicit products. The results are analysed in terms of the ranking in each athlete group and the consequences on the athletes' experience and relationship to doping. Specific prevention campaigns are proposed to limit doping behaviour in general and for each sport.

Cognitive Behavior Therapy for Obsessive-Compulsive Behavior in Children with Autism Spectrum Disorder: A Comparison of Hierarchical Linear Modelling and Visual Analysis

Behavioral researchers commonly use single subject designs to evaluate the effects of a given treatment. Several different methods of data analysis are used, each with their own set of methodological strengths and limitations. Visual inspection is commonly used as a method of analyzing data which assesses the variability, level, and trend both within and between conditions (Cooper, Heron, & Heward, 2007). In an attempt to quantify treatment outcomes, researchers developed two methods for analysing data called Percentage of Non-overlapping Data Points (PND) and Percentage of Data Points Exceeding the Median (PEM). The purpose of the present study is to compare and contrast the use of Hierarchical Linear Modelling (HLM), PND and PEM in single subject research. The present study used 39 behaviours, across 17 participants to compare treatment outcomes of a group cognitive behavioural therapy program, using PND, PEM, and HLM on three response classes of Obsessive Compulsive Behaviour in children with Autism Spectrum Disorder. Findings suggest that PEM and HLM complement each other and both add invaluable information to the overall treatment results. Future research should consider using both PEM and HLM when analysing single subject designs, specifically grouped data with variability.

Psychopathy and Antisocial Behaviour: The Moderating Effects of Maternal Neglect and Warmth

Psychopathy researchers have long debated the role of antisocial behaviour and criminality as part of the construct of psychopathy. The current study examined the relationship between the interpersonal and affective traits (Factor 1) of psychopathy and antisocial behaviour (a facet of Factor 2), examining possible predictors of antisocial behaviour. It was hypothesized that early environment would moderate the relationship between Factor 1 traits and antisocial behaviour. Hierarchical multiple regression analyses were used in order to test for possible moderators. Sex differences were found, where men scored higher in Antisocial Behaviour. Childhood Abuse did not moderate the relationship between Factor 1 traits and Antisocial Behaviour, but predicted higher Antisocial Behaviour scores independently. Maternal Neglect was especially influential as a risk factor, significantly interacting with Factor 1 traits to predict higher Antisocial Behaviour scores. Maternal Warmth was also important, interacting with Factor 1 in a protective fashion, predicting lower Antisocial Behaviour Scores.

Explaining voice behaviour: The roles of personal resources, social interdependence, and supervisor leadership style

This study contributes to current research on voice behaviour by investigating several under-explored drivers that motivate employees’ expression of constructive ideas about work-related issues. It draws from the concept of psychological climate to examine how voice behaviour is influenced by employees’ (1) personal resources (tenacity and passion for work), (2) perceptions of social interdependence (task and outcome interdependence), and (3) supervisor leadership style (transformational and transactional). Using a multi-source research design, surveys were administered to 226 employees and to 24 supervisors at a Canadian-based not-for-profit organization. The hypotheses are tested with hierarchical regression analysis. The results indicate that employees are more likely to engage in voice behaviour to the extent that they exhibit higher levels of passion for work. Further, their voice behaviour is lower to the extent that their supervisor adopts a transformational leadership style characterized by high performance expectations or a transactional leadership style based on contingent rewards and contingent punishment behaviours. The study reveals that there are no significant effects of tenacity, social interdependence, and behaviour-focused transformational leadership on voice. The findings have significant implications for organizations that seek to encourage employee behaviours that help improve current work practices or undo harmful situations.

Testing the Revised Hierarchical Model: evidence from word associations

The main purpose of the work described in this paper is to examine the extent to which the L2 developmental changes predicted by Kroll and Stewart's (1994) Revised Hierarchical Model (RHM) can be understood by word association response behaviour. The RHM attempts to account for the relative “strength of the links between words and concepts in each of the bilingual's languages” (Kroll, Van Hell, Tokowicz & Green, 2010, p. 373). It proposes that bilinguals with higher L2 proficiency tend to rely less on mediation, while less proficient L2 learners tend to rely on mediation and access L2 words by translating from L1 equivalents. In this paper, I present findings from a simple word association task. More proficient learners provided a greater proportion of collocational links, suggesting that they mediate less when compared to less proficient learners. The results provide tentative support for Kroll and Stewart's model

Inter-plant communication through mycorrhizal networks mediates complex adaptive behaviour in plant communities

Adaptive behaviour of plants, including rapid changes in physiology, gene regulation and defence response, can be altered when linked to neighbouring plants by a mycorrhizal network (MN). Mechanisms underlying the behavioural changes include mycorrhizal fungal colonization by the MN or interplant communication via transfer of nutrients, defence signals or allelochemicals. We focus this review on our new findings in ectomycorrhizal ecosystems, and also review recent advances in arbuscular mycorrhizal systems. We have found that the behavioural changes in ectomycorrhizal plants depend on environmental cues, the identity of the plant neighbour and the characteristics of the MN. The hierarchical integration of this phenomenon with other biological networks at broader scales in forest ecosystems, and the consequences we have observed when it is interrupted, indicate that underground ‘tree talk’ is a foundational process in the complex adaptive nature of forest ecosystems.

Oviposition of a leaf-miner on Erythroxylum tortuosum (Erythroxylaceae) leaves: hierarchical variation of physical leaf traits

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES)

Parameter recovery for a skew-normal IRT model under a Bayesian approach: hierarchical framework, prior and kernel sensitivity and sample size

Item response theory (IRT) comprises a set of statistical models which are useful in many fields, especially when there is an interest in studying latent variables (or latent traits). Usually such latent traits are assumed to be random variables and a convenient distribution is assigned to them. A very common choice for such a distribution has been the standard normal. Recently, Azevedo et al. [Bayesian inference for a skew-normal IRT model under the centred parameterization, Comput. Stat. Data Anal. 55 (2011), pp. 353-365] proposed a skew-normal distribution under the centred parameterization (SNCP) as had been studied in [R. B. Arellano-Valle and A. Azzalini, The centred parametrization for the multivariate skew-normal distribution, J. Multivariate Anal. 99(7) (2008), pp. 1362-1382], to model the latent trait distribution. This approach allows one to represent any asymmetric behaviour concerning the latent trait distribution. Also, they developed a Metropolis-Hastings within the Gibbs sampling (MHWGS) algorithm based on the density of the SNCP. They showed that the algorithm recovers all parameters properly. Their results indicated that, in the presence of asymmetry, the proposed model and the estimation algorithm perform better than the usual model and estimation methods. Our main goal in this paper is to propose another type of MHWGS algorithm based on a stochastic representation (hierarchical structure) of the SNCP studied in [N. Henze, A probabilistic representation of the skew-normal distribution, Scand. J. Statist. 13 (1986), pp. 271-275]. Our algorithm has only one Metropolis-Hastings step, in opposition to the algorithm developed by Azevedo et al., which has two such steps. This not only makes the implementation easier but also reduces the number of proposal densities to be used, which can be a problem in the implementation of MHWGS algorithms, as can be seen in [R.J. Patz and B.W. Junker, A straightforward approach to Markov Chain Monte Carlo methods for item response models, J. Educ. Behav. Stat. 24(2) (1999), pp. 146-178; R. J. Patz and B. W. Junker, The applications and extensions of MCMC in IRT: Multiple item types, missing data, and rated responses, J. Educ. Behav. Stat. 24(4) (1999), pp. 342-366; A. Gelman, G.O. Roberts, and W.R. Gilks, Efficient Metropolis jumping rules, Bayesian Stat. 5 (1996), pp. 599-607]. Moreover, we consider a modified beta prior (which generalizes the one considered in [3]) and a Jeffreys prior for the asymmetry parameter. Furthermore, we study the sensitivity of such priors as well as the use of different kernel densities for this parameter. Finally, we assess the impact of the number of examinees, number of items and the asymmetry level on the parameter recovery. Results of the simulation study indicated that our approach performed equally as well as that in [3], in terms of parameter recovery, mainly using the Jeffreys prior. Also, they indicated that the asymmetry level has the highest impact on parameter recovery, even though it is relatively small. A real data analysis is considered jointly with the development of model fitting assessment tools. The results are compared with the ones obtained by Azevedo et al. The results indicate that using the hierarchical approach allows us to implement MCMC algorithms more easily, it facilitates diagnosis of the convergence and also it can be very useful to fit more complex skew IRT models.

Stellar Relics of the hierarchical assembly of the Galaxy

This PhD Thesis is part of a long-term wide research project, carried out by the "Osservatorio Astronomico di Bologna (INAF-OABO)", that has as primary goal the comprehension and reconstruction of formation mechanism of galaxies and their evolution history. There is now substantial evidence, both from theoretical and observational point of view, in favor of the hypothesis that the halo of our Galaxy has been at least partially, built up by the progressive accretion of small fragments, similar in nature to the present day dwarf galaxies of the Local Group. In this context, the photometric and spectroscopic study of systems which populate the halo of our Galaxy (i.e. dwarf spheroidal galaxy, tidal streams, massive globular cluster, etc) permits to discover, not only the origin and behaviour of these systems, but also the structure of our Galactic halo, combined with its formation history. In fact, the study of the population of these objects and also of their chemical compositions, age, metallicities and velocity dispersion, permit us not only an improvement in the understanding of the mechanisms that govern the Galactic formation, but also a valid indirect test for cosmological model itself. Specifically, in this Thesis we provided a complete characterization of the tidal Stream of the Sagittarius dwarf spheroidal galaxy, that is the most striking example of the process of tidal disruption and accretion of a dwarf satellite in to our Galaxy. Using Red Clump stars, extracted from the catalogue of the Sloan Digital Sky Survey (SDSS) we obtained an estimate of the distance, the depth along the line of sight and of the number density for each detected portion of the Stream (and more in general for each detected structure along our line of sight). Moreover comparing the relative number (i.e. the ratio) of Blue Horizontal Branch stars and Red Clump stars (the two features are tracers of different age/different metallicity populations) in the main body of the galaxy and in the Stream, in order to verify the presence of an age-metallicity gradient along the Stream. We also report the detection of a population of Red Clump stars probably associated with the recently discovered Bootes III stellar system. Finally, we also present the results of a survey of radial velocities over a wide region, extending from r ~ 10' out to r ~ 80' within the massive star cluster Omega Centauri. The survey was performed with FLAMES@VLT, to study the velocity dispersion profile in the outer regions of this stellar system. All the results presented in this Thesis, have already been published in refeered journals.