An Online Actor-Critic Algorithm with Function Approximation for Constrained Markov Decision Processes

We develop an online actor-critic reinforcement learning algorithm with function approximation for a problem of control under inequality constraints. We consider the long-run average cost Markov decision process (MDP) framework in which both the objective and the constraint functions are suitable policy-dependent long-run averages of certain sample path functions. The Lagrange multiplier method is used to handle the inequality constraints. We prove the asymptotic almost sure convergence of our algorithm to a locally optimal solution. We also provide the results of numerical experiments on a problem of routing in a multi-stage queueing network with constraints on long-run average queue lengths. We observe that our algorithm exhibits good performance on this setting and converges to a feasible point.

Cultivation methods for Lemnaceae [Translation from: Wiadomosci Botaniczne 7(2) 153-164, 1963]

Representatives from the family of Lemnaceae (duckweed) make ideal experimental material for research into a succession of phytophysiological processes with regard to growth rate and vegetative reproduction. They are also easy to maintain in sterile cultures. Lemnaceae belong to the higher flowering plants (flowers are rarely produced), however they are distinguished by a much simplified morphological and anatomical structure. As water plants they possess the advantage, that they can be cultivated in synthetic media under laboratory conditions controlled by the application of both a known light intensity and temperature. This paper describes experimental research of growth of Lemnaceae in different conditions. Some of the variables were mineral media, illumination and aeration.

O engenho por dentro: cartografia das práticas cotidianas de cuidado em saúde mental dos auxiliares e técnicos de enfermagem na perspectiva da integralidade

Discutem-se os processos de trabalho e de produção do cuidado dos auxiliares e técnicos de enfermagem do Núcleo de Atenção à Crise do Instituto Municipal de Assistência à Saúde Nise da Silveira, no contexto da desinstitucionalização, e seus efeitos na integralidade do cuidado. O foco de análise repousa sobre o cuidado cotidiano prestado por esses profissionais nas unidades de internação psiquiátrica. Buscou-se realizar uma reflexão sobre os aspectos contemporâneos do cotidiano da assistência psiquiátrica e o papel dos hospitais psiquiátricos no atendimento aos pacientes agudos. Para tal objetivou-se, especificamente: contextualizar o IMNS no cenário de saúde mental no município do Rio de Janeiro; descrever as práticas assistenciais de auxiliares e técnicos de enfermagem no cuidado aos pacientes agudos internados; e discutir as práticas de cuidado no contexto da reorientação do modelo hospitalar e suas repercussões na integralidade da assistência em saúde, buscando contradições e aproximações com o discurso da política de saúde mental vigente. Foram abordadas na fundamentação teórica: as questões da prática da equipe de saúde sob o eixo da integralidade, em especial da equipe de enfermagem, e a relação trabalho/saúde/cotidiano na construção dos processos de trabalho e de produção do cuidado da assistência prestada. Como abordagem teórico-metodológica que possibilitou alcançar o objetivo proposto, realizou-se estudo exploratório, de natureza qualitativa, na perspectiva da cartografia, tendo na observação participante seu principal elemento de coleta de dados. O mapeamento das práticas rotineiras de cuidado evidenciou que estas são predominantemente pautadas no modelo asilar, no qual auxiliares e técnicos desempenham suas atividades diárias de modo distanciado dos pacientes e de suas necessidades. Esses profissionais não são vistos como elementos da equipe de saúde mental, e assim, desprestigiados em relação à política de formação e capacitação em saúde mental, assumem o cuidado aos pacientes institucionalizados de modo pouco estruturado, sem levar em consideração os anseios e sofrimentos dos usuários. Acolhimento, vínculo e integralidade fizeram parte do discurso oficial de reorganização da assistência, num contexto macroestrutural, mas ainda não se materializam nas práticas assistenciais da unidade de saúde pesquisada. A importância do estudo está pautada no fornecimento de subsídios para a formulação e implantação de ações que auxiliem na melhoria da gestão do trabalho em saúde mental. Sua justificativa se faz na medida em que essa área é permeada por conflitos e questões que interferem decisivamente na qualidade dos serviços prestados.

When Working Memory and Attention Compete: Characterizing the Dynamic Interdependence between our Mental Workspace and External Environment

<p>All of us are taxed with juggling our inner mental lives with immediate external task demands. For many years, the temporary maintenance of internal information was considered to be handled by a dedicated working memory (WM) system. It has recently become increasingly clear, however, that such short-term internal activation interacts with attention focused on external stimuli. It is unclear, however, exactly why these two interact, at what level of processing, and to what degree. Because our internal maintenance and external attention processes co-occur with one another, the manner of their interaction has vast implications for functioning in daily life. The work described here has employed original experimental paradigms combining WM and attention task elements, functional magnetic resonance imaging (fMRI) to illuminate the associated neural processes, and transcranial magnetic stimulation (TMS) to clarify the causal substrates of attentional brain function. These studies have examined a mechanism that might explain why (and when) the content of WM can involuntarily capture visual attention. They have, furthermore, tested whether fundamental attentional selection processes operate within WM, and whether they are reciprocal with attention. Finally, they have illuminated the neural consequences of competing attentional demands. The findings indicate that WM shares representations, operating principles, and cognitive resources with externally-oriented attention.</p>

‘Hearing Voices': Punishing women’s mental ill-health in Northern Ireland’s jails

Informed by primary interviews and observational research conducted by the authors with women prisoners in Northern Ireland, this article focuses on prison as an institutional manifestation of women’s powerlessness and vulnerability, particularly those enduring mental ill-health. It contextualises their experiences within continua of violence and ‘unsafety’. It also considers official responses to critical inspection reports and those of the Northern Ireland Human Rights Commission based on the authors’ research findings. Finally, the primary research demonstrates that three decades on from publication the first critical analyses of women’s imprisonment, the conditions of gendered marginalisation, medicalisation and punishment remain. This is brought into stark relief in the punitive regimes imposed on those most vulnerable through mental ill-health.

Causal conjunction fallacies: The roles of causal strength and mental resources

In two experiments we tested the prediction derived from Tversky and Kahneman's (1983) work on the causal conjunction fallacy that the strength of the causal connection between constituent events directly affects the magnitude of the causal conjunction fallacy. We also explored whether any effects of perceived causal strength were due to graded output from heuristic Type 1 reasoning processes or the result of analytic Type 2 reasoning processes. As predicted, Experiment 1 demonstrated that fallacy rates were higher for strongly than for weakly related conjunctions. Weakly related conjunctions in turn attracted higher rates of fallacious responding than did unrelated conjunctions. Experiment 2 showed that a concurrent memory load increased rates of fallacious responding for strongly related but not for weakly related conjunctions. We interpret these results as showing that manipulations of the strength of the perceived causal relationship between the conjuncts result in graded output from heuristic reasoning process and that additional mental resources are required to suppress strong heuristic output.

Frontal EEG/ERP correlates of attentional processes, cortisol and motivational states in adolescents from lower and higher socioeconomic status

Event-related potentials (ERPs) and other electroencephalographic (EEG) evidence show that frontal brain areas of higher and lower socioeconomic status (SES) children are recruited differently during selective attention tasks. We assessed whether multiple variables related to self-regulation (perceived mental effort) emotional states (e.g., anxiety, stress, etc.) and motivational states (e.g., boredom, engagement, etc.) may co-occur or interact with frontal attentional processing probed in two matched-samples of fourteen lower-SES and higher-SES adolescents. ERP and EEG activation were measured during a task probing selective attention to sequences of tones. Pre- and post-task salivary cortisol and self-reported emotional states were also measured. At similar behavioural performance level, the higher-SES group showed a greater ERP differentiation between attended (relevant) and unattended (irrelevant) tones than the lower-SES group. EEG power analysis revealed a cross-over interaction, specifically, lower-SES adolescents showed significantly higher theta power when ignoring rather than attending to tones, whereas, higher-SES adolescents showed the opposite pattern. Significant theta asymmetry differences were also found at midfrontal electrodes indicating left hypo-activity in lower-SES adolescents. The attended vs. unattended difference in right midfrontal theta increased with individual SES rank, and (independently from SES) with lower cortisol task reactivity and higher boredom. Results suggest lower-SES children used additional compensatory resources to monitor/control response inhibition to distracters, perceiving also more mental effort, as compared to higher-SES counterparts. Nevertheless, stress, boredom and other task-related perceived states were unrelated to SES. Ruling out presumed confounds, this study confirms the midfrontal mechanisms responsible for the SES effects on selective attention reported previously and here reflect genuine cognitive differences.

Stress processes in caring for an end-of-life family member:Application of a theoretical model

Objectives: Family caregivers play a vital role in maintaining the lives of individuals with advanced illness living in the community. However, the responsibility of caregiving for an end-of-life family member can have profound consequences on the psychological, physical and financial well-being of the caregiver. While the literature has identified caregiver stress or strain as a complex process with multiple contributing factors, few comprehensive studies exist. This study examined a wide range of theory-driven variables contributing to family caregiver stress. Method: Data variables from interviews with primary family caregivers were mapped onto the factors within the Stress Process Model theoretical framework. A hierarchical multiple linear regression analysis was used to determine the strongest predictors of caregiver strain as measured by a validated composite index, the Caregiver Strain Index. Results: The study included 132 family caregivers across south-central/western Ontario, Canada. About half of these caregivers experienced high strain, the extent of which was predicted by lower perceived program accessibility, lower functional social support, greater weekly amount of time caregivers committed to the care recipient, younger caregiver age and poorer caregiver self-perceived health. Conclusion: This study examined the influence of a multitude of factors in the Stress Process Model on family caregiver strain, finding stress to be a multidimensional construct. Perceived program accessibility was the strongest predictor of caregiver strain, more so than intensity of care, highlighting the importance of the availability of community resources to support the family caregiving role.

Combating the mental health stigma with nostalgia

We report research implicating nostalgia as an intrapersonal means of warding off the stigmatization of persons with mental illness. We hypothesized and found that nostalgia about an encounter with a person with mental illness improves attitudes toward the mentally ill. In Experiment 1, undergraduates who recalled an encounter with a mentally ill person while focusing on central (vs. peripheral) features of the nostalgia prototype reported a more positive outgroup attitude. This beneficial effect of nostalgia was mediated by greater inclusion of the outgroup in the self (IOGS). In Experiment 2, undergraduates who recalled a nostalgic (vs. ordinary) interaction with a mentally ill person subsequently showed a more positive outgroup attitude. Results supported a serial mediation model whereby nostalgia increased social connectedness, which predicted greater IOGS and outgroup trust. IOGS and outgroup trust, in turn, predicted more positive outgroup attitudes. We ruled out alternative explanations for the results (i.e., mood, perceived positivity, and typicality of the recalled outgroup member). The findings speak to the intricate psychological processes underlying the prejudice-reduction function of nostalgia and their interventional potential. Copyright © 2013 John Wiley &amp; Sons, Ltd.

The effects of global climate change on the cycling and processes of persistent organic pollutants (POPs) in the North Sea

The fate and cycling of two selected legacy persistent organic pollutants (POPs), PCB 153 and gamma-HCH, in the North Sea in the 21st century have been modelled with combined hydrodynamic and fate and transport ocean models<br/>(HAMSOM and FANTOM, respectively). To investigate the impact of climate variability on POPs in the North Sea in the 21st century, future scenario model runs for three 10-year periods to the year 2100 using plausible levels of both in<br/>situ concentrations and atmospheric, river and open boundary inputs are performed. This slice mode under a moderate scenario (A1B) is sufficient to provide a basis for further analysis. For the HAMSOM and atmospheric forcing, results of the IPCC A1B (SRES) 21st century scenario are utilized, where surface forcing is provided by the REMO downscaling of the ECHAM5 global atmospheric model, and open boundary conditions are provided by the MPIOM global ocean model.<br/>Dry gas deposition and volatilization of gamma-HCH increase in the future relative to the present by up to 20% (in the spring and summer months for deposition and in summer for volatilization). In the water column, total mass of <br/>gamma-HCH and PCB 153 remain fairly steady in all three runs. In sediment,<br/>gamma-HCH increases in the future runs, relative to the present, while PCB 153 in sediment decreases exponentially in all three runs, but even faster in the future, due to the increased number of storms, increased duration of gale wind conditions and increased water and air temperatures, all of which are the result of climate change. Annual net sinks exceed sources at the ends of all periods.<br/>Overall, the model results indicate that the climate change scenarios considered here generally have a negligible influence on the simulated fate and transport of the two POPs in the North Sea, although the increased number and magnitude of storms in the 21st century will result in POP resuspension and ensuing revolatilization events. Trends in emissions from primary and secondary sources will remain the key driver of levels of these contaminants over time.

Applying the recovery approach to the interface between mental health and child protection services

There are a range of theoretical approaches which may inform the interface between child protection and adult mental health services. These theoretical perspectives tend to be focused on either child protection or mental health with no agreed integrating framework. The interface continues to be identified, in research, case management reviews and inquiry reports, as complex and problematic. This paper proposes that more positive, integrated approaches to service user engagement, risk assessment and management may lead to better outcomes in working with families experiencing parental mental health problems and child protection concerns. It is proposed that the recovery approach, increasingly used in mental health services, can inform the processes of engagement, assessment and intervention at the mental health and child protection interface. The article provides a critical overview of the recovery approach and compares it with approaches typifying interventions in child protection work to date. Relevant research and inquiries are also examined as a context for how to more effectively respond to cases where there are issues around parental mental health problems and child protection. The article concludes with case material to illustrate the potential application of the recovery approach to the interface between mental health and child protection services.<br/><br/>

Mental Health Nurses' Family Focused Practice with Parents who have Mental Illness, their Children and Families in Adult Mental Health Services in Ireland

Introduction: While it is recommended that mental health professionals engage in family focused practice (FFP), there is limited understanding regarding psychiatric nurses’ practice with parents who have mental illness, their children and families in adult mental health services. <br/><br/>Methods: This study utilized a mixed methods approach to measure the extent of psychiatric nurses’ family focused practice and factors that predicted it. It also sought to explore the nature and scope of high scoring psychiatric nurses’ FFP and factors that affected their capacity to engage in FFP. Three hundred and forty three psychiatric nurses in 12 mental health services throughout Ireland completed the Family Focused Mental Health Practice Questionnaire (FFMHPQ). Fourteen nurses who achieved high scores on the FFMHPQ also participated in semi-structured interviews.<br/><br/>Results: Whilst the majority of nurses were not family focused a substantial minority were. High scoring nurses’ practice was complex and multifaceted, comprising various family focused activities, principles and processes. Nurses’ capacity to engage in FFP was determined by their knowledge and skills, working in community settings and own parenting experience. <br/><br/>Conclusions: Generally, low levels of family focused practice suggest the need for organizations to develop and implement guidelines, policies and training to support mental health professionals to adopt a whole family approach.