Natural actor-critic algorithms

We present four new reinforcement learning algorithms based on actor-critic, natural-gradient and functi approximation ideas,and we provide their convergence proofs. Actor-critic reinforcement learning methods are online approximations to policy iteration in which the value-function parameters are estimated using temporal difference learning and the policy parameters are updated by stochastic gradient descent. Methods based on policy gradients in this way are of special interest because of their compatibility with function-approximation methods, which are needed to handle large or infinite state spaces. The use of temporal difference learning in this way is of special interest because in many applications it dramatically reduces the variance of the gradient estimates. The use of the natural gradient is of interest because it can produce better conditioned parameterizations and has been shown to further reduce variance in some cases. Our results extend prior two-timescale convergence results for actor-critic methods by Konda and Tsitsiklis by using temporal difference learning in the actor and by incorporating natural gradients. Our results extend prior empirical studies of natural actor-critic methods by Peters, Vijayakumar and Schaal by providing the first convergence proofs and the first fully incremental algorithms.

Management of semi-natural grasslands for butterfly and moth communities

This work focuses on the factors affecting species richness, abundance and species composition of butterflies and moths in Finnish semi-natural grasslands, with a special interest in the effects of grazing management. In addition, an aim was set at evaluating the effectiveness of the support for livestock grazing in semi-natural grasslands, which is included in the Finnish agri-environment scheme. In the first field study, butterfly and moth communities in resumed semi-natural pastures were com-pared to old, annually grazed and abandoned previous pastures. Butterfly and moth species compo-sition in restored pastures resembled the compositions observed in old pastures after circa five years of resumed cattle grazing, but diversity of butterflies and moths in resumed pastures remained at a lower level compared with old pastures. None of the butterfly and moth species typical of old pas-tures had become more abundant in restored pastures compared with abandoned pastures. There-fore, it appears that restoration of butterfly and moth communities inhabiting semi-natural grass-lands requires a longer time that was available for monitoring in this study. In the second study, it was shown that local habitat quality has the largest impact on the occurrence and abundance of butterflies and moths compared to the effects of grassland patch area and connec-tivity of the regional grassland network. This emphasizes the importance of current and historical management of semi-natural grasslands on butterfly and moth communities. A positive effect of habitat connectivity was observed on total abundance of the declining butterflies and moths, sug-gesting that these species have strongest populations in well-connected habitat networks. Highest species richness and peak abundance of most individual species of butterflies and moths were generally observed in taller grassland vegetation compared with vascular plants, suggesting a preference towards less intensive management in insects. These differences between plants and their insect herbivores may be understood in the light of both (1) the higher structural diversity of tall vegetation and (2) weaker tolerance of disturbances by herbivorous insects due to their higher trophic level compared to plants. The ecological requirements of all species and species groups inhabiting semi-natural grasslands are probably never met at single restricted sites. Therefore, regional implementation of management to create differently managed areas is imperative for the conservation of different species and species groups dependent on semi-natural grasslands. With limited resources it might be reasonable to focus much of the management efforts in the densest networks of suitable habitat to minimise the risk of extinction of the declining species.

Unsteady Natural Convection Flow over a Heated Cylinder Buried in a Fluid Saturated Porous Medium

Transient natural convection flow on a heated cylinder buried in a semi-infinite liquid-saturated porous medium has been studied. The unsteadiness in the problem arises due to the cylinder which is heated (cooled) suddenly and then maintained at that temperature. The coupled partial differential equations governing the flow and heat transfer are cast into stream function-temperature formulation, and the solutions are obtained from the initial time to the time when steady state is reached. The heat transfer is found to change significantly with increasing time in a small time interval immediately after the start of the impulsive change, and steady state is reached after some time. The average Nusselt number is found to increase with Rayleigh number When the surface of the cylinder is suddenly cooled, there is a change in the direction of the heat transfer in a small time interval immediately after the start of the impulsive change in the surface temperature;however when the surface temperature is suddenly increased, no such phenomenon is observed.

Identification of alpha- and beta-Hydroxy Acid Containing Cyclodepsipeptides in Natural Peptide Mixtures Using Negative Ion Mass Spectrometry

Natural peptide libraries often contain cyclodepsipeptides containing alpha or beta hydroxy residues. Extracts of fungal hyphae of Isaria yield a microheterogenous cyclodepsipeptide mixture in which two classes of molecules can be identified by mass spectral fragmentation of negative ions. In the case of isaridins, which contain an alpha-hydroxy residue and a beta-amino acid residue, a characteristic product ion corresponding to a neutral loss of 72 Da is obtained. hi addition, neutral loss of water followed by a 72 Da loss is also observed. Two distinct modes of fragmentation rationalize the observed product ion distribution. The neutral loss of 72 Da has also been obtained for a roseotoxin component, which is also an alpha-hydroxy residue containing cyclodepsipeptide. In the case of isariins, which contain a beta-hydroxy acid residue, ring opening and subsequent loss of the terminal residue as an unsaturated ketene fragment, rationalizes the observed product ion formation. Fragmentation of negative ions provide characteristic neutral losses, which are diagnostic of the presence of alpha-hydroxy or beta-hydroxy residues.

Natural recovery from cannabis use in people with psychosis: A qualitative study

- Objective There is rapidly growing evidence of natural recovery from cannabis use in people with psychosis, but little is known about how it occurs. This qualitative study explores what factors influence the decision to cease cannabis use, maintain cessation, and prevent relapse. - Methods Ten people with early psychosis and lifetime cannabis misuse, who had been abstinent for at least a month, were recruited from public adult mental health services. These six men and four women participated in a semi-structured qualitative interview assessing reasons for addressing cannabis use, effective change strategies, lapse contexts, and methods used to regain control. Interpretative phenomenological analysis was used to identify themes in their responses. - Results Participants had a mean age of 23 years (SD = 3.7), started using cannabis at age 13.7 (SD = 1.6), began daily use at 17 (SD = 3.1), and had abstained from cannabis for 7.9 months (SD = 5.4). Awareness of the negative impact of substance use across multiple domains and the presence of social support for cannabis cessation were seen as vital to sustained success, as was utilization of a combination of coping strategies. The ability to address pressure from substance-using peers was commonly mentioned. - Conclusions Maximally effective treatment may need to focus on eliciting a range of benefits of cessation and control strategies and on maximizing both support for change and resistance to peer pressure. Further research might focus on comparing perceived effective strategies between individuals who obtain sustained cessation versus those who relapse.

The RecA intein of Mycobacterium tuberculosis promotes cleavage of ectopic DNA sites. Implications for the dispersal of inteins in natural populations

The RecA intein of Mycobacterium tuberculosis, a novel double-stranded DNA endonuclease, requires both Mn(2+) and ATP for efficient cleavage of the inteinless recA allele. In this study, we show that Mg(2+) alone was sufficient to stimulate PI-MtuI to cleave double-stranded DNA at ectopic sites. In the absence of Mg(2+), PI-MtuI formed complexes with topologically different forms of DNA containing ectopic recognition sequences with equal affinity but failed to cleave DNA. We observed that PI-MtuI was able to inflict double-strand breaks robustly within the ectopic recognition sequence to generate either a blunt end or 1-2-nucleotide 3'-hydroxyl overhangs. Mutational analyses of the presumptive metal ion-binding ligands (Asp(122), Asp(222), and Glu(220)) together with immunoprecipitation assays provided compelling evidence to link both the Mg(2+)- and Mn(2+) and ATP-dependent endonuclease activities to PI-MtuI. The kinetic mechanism of PI-MtuI promoted cleavage of ectopic DNA sites proceeded through a sequential mechanism with transient accumulation of nicked circular duplex DNA as an intermediate. Together, these data suggest that PI-MtuI, like group II introns, might mediate ectopic DNA transposition and hence its lateral transfer in natural populations.

Wear Next: An Exploration into the Future of Wearable Technology

Technology is increasingly infiltrating all aspects of our lives and the rapid uptake of devices that live near, on or in our bodies are facilitating radical new ways of working, relating and socialising. This distribution of technology into the very fabric of our everyday life creates new possibilities, but also raises questions regarding our future relationship with data and the quantified self. By embedding technology into the fabric of our clothes and accessories, it becomes ‘wearable’. Such ‘wearables’ enable the acquisition of and the connection to vast amounts of data about people and environments in order to provide life-augmenting levels of interactivity. Wearable sensors for example, offer the potential for significant benefits in the future management of our wellbeing. Fitness trackers such as ‘Fitbit’ and ‘Garmen’ provide wearers with the ability to monitor their personal fitness indicators while other wearables provide healthcare professionals with information that improves diagnosis. While the rapid uptake of wearables may offer unique and innovative opportunities, there are also concerns surrounding the high levels of data sharing that come as a consequence of these technologies. As more ‘smart’ devices connect to the Internet, and as technology becomes increasingly available (e.g. via Wi-Fi, Bluetooth), more products, artefacts and things are becoming interconnected. This digital connection of devices is called The ‘Internet of Things’ (IoT). IoT is spreading rapidly, with many traditionally non-online devices becoming increasingly connected; products such as mobile phones, fridges, pedometers, coffee machines, video cameras, cars and clothing. The IoT is growing at a rapid rate with estimates indicating that by 2020 there will be over 25 billion connected things globally. As the number of devices connected to the Internet increases, so too does the amount of data collected and type of information that is stored and potentially shared. The ability to collect massive amounts of data - known as ‘big data’ - can be used to better understand and predict behaviours across all areas of research from societal and economic to environmental and biological. With this kind of information at our disposal, we have a more powerful lens with which to perceive the world, and the resulting insights can be used to design more appropriate products, services and systems. It can however, also be used as a method of surveillance, suppression and coercion by governments or large organisations. This is becoming particularly apparent in advertising that targets audiences based on the individual preferences revealed by the data collected from social media and online devices such as GPS systems or pedometers. This type of technology also provides fertile ground for public debates around future fashion, identity and broader social issues such as culture, politics and the environment. The potential implications of these type of technological interactions via wearables, through and with the IoT, have never been more real or more accessible. But, as highlighted, this interconnectedness also brings with it complex technical, ethical and moral challenges. Data security and the protection of privacy and personal information will become ever more present in current and future ethical and moral debates of the 21st century. This type of technology is also a stepping-stone to a future that includes implantable technology, biotechnologies, interspecies communication and augmented humans (cyborgs). Technologies that live symbiotically and perpetually in our bodies, the built environment and the natural environment are no longer the stuff of science fiction; it is in fact a reality. So, where next?... The works exhibited in Wear Next_ provide a snapshot into the broad spectrum of wearables in design and in development internationally. This exhibition has been curated to serve as a platform for enhanced broader debate around future technology, our mediated future-selves and the evolution of human interactions. As you explore the exhibition, may we ask that you pause and think to yourself, what might we... Wear Next_? WEARNEXT ONLINE LISTINGS AND MEDIA COVERAGE: http://indulgemagazine.net/wear-next/ http://www.weekendnotes.com/wear-next-exhibition-gallery-artisan/ http://concreteplayground.com/brisbane/event/wear-next_/ http://www.nationalcraftinitiative.com.au/news_and_events/event/48/wear-next http://bneart.com/whats-on/wear-next_/ http://creativelysould.tumblr.com/post/124899079611/creative-weekend-art-edition http://www.abc.net.au/radionational/programs/breakfast/smartly-dressed-the-future-of-wearable-technology/6744374 http://couriermail.newspaperdirect.com/epaper/viewer.aspx RADIO COVERAGE http://www.abc.net.au/radionational/programs/breakfast/wear-next-exhibition-whats-next-for-wearable-technology/6745986 TELEVISION COVERAGE http://www.abc.net.au/radionational/programs/breakfast/wear-next-exhibition-whats-next-for-wearable-technology/6745986 https://au.news.yahoo.com/video/watch/29439742/how-you-could-soon-be-wearing-smart-clothes/#page1

Chemical modifications of natural triterpenes—glycyrrhetinic and boswellic acids: evaluation of their biological activity

Synthetic analogues of naturally occurring triterpenoids; glycyrrhetinic acid, arjunolic acid, and boswellic acids, by modification of A-ring with a cyano- and enone-functionality, have been reported. A novel method of synthesis of α-cyanoenones from isoxazoles is reported. Bioassays using primary mouse macrophages and tumor cell lines indicate potent anti-inflammatory and cytotoxic activities associated with cyano-enones of boswellic acid and glycyrrhetinic acid.

Decision fusion for reliable flood mapping using remote sensing images

Flood extent mapping is a basic tool for flood damage assessment, which can be done by digital classification techniques using satellite imageries, including the data recorded by radar and optical sensors. However, converting the data into the information we need is not a straightforward task. One of the great challenges involved in the data interpretation is to separate the permanent water bodies and flooding regions, including both the fully inundated areas and the wet areas where trees and houses are partly covered with water. This paper adopts the decision fusion technique to combine the mapping results from radar data and the NDVI data derived from optical data. An improved capacity in terms of identifying the permanent or semi-permanent water bodies from flood inundated areas has been achieved. Computer software tools Multispec and Matlab were used.

Block-based feature detection and matching for mosaicing of camera-captured document images

In this paper, we present a new feature-based approach for mosaicing of camera-captured document images. A novel block-based scheme is employed to ensure that corners can be reliably detected over a wide range of images. 2-D discrete cosine transform is computed for image blocks defined around each of the detected corners and a small subset of the coefficients is used as a feature vector A 2-pass feature matching is performed to establish point correspondences from which the homography relating the input images could be computed. The algorithm is tested on a number of complex document images casually taken from a hand-held camera yielding convincing results.

Serial Changes in Markers Measuring Coagulation, Fibrinolysis, and Vasoactivity in Patients with Ischemic Stroke

Ischemic stroke (IS) is a heterogeneous disease in which outcome is influenced by many factors. The hemostatic system is activated in association with cerebral ischemia, and thus, markers measuring coagulation, fibrinolysis, and vasoactivity could be useful tools in clinical practice. We investigated whether repeated measurements of these markers reveal patterns that might help in evaluating IS patients, including the early diagnosis of stroke subtypes, in estimating prognosis and risk of recurrence, and in selecting a treatment for secondary prevention of stroke. Vasoconstrictor peptide endothelin-1 (ET-1), homocysteine (Hcy), indicators of thrombin formation and activation (prothrombin fragment 1+2/F1+2, thrombin-antithrombin complex/TAT), indicators of plasmin formation and fibrinolysis (tissue plasminogen activator/t-PA, plasminogen activator inhibitor-1/PAI-1, and D-dimer), and natural anticoagulants (antithrombin/AT, protein C/PC, and protein S/PS) were measured in 102 consecutive mild to moderate IS patients on four occasions: on admission and at 1 week, 1 month, and 3 months after stroke, and once in controls. All patients underwent neurological examination and blood sampling in the same session. Furthermore, 42 IS patients with heterozygous factor V Leiden mutation (FVLm) were selected from 740 IS patients without an obvious etiology, and evaluated in detail for specific clinical, laboratory, and radiological features. Measurements of ET-1 and Hcy levels did not disclose information that could aid in the diagnostic evaluation of IS patients. F1+2 level at 3 months after IS had a positive correlation with recurrence of thromboembolic events, and thus, may be used as a predictive marker of subsequent cerebral events. The D-dimer and AT levels on admission and 1 week after IS were strongly associated with stroke severity, outcome, and disability. The specific analysis of IS patients with FVLm more often revealed a positive family history of thrombosis, a higher prevalence of peripheral vascular disease, and multiple infarctions in brain images, most of which were `silent infarcts´. Results of this study support the view that IS patients with sustained activation of both the fibrinolytic and the coagulation systems and increased thrombin generation may have an unfavorable prognosis. The level of activation may reflect the ongoing thrombotic process and the extent of thrombosis. Changes in these markers could be useful in predicting prognosis of IS patients. A clear need exists for a randomized prospective study to determine whether a subgroup of IS patients with markers indicating activation of fibrinolytic and coagulation systems might benefit from more aggressive secondary prevention of IS.

Edge-based connected component approach for skew correction of complex document images

Skew correction of complex document images is a difficult task. We propose an edge-based connected component approach for robust skew correction of documents with complex layout and content. The algorithm essentially consists of two steps - an 'initialization' step to determine the image orientation from the centroids of the connected components and a 'search' step to find the actual skew of the image. During initialization, we choose two different sets of points regularly spaced across the the image, one from the left to right and the other from top to bottom. The image orientation is determined from the slope between the two succesive nearest neighbors of each of the points in the chosen set. The search step finds succesive nearest neighbors that satisfy the parameters obtained in the initialization step. The final skew is determined from the slopes obtained in the 'search' step. Unlike other connected component based methods, the proposed method does not require any binarization step that generally precedes connected component analysis. The method works well for scanned documents with complex layout of any skew with a precision of 0.5 degrees.