Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Modulation-Mode Assignment in SVD-Aided Downlink Multiuser MIMO-OFDM Systems

Multicarrier transmission such as OFDM (orthogonal frequency division multiplexing) is an established technique for radio transmission systems and it can be considered as a promising approach for next generation wireless systems. However, in order to comply with the demand on increasing available data rates in particular in wireless technologies, systems with multiple transmit and receive antennas, also called MIMO (multiple-input multiple-output) systems, have become indispensable for future generations of wireless systems. Due to the strongly increasing demand in high-data rate transmission systems, frequency non-selective MIMO links have reached a state of maturity and frequency selective MIMO links are in the focus of interest. In this field, the combination of MIMO transmission and OFDM can be considered as an essential part of fulfilling the requirements of future generations of wireless systems. However, single-user scenarios have reached a state of maturity. By contrast multiple users' scenarios require substantial further research, where in comparison to ZF (zero-forcing) multiuser transmission techniques, the individual user's channel characteristics are taken into consideration in this contribution. The performed joint optimization of the number of activated MIMO layers and the number of transmitted bits per subcarrier shows that not necessarily all user-specific MIMO layers per subcarrier have to be activated in order to minimize the overall BER under the constraint of a given fixed data throughput.

Modulation-mode assignment in SVD-assisted multiuser MIMO-OFDM systems

In order to comply with the demand on increasing available data rates in particular in wireless technologies, systems with multiple transmit and receive antennas, also called MIMO (multiple-input multiple-output) systems, have become indispensable for future generations of wireless systems. Due to the strongly increasing demand in high-data rate transmission systems, frequency non-selective MIMO links have reached a state of maturity and frequency selective MIMO links are in the focus of interest. In this field, the combination of MIMO transmission and OFDM (orthogonal frequency division multiplexing) can be considered as an essential part of fulfilling the requirements of future generations of wireless systems. However, single-user scenarios have reached a state of maturity. By contrast multiple users’ scenarios require substantial further research, where in comparison to ZF (zero-forcing) multiuser transmission techniques, the individual user’s channel characteristics are taken into consideration in this contribution. The performed joint optimization of the number of activated MIMO layers and the number of transmitted bits per subcarrier along with the appropriate allocation of the transmit power shows that not necessarily all user-specific MIMO layers per subcarrier have to be activated in order to minimize the overall BER under the constraint of a given fixed data throughput.

Modulation-mode Assignment in SVD-assisted Multiuser MIMO-OFDM Systems

In order to comply with the demand on increasing available data rates in particular in wireless technologies, systems with multiple transmit and receive antennas, also called MIMO (multiple-input multiple-output) systems, have become indispensable for future generations of wireless systems. Due to the strongly increasing demand in high-data rate transmission systems, frequency non-selective MIMO links have reached a state of maturity and frequency selective MIMO links are in the focus of interest. In this field, the combination of MIMO transmission and OFDM (orthogonal frequency division multiplexing) can be considered as an essential part of fulfilling the requirements of future generations of wireless systems. However, single-user scenarios have reached a state of maturity. By contrast multiple users’ scenarios require substantial further research, where in comparison to ZF (zero-forcing) multiuser transmission techniques, the individual user’s channel characteristics are taken into consideration in this contribution. The performed joint optimization of the number of activated MIMO layers and the number of transmitted bits per subcarrier along with the appropriate allocation of the transmit power shows that not necessarily all user-specific MIMO layers per subcarrier have to be activated in order to minimize the overall BER under the constraint of a given fixed data throughput

Robustness Analysis of Eigenstructure Assignment Controllers on Rigid-Flexible Satellites

Flexible spacecraft with attached solar panels may exhibit undesired vibrations and structural deformations. These types of vehicles show an intrinsic coupling of the elements of the structure. The attitude maneuvers performed by flexible spacecraft may cause non-desired deflections of attached flexible elements. Any attitude and orbit control system generally solves these problems using filters that are designed to attenuate the relative deflections of flexible appendages. In this paper, we propose a method for designing attitude static controllers using an eigenstructure assignment (EA) method. A set of requirements were specified from our understanding of the system modes in an open loop. Exhaustive theoretical and numerical simulations were performed on special cases to verify the controller design procedure. In the design of the controller, we considered all of the aspects that relate to the eigenstructure assignment. The primary objective of this paper is to demonstrate the feasibility of obtaining a high degree of decoupling for some selected modes via the application of an EA method. Finally a robustness analysis is perform to the system together with the designed controller by means of a mu-analysis

A dual-band priority assignment algorithm for dynamic QoS resource management

Future high-quality consumer electronics will contain a number of applications running in a highly dynamic environment, and their execution will need to be efficiently arbitrated by the underlying platform software. The multimedia applications that currently execute in such similar contexts face frequent run-time variations in their resource demands, originated by the greedy nature of the multimedia processing itself. Changes in resource demands are triggered by numerous reasons (e.g. a switch in the input media compression format). Such situations require real-time adaptation mechanisms to adjust the system operation to the new requirements, and this must be done seamlessly to satisfy the user experience. One solution for efficiently managing application execution is to apply quality of service resource management techniques, based on assigning and enforcing resource contracts to applications. Most resource management solutions provide temporal isolation by enforcing resource assignments and avoiding any resource overruns. However, this has a clear limitation over the cost-effective resource usage. This paper presents a simple priority assignment scheme based on uniform priority bands to allow that greedy multimedia tasks incur in safe overruns that increase resource usage and do not threaten the timely execution of non-overrunning tasks. Experimental results show that the proposed priority assignment scheme in combination with a resource accounting mechanism preserves timely multimedia execution and delivery, achieves a higher cost-effective processor usage, and guarantees the execution isolation of non-overrunning tasks.

Genetic variation and inference of demographic histories in non-model species

Both long-term environmental changes such as those driven by the glacial cycles and more recent anthropogenic impacts have had major effects on the past demography in wild organisms. Within species, these changes are reflected in the amount and distribution of neutral genetic variation. In this thesis, mitochondrial and microsatellite DNA was analysed to investigate how environmental and anthropogenic factors have affected genetic diversity and structure in four ecologically different animal species. Paper I describes the post-glacial recolonisation history of the speckled-wood butterfly (Pararge aegeria) in Northern Europe. A decrease in genetic diversity with latitude and a marked population structure were uncovered, consistent with a hypothesis of repeated founder events during the postglacial recolonisation. Moreover, Approximate Bayesian Computation analyses indicate that the univoltine populations in Scandinavia and Finland originate from recolonisations along two routes, one on each side of the Baltic. Paper II aimed to investigate how past sea-level rises affected the population history of the convict surgeonfish (Acanthurus triostegus) in the Indo-Pacific. Assessment of the species’ demographic history suggested a population expansion that occurred approximately at the end of the last glaciation. Moreover, the results demonstrated an overall lack of phylogeographic structure, probably due to the high dispersal rates associated with the species’ pelagic larval stage. Populations at the species’ eastern range margin were significantly differentiated from other populations, which likely is a consequence of their geographic isolation. In Paper III, we assessed the effect of human impact on the genetic variation of European moose (Alces alces) in Sweden. Genetic analyses revealed a spatial structure with two genetic clusters, one in northern and one in southern Sweden, which were separated by a narrow transition zone. Moreover, demographic inference suggested a recent population bottleneck. The inferred timing of this bottleneck coincided with a known reduction in population size in the 19th and early 20th century due to high hunting pressure. In Paper IV, we examined the effect of an indirect but well-described human impact, via environmental toxic chemicals (PCBs), on the genetic variation of Eurasian otters (Lutra lutra) in Sweden. Genetic clustering assignment revealed differentiation between otters in northern and southern Sweden, but also in the Stockholm region. ABC analyses indicated a decrease in effective population size in both northern and southern Sweden. Moreover, comparative analyses of historical and contemporary samples demonstrated a more severe decline in genetic diversity in southern Sweden compared to northern Sweden, in agreement with the levels of PCBs found.

Non-indigenous species in the North and Baltic Seas and the Great Lakes-St. Lawrence River region

List of non-indigenous species (NIS) established in the Great Lakes-St. Lawrence River region and the North and Baltic Seas region, their geographic origin, and taxonomic assignment. Asterisks mark the NIS that occur in both the North and Baltic Seas and the Great Lakes-St. Lawrence River regions. GL, SL, NW, NE, SW and SE denote the Great Lakes, St. Lawrence River, north-west, north-east, south-west, and south-east, respectively. Eurasia represents inland freshwaters except Yangtze River, Indo-Pacific represents Indian Ocean and the archipelago of Indonesia, Malaysia, and Pilipinas, North America (N America) represents inland freshwaters except the Laurentian Great Lakes, St. Lawrence and Mississippi Rivers, while Australia, New Zealand, Africa and South America (S America) cover all inland freshwaters in these areas.

Self-refraction, ready-made glasses and quality of life among rural myopic Chinese children: a non-inferiority randomized trial.

PURPOSE: To study, for the first time, the effect of wearing ready-made glasses and glasses with power determined by self-refraction on children's quality of life. METHODS: This is a randomized, double-masked non-inferiority trial. Children in grades 7 and 8 (age 12-15 years) in nine Chinese secondary schools, with presenting visual acuity (VA) ≤6/12 improved with refraction to ≥6/7.5 bilaterally, refractive error ≤-1.0 D and <2.0 D of anisometropia and astigmatism bilaterally, were randomized to receive ready-made spectacles (RM) or identical-appearing spectacles with power determined by: subjective cycloplegic retinoscopy by a university optometrist (U), a rural refractionist (R) or non-cycloplegic self-refraction (SR). Main study outcome was global score on the National Eye Institute Refractive Error Quality of Life-42 (NEI-RQL-42) after 2 months of wearing study glasses, comparing other groups with the U group, adjusting for baseline score. RESULTS: Only one child (0.18%) was excluded for anisometropia or astigmatism. A total of 426 eligible subjects (mean age 14.2 years, 84.5% without glasses at baseline) were allocated to U [103 (24.2%)], RM [113 (26.5%)], R [108 (25.4%)] and SR [102 (23.9%)] groups, respectively. Baseline and endline score data were available for 398 (93.4%) of subjects. In multiple regression models adjusting for baseline score, older age (p = 0.003) and baseline spectacle wear (p = 0.016), but not study group assignment, were significantly associated with lower final score. CONCLUSION: Quality of life wearing ready-mades or glasses based on self-refraction did not differ from that with cycloplegic refraction by an experienced optometrist in this non-inferiority trial.

JOINT ENERGY BEAMFORMING AND TIME ASSIGNMENT FOR COOPERATIVE WIRELESS POWERED NETWORKS

Wireless power transfer (WPT) and radio frequency (RF)-based energy har- vesting arouses a new wireless network paradigm termed as wireless powered com- munication network (WPCN), where some energy-constrained nodes are enabled to harvest energy from the RF signals transferred by other energy-sufficient nodes to support the communication operations in the network, which brings a promising approach for future energy-constrained wireless network design. In this paper, we focus on the optimal WPCN design. We consider a net- work composed of two communication groups, where the first group has sufficient power supply but no available bandwidth, and the second group has licensed band- width but very limited power to perform required information transmission. For such a system, we introduce the power and bandwidth cooperation between the two groups so that both group can accomplish their expected information delivering tasks. Multiple antennas are employed at the hybrid access point (H-AP) to en- hance both energy and information transfer efficiency and the cooperative relaying is employed to help the power-limited group to enhance its information transmission throughput. Compared with existing works, cooperative relaying, time assignment, power allocation, and energy beamforming are jointly designed in a single system. Firstly, we propose a cooperative transmission protocol for the considered system, where group 1 transmits some power to group 2 to help group 2 with information transmission and then group 2 gives some bandwidth to group 1 in return. Sec- ondly, to explore the information transmission performance limit of the system, we formulate two optimization problems to maximize the system weighted sum rate by jointly optimizing the time assignment, power allocation, and energy beamforming under two different power constraints, i.e., the fixed power constraint and the aver- age power constraint, respectively. In order to make the cooperation between the two groups meaningful and guarantee the quality of service (QoS) requirements of both groups, the minimal required data rates of the two groups are considered as constraints for the optimal system design. As both problems are non-convex and have no known solutions, we solve it by using proper variable substitutions and the semi-definite relaxation (SDR). We theoretically prove that our proposed solution method can guarantee to find the global optimal solution. Thirdly, consider that the WPCN has promising application potentials in future energy-constrained net- works, e.g., wireless sensor network (WSN), wireless body area network (WBAN) and Internet of Things (IoT), where the power consumption is very critical. We investigate the minimal power consumption optimal design for the considered co- operation WPCN. For this, we formulate an optimization problem to minimize the total consumed power by jointly optimizing the time assignment, power allocation, and energy beamforming under required data rate constraints. As the problem is also non-convex and has no known solutions, we solve it by using some variable substitutions and the SDR method. We also theoretically prove that our proposed solution method for the minimal power consumption design guarantees the global optimal solution. Extensive experimental results are provided to discuss the system performance behaviors, which provide some useful insights for future WPCN design. It shows that the average power constrained system achieves higher weighted sum rate than the fixed power constrained system. Besides, it also shows that in such a WPCN, relay should be placed closer to the multi-antenna H-AP to achieve higher weighted sum rate and consume lower total power.

Une comparaison des journaux non structurés et des journaux structurés rédigés par les étudiants en hygiène dentaire

The Canadian Dental Hygienists Association (CDHA) has indicated that there is a need for research in education in the field of dental hygiene. It seems that when compared to the nursing profession, the profession of dental hygiene is only in the earliest stages of investigating ways of teaching critical thinking. The faculty of the dental hygiene program at John Abbott College has always valued the skill of self-assessment in the students, yet there are few specific learning activities provided whereby the students can learn how to perfect and work on this invaluable skill of self-reflection in order to better self-assess. Although self-assessment is required of the students upon the completion of each clinical experience in Clinic 1, 2 and 3, a modest amount of clinical time is allotted to reflect upon this most important skill. It appears that more could be done to prepare our students to assess their learning and clinical practice. Self-reflection as an essential element of practice has a valid place in professional education. The purpose of conducting this study was to find out whether unstructured or structured self-reflective journal writing is a sound pedagogical technique to encourage dental hygiene students’ self-assessment through self-reflection. The research design for the project was a single case study. The paradigm for the study was chosen with a purposeful selection of participants, involving twenty-seven, third-year dental hygiene students at John Abbott College. The students were arbitrarily enrolled in two sections, which for the purpose of this study were referred to as Group A and Group B. Three duplicated coded anonymous journal entries from each student were collected over a ten-week period during the Fall 2009 semester. To examine the students’ level of self-reflection, two methods were used. First a content analysis of reflective journals was used to ascertain the level and substance of the reflections from their clinical experiences with the intent of looking more specifically at the students’ self-assessment. The journal entries were coded and analyzed after the grades were submitted at the end of the school term. This was followed by the distribution of an anonymous questionnaire to the students in both sections. The responses of the questionnaire were tabulated and analyzed. An analysis was done on the data collected in order to determine whether age, education and or mother tongue of the students in both Groups A and B had an influence on their perceptions of journal writing, as well as the student’s opinions about the value of journal writing. This questionnaire included two open-ended questions to assist in gathering additional data on the student’s thoughts on writing journals. A content analysis of the qualitative data collected from the open-ended questions in the questionnaire was also analyzed. Results indicated there were very few differences in the level of self-reflection leading to self-assessment. However, students in Group B who were assigned structured journals showed more evidence of deeper learning. Taken as a whole, the journal entries clearly showed the students were involved in ‘reflection-on-action’ of their clinical experiences (Schon 1987, as cited in Asadoorian & Batty, 2005). The quality of the responses for the most part indicated the students took the time and effort to record their perceptions of their clinical experiences. It is important to note that the results do indicate that students did show a need to self-reflect and assess. The students did in fact validate the importance of reflection through journal writing, even though they did not particularly like it as an added assignment. The journals were found to be very helpful to the research in getting to know what the issues were that held the students’ attention. They explained how and to what extent the students developed relationships with their clients. It was obvious that clinicians have an impact and influence on student learning. The students value the help, role modeling, patience, encouraging words and or gestures, positive reinforcement, and understanding provided by their clinicians. This research provides some evidence that students do believe that self-reflection through structured journal writing helped them better prepare for future clinical sessions with their clients. Our goal as educators should be to encourage dental hygiene students to self-assess through written self-reflection as an established practice for deeper learning.

Egfr Activating Mutations And Their Association With Response To Platinum-doublet Chemotherapy In Brazilian Non-small Cell Lung Cancer Patients.

The aim of the study was to analyze the frequency of epidermal growth factor receptor (EGFR) mutations in Brazilian non-small cell lung cancer patients and to correlate these mutations with response to benefit of platinum-based chemotherapy in non-small cell lung cancer (NSCLC). Our cohort consisted of prospective patients with NSCLCs who received chemotherapy (platinum derivates plus paclitaxel) at the [UNICAMP], Brazil. EGFR exons 18-21 were analyzed in tumor-derived DNA. Fifty patients were included in the study (25 with adenocarcinoma). EGFR mutations were identified in 6/50 (12 %) NSCLCs and in 6/25 (24 %) adenocarcinomas; representing the frequency of EGFR mutations in a mostly self-reported White (82.0 %) southeastern Brazilian population of NSCLCs. Patients with NSCLCs harboring EGFR exon 19 deletions or the exon 21 L858R mutation were found to have a higher chance of response to platinum-paclitaxel (OR 9.67 [95 % CI 1.03-90.41], p = 0.047). We report the frequency of EGFR activating mutations in a typical southeastern Brazilian population with NSCLC, which are similar to that of other countries with Western European ethnicity. EGFR mutations seem to be predictive of a response to platinum-paclitaxel, and additional studies are needed to confirm or refute this relationship.

Fatty Acid Synthase Inhibitors Induce Apoptosis In Non-tumorigenic Melan-a Cells Associated With Inhibition Of Mitochondrial Respiration.

The metabolic enzyme fatty acid synthase (FASN) is responsible for the endogenous synthesis of palmitate, a saturated long-chain fatty acid. In contrast to most normal tissues, a variety of human cancers overexpress FASN. One such cancer is cutaneous melanoma, in which the level of FASN expression is associated with tumor invasion and poor prognosis. We previously reported that two FASN inhibitors, cerulenin and orlistat, induce apoptosis in B16-F10 mouse melanoma cells via the intrinsic apoptosis pathway. Here, we investigated the effects of these inhibitors on non-tumorigenic melan-a cells. Cerulenin and orlistat treatments were found to induce apoptosis and decrease cell proliferation, in addition to inducing the release of mitochondrial cytochrome c and activating caspases-9 and -3. Transfection with FASN siRNA did not result in apoptosis. Mass spectrometry analysis demonstrated that treatment with the FASN inhibitors did not alter either the mitochondrial free fatty acid content or composition. This result suggests that cerulenin- and orlistat-induced apoptosis events are independent of FASN inhibition. Analysis of the energy-linked functions of melan-a mitochondria demonstrated the inhibition of respiration, followed by a significant decrease in mitochondrial membrane potential (ΔΨm) and the stimulation of superoxide anion generation. The inhibition of NADH-linked substrate oxidation was approximately 40% and 61% for cerulenin and orlistat treatments, respectively, and the inhibition of succinate oxidation was approximately 46% and 52%, respectively. In contrast, no significant inhibition occurred when respiration was supported by the complex IV substrate N,N,N',N'-tetramethyl-p-phenylenediamine (TMPD). The protection conferred by the free radical scavenger N-acetyl-cysteine indicates that the FASN inhibitors induced apoptosis through an oxidative stress-associated mechanism. In combination, the present results demonstrate that cerulenin and orlistat induce apoptosis in non-tumorigenic cells via mitochondrial dysfunction, independent of FASN inhibition.

Imported Malaria In A Non-endemic Area: The Experience Of The University Of Campinas Hospital In The Brazilian Southeast.

Although malaria in Brazil almost exclusively occurs within the boundaries of the Amazon Region, some concerns are raised regarding imported malaria to non-endemic areas of the country, notably increased incidence of complications due to delayed diagnoses. However, although imported malaria in Brazil represents a major health problem, only a few studies have addressed this subject. A retrospective case series is presented in which 263 medical charts were analysed to investigate the clinical and epidemiological characterization of malaria cases that were diagnosed and treated at Hospital & Clinics, State University of Campinas between 1998 and 2011. Amongst all medical charts analysed, 224 patients had a parasitological confirmed diagnosis of malaria. Plasmodium vivax and Plasmodium falciparum were responsible for 67% and 30% of the infections, respectively. The majority of patients were male (83%) of a productive age (median, 37 years old). Importantly, severe complications did not differ significantly between P. vivax (14 cases, 9%) and P. falciparum (7 cases, 10%) infections. Severe malaria cases were frequent among imported cases in Brazil outside of the Amazon area. The findings reinforce the idea that P. vivax infections in Brazil are not benign, regardless the endemicity of the area studied. Moreover, as the hospital is located in a privileged site, it could be used for future studies of malaria relapses and primaquine resistance mechanisms. Finally, based on the volume of cases treated and the secondary complications, referral malaria services are needed in the non-endemic areas of Brazil for a rapid and efficient and treatment.