Gambling against neglect: unconscious spatial biases induced by reward reinforcement in healthy people and brain-damaged patients.

Orienting attention in space recruits fronto-parietal networks whose damage results in unilateral spatial neglect. However, attention orienting may also be governed by emotional and motivational factors; but it remains unknown whether these factors act through a modulation of the fronto-parietal attentional systems or distinct neural pathways. Here we asked whether attentional orienting is affected by learning about the reward value of targets in a visual search task, in a spatially specific manner, and whether these effects are preserved in right-brain damaged patients with left spatial neglect. We found that associating rewards with left-sided (but not right-sided) targets during search led to progressive exploration biases towards left space, in both healthy people and neglect patients. Such spatially specific biases occurred even without any conscious awareness of the asymmetric reward contingencies. These results show that reward-induced modulations of space representation are preserved despite a dysfunction of fronto-parietal networks associated with neglect, and therefore suggest that they may arise through spared subcortical networks directly acting on sensory processing and/or oculomotor circuits. These effects could be usefully exploited for potentiating rehabilitation strategies in neglect patients.

Reinforcement learning utilizes proxemics: an avatar learns to manipulate the position of people in immersive virtual reality

A reinforcement learning (RL) method was used to train a virtual character to move participants to a specified location. The virtual environment depicted an alleyway displayed through a wide field-of-view head-tracked stereo head-mounted display. Based on proxemics theory, we predicted that when the character approached within a personal or intimate distance to the participants, they would be inclined to move backwards out of the way. We carried out a between-groups experiment with 30 female participants, with 10 assigned arbitrarily to each of the following three groups: In the Intimate condition the character could approach within 0.38m and in the Social condition no nearer than 1.2m. In the Random condition the actions of the virtual character were chosen randomly from among the same set as in the RL method, and the virtual character could approach within 0.38m. The experiment continued in each case until the participant either reached the target or 7 minutes had elapsed. The distributions of the times taken to reach the target showed significant differences between the three groups, with 9 out of 10 in the Intimate condition reaching the target significantly faster than the 6 out of 10 who reached the target in the Social condition. Only 1 out of 10 in the Random condition reached the target. The experiment is an example of applied presence theory: we rely on the many findings that people tend to respond realistically in immersive virtual environments, and use this to get people to achieve a task of which they had been unaware. This method opens up the door for many such applications where the virtual environment adapts to the responses of the human participants with the aim of achieving particular goals.

The use of external mesh reinforcement to reduce intimal hyperplasia and preserve the structure of human saphenous veins.

The saphenous vein is the conduit of choice in bypass graft procedures. Haemodynamic factors play a major role in the development of intimal hyperplasia (IH), and subsequent bypass failure. To evaluate the potential protective effect of external reinforcement on such a failure, we developed an ex vivo model for the perfusion of segments of human saphenous veins under arterial shear stress. In veins submitted to pulsatile high pressure (mean pressure at 100 mmHg) for 3 or 7 days, the use of an external macroporous polyester mesh 1) prevented the dilatation of the vessel, 2) decreased the development of IH, 3) reduced the apoptosis of smooth muscle cells, and the subsequent fibrosis of the media layer, 4) prevented the remodelling of extracellular matrix through the up-regulation of matrix metalloproteinases (MMP-2, MMP-9) and plasminogen activator type I. The data show that, in an experimental ex vivo setting, an external scaffold decreases IH and maintains the integrity of veins exposed to arterial pressure, via increase in shear stress and decrease wall tension, that likely contribute to trigger selective molecular and cellular changes.

A model for the evolution of reinforcement learning in fluctuating games

Many species are able to learn to associate behaviours with rewards as this gives fitness advantages in changing environments. Social interactions between population members may, however, require more cognitive abilities than simple trial-and-error learning, in particular the capacity to make accurate hypotheses about the material payoff consequences of alternative action combinations. It is unclear in this context whether natural selection necessarily favours individuals to use information about payoffs associated with nontried actions (hypothetical payoffs), as opposed to simple reinforcement of realized payoff. Here, we develop an evolutionary model in which individuals are genetically determined to use either trial-and-error learning or learning based on hypothetical reinforcements, and ask what is the evolutionarily stable learning rule under pairwise symmetric two-action stochastic repeated games played over the individual's lifetime. We analyse through stochastic approximation theory and simulations the learning dynamics on the behavioural timescale, and derive conditions where trial-and-error learning outcompetes hypothetical reinforcement learning on the evolutionary timescale. This occurs in particular under repeated cooperative interactions with the same partner. By contrast, we find that hypothetical reinforcement learners tend to be favoured under random interactions, but stable polymorphisms can also obtain where trial-and-error learners are maintained at a low frequency. We conclude that specific game structures can select for trial-and-error learning even in the absence of costs of cognition, which illustrates that cost-free increased cognition can be counterselected under social interactions.

Reinforcement versus Fluidization in Cytoskeletal Mechanoresponsiveness

Every adherent eukaryotic cell exerts appreciable traction forces upon its substrate. Moreover, every resident cell within the heart, great vessels, bladder, gut or lung routinely experiences large periodic stretches. As an acute response to such stretches the cytoskeleton can stiffen, increase traction forces and reinforce, as reported by some, or can soften and fluidize, as reported more recently by our laboratory, but in any given circumstance it remains unknown which response might prevail or why. Using a novel nanotechnology, we show here that in loading conditions expected in most physiological circumstances the localized reinforcement response fails to scale up to the level of homogeneous cell stretch; fluidization trumps reinforcement. Whereas the reinforcement response is known to be mediated by upstream mechanosensing and downstream signaling, results presented here show the fluidization response to be altogether novel: it is a direct physical effect of mechanical force acting upon a structural lattice that is soft and fragile. Cytoskeletal softness and fragility, we argue, is consistent with early evolutionary adaptations of the eukaryotic cell to material properties of a soft inert microenvironment.

Research on the suitability of organosolv semi-chemical triticale fibers as reinforcement for recycled hdpe composites

The main objective of this research was to study the feasibility of incorporating organosolv semi-chemical triticale fibers as the reinforcing element in recycled high density polyethylene (HDPE). In the first step, triticale fibers were characterized in terms of chemical composition and compared with other biomass species (wheat, rye, softwood, and hardwood). Then, organosolv semi-chemical triticale fibers were prepared by the ethanolamine process. These fibers were characterized in terms of its yield, kappa number, fiber length/diameter ratio, fines, and viscosity; the obtained results were compared with those of eucalypt kraft pulp. In the second step, the prepared fibers were examined as a reinforcing element for recycled HDPE composites. Coupled and non-coupled HDPE composites were prepared and tested for tensile properties. Results showed that with the addition of the coupling agent maleated polyethylene (MAPE), the tensile properties of composites were significantly improved, as compared to non-coupled samples and the plain matrix. Furthermore, the influence of MAPE on the interfacial shear strength (IFSS) was studied. The contributions of both fibers and matrix to the composite strength were also studied. This was possible by the use of a numerical iterative method based on the Bowyer-Bader and Kelly-Tyson equations

Signaled two-way avoidance learning using electrical stimulation of the inferior colliculus as negative reinforcement: effects of visual and auditory cues as warning stimuli

The inferior colliculus is a primary relay for the processing of auditory information in the brainstem. The inferior colliculus is also part of the so-called brain aversion system as animals learn to switch off the electrical stimulation of this structure. The purpose of the present study was to determine whether associative learning occurs between aversion induced by electrical stimulation of the inferior colliculus and visual and auditory warning stimuli. Rats implanted with electrodes into the central nucleus of the inferior colliculus were placed inside an open-field and thresholds for the escape response to electrical stimulation of the inferior colliculus were determined. The rats were then placed inside a shuttle-box and submitted to a two-way avoidance paradigm. Electrical stimulation of the inferior colliculus at the escape threshold (98.12 ± 6.15 (A, peak-to-peak) was used as negative reinforcement and light or tone as the warning stimulus. Each session consisted of 50 trials and was divided into two segments of 25 trials in order to determine the learning rate of the animals during the sessions. The rats learned to avoid the inferior colliculus stimulation when light was used as the warning stimulus (13.25 ± 0.60 s and 8.63 ± 0.93 s for latencies and 12.5 ± 2.04 and 19.62 ± 1.65 for frequencies in the first and second halves of the sessions, respectively, P<0.01 in both cases). No significant changes in latencies (14.75 ± 1.63 and 12.75 ± 1.44 s) or frequencies of responses (8.75 ± 1.20 and 11.25 ± 1.13) were seen when tone was used as the warning stimulus (P>0.05 in both cases). Taken together, the present results suggest that rats learn to avoid the inferior colliculus stimulation when light is used as the warning stimulus. However, this learning process does not occur when the neutral stimulus used is an acoustic one. Electrical stimulation of the inferior colliculus may disturb the signal transmission of the stimulus to be conditioned from the inferior colliculus to higher brain structures such as amygdala

Polymer Nanocomposites: Crystallization, Reinforcement and Conductivity through SWNTs

The current research investigates the possibility of using single walled carbon nanotubes (SWNTs) as filler in polymers to impart several properties to the matrix polymer. SWNTs in a polymer matrix like poly(ethylene terephthalate) induce nucleation in its melt crystallization, provide effective reinforcement and impart electrical conductivity. We adopt a simple melt compounding technique for incorporating the nanotubes into the polymer matrix. For attaining a better dispersion of the filler, an ultrasound assisted dissolution-evaporation method has also been tried. The resulting enhancement in the materials properties indicates an improved disentanglement of the nanotube ropes, which in turn provides effective matrix-filler interaction. PET-SWNT nanocomposite fibers prepared through melt spinning followed by subsequent drawing are also found to have significantly higher mechanical propertiesas compared to pristine PET fiber.SWNTs also find applications in composites based on elastomers such as natural rubber as they can impart electrical conductivity with simultaneous improvement in the mechanical properties.

Reinforcement Learning Approaches to Power System Scheduling

One major component of power system operation is generation scheduling. The objective of the work is to develop efficient control strategies to the power scheduling problems through Reinforcement Learning approaches. The three important active power scheduling problems are Unit Commitment, Economic Dispatch and Automatic Generation Control. Numerical solution methods proposed for solution of power scheduling are insufficient in handling large and complex systems. Soft Computing methods like Simulated Annealing, Evolutionary Programming etc., are efficient in handling complex cost functions, but find limitation in handling stochastic data existing in a practical system. Also the learning steps are to be repeated for each load demand which increases the computation time.Reinforcement Learning (RL) is a method of learning through interactions with environment. The main advantage of this approach is it does not require a precise mathematical formulation. It can learn either by interacting with the environment or interacting with a simulation model. Several optimization and control problems have been solved through Reinforcement Learning approach. The application of Reinforcement Learning in the field of Power system has been a few. The objective is to introduce and extend Reinforcement Learning approaches for the active power scheduling problems in an implementable manner. The main objectives can be enumerated as:(i) Evolve Reinforcement Learning based solutions to the Unit Commitment Problem.(ii) Find suitable solution strategies through Reinforcement Learning approach for Economic Dispatch. (iii) Extend the Reinforcement Learning solution to Automatic Generation Control with a different perspective. (iv) Check the suitability of the scheduling solutions to one of the existing power systems.First part of the thesis is concerned with the Reinforcement Learning approach to Unit Commitment problem. Unit Commitment Problem is formulated as a multi stage decision process. Q learning solution is developed to obtain the optimwn commitment schedule. Method of state aggregation is used to formulate an efficient solution considering the minimwn up time I down time constraints. The performance of the algorithms are evaluated for different systems and compared with other stochastic methods like Genetic Algorithm.Second stage of the work is concerned with solving Economic Dispatch problem. A simple and straight forward decision making strategy is first proposed in the Learning Automata algorithm. Then to solve the scheduling task of systems with large number of generating units, the problem is formulated as a multi stage decision making task. The solution obtained is extended in order to incorporate the transmission losses in the system. To make the Reinforcement Learning solution more efficient and to handle continuous state space, a fimction approximation strategy is proposed. The performance of the developed algorithms are tested for several standard test cases. Proposed method is compared with other recent methods like Partition Approach Algorithm, Simulated Annealing etc.As the final step of implementing the active power control loops in power system, Automatic Generation Control is also taken into consideration.Reinforcement Learning has already been applied to solve Automatic Generation Control loop. The RL solution is extended to take up the approach of common frequency for all the interconnected areas, more similar to practical systems. Performance of the RL controller is also compared with that of the conventional integral controller.In order to prove the suitability of the proposed methods to practical systems, second plant ofNeyveli Thennal Power Station (NTPS IT) is taken for case study. The perfonnance of the Reinforcement Learning solution is found to be better than the other existing methods, which provide the promising step towards RL based control schemes for practical power industry.Reinforcement Learning is applied to solve the scheduling problems in the power industry and found to give satisfactory perfonnance. Proposed solution provides a scope for getting more profit as the economic schedule is obtained instantaneously. Since Reinforcement Learning method can take the stochastic cost data obtained time to time from a plant, it gives an implementable method. As a further step, with suitable methods to interface with on line data, economic scheduling can be achieved instantaneously in a generation control center. Also power scheduling of systems with different sources such as hydro, thermal etc. can be looked into and Reinforcement Learning solutions can be achieved.

Studies On Vulcanization, Rheology And Reinforcement Of Natural Rubber Latex With Special Reference To Accelerator Combinations, Surface Active Agents And Gamma Irradiation

In the present work, studies on vulcanization, rheology and reinforcement of natural rubber latex with special reference to accelerator combinations, surface active agents and gamma irradiation have been undertaken. In vulcanization, the choice of vulcanization system, the extent and mc-zie of vulcanization and network structure of the vulcanizate are important factors contributing to the overall quality of the product. The vulcanization system may be conventional type using elemental sulfur or a system involving sulfur donors. The latter type is used mainly in the manufacture of heat resistant products. For improving the technical properties of the products such as modulus and tensile strength, different accelerator combinations are used. It is known that accelerators have a strong effect on the physical properties of rubber vulcanizates. A perusal of the literature indicates that fundamental studies on the above aspects of latex technology are very limited. Thereforea systematic study on vulcanization, rheology and reinforcement of natural rubber latex with reference to the effect of accelerator combinations, surface active agents and gamma irradiation has been undertaken. The preparation and evaluation of some products like latex thread was also undertaken as a part of the study. The thesis consists of six chapter

A Reinforcement Learning Algorithm to Economic Dispatch Considering Transmission Losses

Reinforcement Learning (RL) refers to a class of learning algorithms in which learning system learns which action to take in different situations by using a scalar evaluation received from the environment on performing an action. RL has been successfully applied to many multi stage decision making problem (MDP) where in each stage the learning systems decides which action has to be taken. Economic Dispatch (ED) problem is an important scheduling problem in power systems, which decides the amount of generation to be allocated to each generating unit so that the total cost of generation is minimized without violating system constraints. In this paper we formulate economic dispatch problem as a multi stage decision making problem. In this paper, we also develop RL based algorithm to solve the ED problem. The performance of our algorithm is compared with other recent methods. The main advantage of our method is it can learn the schedule for all possible demands simultaneously.

Reinforcement Learning approaches to Economic Dispatch problem

This paper presents Reinforcement Learning (RL) approaches to Economic Dispatch problem. In this paper, formulation of Economic Dispatch as a multi stage decision making problem is carried out, then two variants of RL algorithms are presented. A third algorithm which takes into consideration the transmission losses is also explained. Efficiency and flexibility of the proposed algorithms are demonstrated through different representative systems: a three generator system with given generation cost table, IEEE 30 bus system with quadratic cost functions, 10 generator system having piecewise quadratic cost functions and a 20 generator system considering transmission losses. A comparison of the computation times of different algorithms is also carried out.

Reinforcement Learning Solution for Unit Commitment Problem Considering Minimum Start Up and Shut Down Times

Unit Commitment Problem (UCP) in power system refers to the problem of determining the on/ off status of generating units that minimize the operating cost during a given time horizon. Since various system and generation constraints are to be satisfied while finding the optimum schedule, UCP turns to be a constrained optimization problem in power system scheduling. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision making task and an efficient Reinforcement Learning solution is formulated considering minimum up time /down time constraints. The correctness and efficiency of the developed solutions are verified for standard test systems

A Reinforcement Learning Approach to Economic Dispatch using Neural Networks

This paper presents a Reinforcement Learning (RL) approach to economic dispatch (ED) using Radial Basis Function neural network. We formulate the ED as an N stage decision making problem. We propose a novel architecture to store Qvalues and present a learning algorithm to learn the weights of the neural network. Even though many stochastic search techniques like simulated annealing, genetic algorithm and evolutionary programming have been applied to ED, they require searching for the optimal solution for each load demand. Also they find limitation in handling stochastic cost functions. In our approach once we learn the Q-values, we can find the dispatch for any load demand. We have recently proposed a RL approach to ED. In that approach, we could find only the optimum dispatch for a set of specified discrete values of power demand. The performance of the proposed algorithm is validated by taking IEEE 6 bus system, considering transmission losses

Reinforcement Learning solution for Unit Commitment Problem through pursuit method

Unit commitment is an optimization task in electric power generation control sector. It involves scheduling the ON/OFF status of the generating units to meet the load demand with minimum generation cost satisfying the different constraints existing in the system. Numerical solutions developed are limited for small systems and heuristic methodologies find difficulty in handling stochastic cost functions associated with practical systems. This paper models Unit Commitment as a multi stage decision task and Reinforcement Learning solution is formulated through one efficient exploration strategy: Pursuit method. The correctness and efficiency of the developed solutions are verified for standard test systems