Simple heuristics for the assembly line worker assignment and balancing problem

We propose simple heuristics for the assembly line worker assignment and balancing problem. This problem typically occurs in assembly lines in sheltered work centers for the disabled. Different from the well-known simple assembly line balancing problem, the task execution times vary according to the assigned worker. We develop a constructive heuristic framework based on task and worker priority rules defining the order in which the tasks and workers should be assigned to the workstations. We present a number of such rules and compare their performance across three possible uses: as a stand-alone method, as an initial solution generator for meta-heuristics, and as a decoder for a hybrid genetic algorithm. Our results show that the heuristics are fast, they obtain good results as a stand-alone method and are efficient when used as a initial solution generator or as a solution decoder within more elaborate approaches.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior, and in particular in behavioral decision making. Such decision making is likely to involve the integration of many synaptic events in space and time. However, using a single reinforcement signal to modulate synaptic plasticity, as suggested in classical reinforcement learning algorithms, a twofold problem arises. Different synapses will have contributed differently to the behavioral decision, and even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike-time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward, but also by a population feedback signal. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference (TD) based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task, the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second task involves an action sequence which is itself extended in time and reward is only delivered at the last action, as it is the case in any type of board-game. The third task is the inspection game that has been studied in neuroeconomics, where an inspector tries to prevent a worker from shirking. Applying our algorithm to this game yields a learning behavior which is consistent with behavioral data from humans and monkeys, revealing themselves properties of a mixed Nash equilibrium. The examples show that our neuronal implementation of reward based learning copes with delayed and stochastic reward delivery, and also with the learning of mixed strategies in two-opponent games.

Spatio-temporal credit assignment in population learning

Learning by reinforcement is important in shaping animal behavior. But behavioral decision making is likely to involve the integration of many synaptic events in space and time. So in using a single reinforcement signal to modulate synaptic plasticity a twofold problem arises. Different synapses will have contributed differently to the behavioral decision and, even for one and the same synapse, releases at different times may have had different effects. Here we present a plasticity rule which solves this spatio-temporal credit assignment problem in a population of spiking neurons. The learning rule is spike time dependent and maximizes the expected reward by following its stochastic gradient. Synaptic plasticity is modulated not only by the reward but by a population feedback signal as well. While this additional signal solves the spatial component of the problem, the temporal one is solved by means of synaptic eligibility traces. In contrast to temporal difference based approaches to reinforcement learning, our rule is explicit with regard to the assumed biophysical mechanisms. Neurotransmitter concentrations determine plasticity and learning occurs fully online. Further, it works even if the task to be learned is non-Markovian, i.e. when reinforcement is not determined by the current state of the system but may also depend on past events. The performance of the model is assessed by studying three non-Markovian tasks. In the first task the reward is delayed beyond the last action with non-related stimuli and actions appearing in between. The second one involves an action sequence which is itself extended in time and reward is only delivered at the last action, as is the case in any type of board-game. The third is the inspection game that has been studied in neuroeconomics. It only has a mixed Nash equilibrium and exemplifies that the model also copes with stochastic reward delivery and the learning of mixed strategies.

Spatio-temporal credit assignment in population learning

We present a model for plasticity induction in reinforcement learning which is based on a cascade of synaptic memory traces. In the cascade of these so called eligibility traces presynaptic input is first corre lated with postsynaptic events, next with the behavioral decisions and finally with the external reinforcement. A population of leaky integrate and fire neurons endowed with this plasticity scheme is studied by simulation on different tasks. For operant co nditioning with delayed reinforcement, learning succeeds even when the delay is so large that the delivered reward reflects the appropriateness, not of the immediately preceeding response, but of a decision made earlier on in the stimulus - decision sequence . So the proposed model does not rely on the temporal contiguity between decision and pertinent reward and thus provides a viable means of addressing the temporal credit assignment problem. In the same task, learning speeds up with increasing population si ze, showing that the plasticity cascade simultaneously addresses the spatial problem of assigning credit to the different population neurons. Simulations on other task such as sequential decision making serve to highlight the robustness of the proposed sch eme and, further, contrast its performance to that of temporal difference based approaches to reinforcement learning.

The moving targets algorithm for difficult temporal credit assignment problems

The 'moving targets' algorithm for training recurrent networks is reviewed and applied to a task which demonstrates the ability of this algorithm to use distant contextual information. Some practical difficulties are discussed, especially with regard to the minimization process. Results on performance and computational requirements of several different 2nd-order minimization algorithms are presented for moving target problems.

A Reinforcement Learning Agent for Distributed Task Allocation

Al giorno d'oggi il reinforcement learning ha dimostrato di essere davvero molto efficace nel machine learning in svariati campi, come ad esempio i giochi, il riconoscimento vocale e molti altri. Perciò, abbiamo deciso di applicare il reinforcement learning ai problemi di allocazione, in quanto sono un campo di ricerca non ancora studiato con questa tecnica e perchè questi problemi racchiudono nella loro formulazione un vasto insieme di sotto-problemi con simili caratteristiche, per cui una soluzione per uno di essi si estende ad ognuno di questi sotto-problemi. In questo progetto abbiamo realizzato un applicativo chiamato Service Broker, il quale, attraverso il reinforcement learning, apprende come distribuire l'esecuzione di tasks su dei lavoratori asincroni e distribuiti. L'analogia è quella di un cloud data center, il quale possiede delle risorse interne - possibilmente distribuite nella server farm -, riceve dei tasks dai suoi clienti e li esegue su queste risorse. L'obiettivo dell'applicativo, e quindi del data center, è quello di allocare questi tasks in maniera da minimizzare il costo di esecuzione. Inoltre, al fine di testare gli agenti del reinforcement learning sviluppati è stato creato un environment, un simulatore, che permettesse di concentrarsi nello sviluppo dei componenti necessari agli agenti, invece che doversi anche occupare di eventuali aspetti implementativi necessari in un vero data center, come ad esempio la comunicazione con i vari nodi e i tempi di latenza di quest'ultima. I risultati ottenuti hanno dunque confermato la teoria studiata, riuscendo a ottenere prestazioni migliori di alcuni dei metodi classici per il task allocation.

Synthesis of novel O-acylated-D-ribono-1,5-lactones and structural assignment supported by conventional NOESY-NMR and x-ray analysis

A practical method for the structural assignment of 3,4-O-benzylidene-D-ribono-1,5-lactones and analogues using conventional NMR techniques and NOESY measurements in solution is described. 2-O-Acyl-3,4-O-benzylidene-D-ribono-1,5-lactones were prepared in good yields by acylation of Zinner’s lactone with acyl chlorides under mildly basic conditions. Structural determination of 2-O-(4-nitrobenzoyl)-3,4-O-benzylidene-D-ribono-1,5-lactone was achieved by single crystal x-ray diffraction, which supports the results based on spectroscopic data.

Synthesis and total H-1- and C-13-NMR assignment of cephem derivatives for use in ADEPT approaches

We report the synthesis and total NMR characterization of 5-thia-1-azabicyclo[4.2.0]oct-2-ene-2-carboxylic acid-3-[[[(4''-nitrophenoxy)carbonyl]oxy]-methyl]-8-oxo-7[(2-thienyloxoacetyl)amino]-diphenylmethyl ester-5-dioxide (5), a new cephalosporin derivative. This compound can be used as the carrier of a wide range of drugs containing an amino group. The preparation of the intermediate product, 5-thia-1-azabicyclo[4.2.0]oct-2-ene-2-carboxylic acid-3-[methyl-4-(6-methoxyquinolin-8-ylamino) pentylcarbamate]-8-oxo-7-[(2-thienyloxoacetyl)amino]-diphenylmethyl ester-5-dioxide (6), as well as the synthesis of the antimalarial primaquine prodrug 5-thia-1-azabicyclo[4.2.0]oct-2-ene-2-carboxylic acid-3-[methyl-4-(6-methoxyquinolin-8-ylamino) pentylcarbamate]-8-oxo-7-[(2-thienyloxoacetyl)amino]-5-dioxide (7) are also described, together with their total H-1- and C-13-NMR assignments.

Var transcription profiling of Plasmodium falciparum 3D7: assignment of cytoadherent phenotypes to dominant transcripts

Background: Cytoadherence of Plasmodium falciparum-infected red blood cells is mediated by var gene-encoded P. falciparum erythrocyte membrane protein-1 and host receptor preference depends in most cases on which of the 50-60 var genes per genome is expressed. Enrichment of phenotypically homogenous parasites by panning on receptor expressing cells is fundamental for the identification of the corresponding var transcript. Methods: P. falciparum 3D7 parasites were panned on several transfected CHO-cell lines and their var transcripts analysed by i) reverse transcription/PCR/cloning/sequencing using a universal DBL alpha specific oligonucleotide pair and ii) by reverse transcription followed by quantitative PCR using 57 different oligonucleotide pairs. Results: Each cytoadherence selected parasite line also adhered to untransfected CHO-745 cells and upregulation of the var gene PFD995/PFD1000c was consistently associated with cytoadherence to all but one CHO cell line. In addition, parasites panned on different CHO cell lines revealed candidate var genes which reproducibly associated to the respective cytoadherent phenotype. The transcription profile obtained by RT-PCR/cloning/sequencing differed significantly from that of RT-quantitative PCR. Conclusion: Transfected CHO cell lines are of limited use for the creation of monophenotypic cytoadherent parasite lines. Nevertheless, 3D7 parasites can be reproducibly selected for the transcription of different determined var genes without genetic manipulation. Most importantly, var transcription analysis by RT-PCR/cloning/sequencing may lead to erroneous interpretation of var transcription profiles.

Dual-task interference: Attentional and neurophysiological influences

Performing two tasks simultaneously often degrades performance of one or both tasks. While this dual-task interference is classically interpreted in terms of shared attentional resources, where two motor tasks are performed simultaneously interactions within primary motor cortex (i.e., activity-dependent coupling) may also be a contributing factor. In the present study TMS (transcranial magnetic stimulation) was used to examine the contribution of activity-dependent coupling to dual-task interference during concurrent performance of a bimanual coordination task and a discrete probe reaction time (RT) task involving the foot. Experiments 1 and 2 revealed that activity-dependent coupling within the leg corticomotor pathway was greater during dual-task performance than single-task performance, and this was associated with interference on the probe RT task (i.e., increased RT). Experiment 3 revealed that dual-task interference occurred regardless of whether the dual-task involved two motor tasks or a motor and cognitive task, however activity-dependent coupling was present only when a dual motor task was performed. This suggests that activity-dependent coupling is less detrimental to performance than attentional processes operating upstream of the corticomotor system. Finally, while prioritising the RT task reduced, but did not eliminate, dual-task interference the contribution of activity-dependent coupling to dual-task interference was not affected by task prioritisation. This suggests that although activity-dependent coupling may contribute to dual motor-task interference, attentional processes appear to be more important. It also suggests that activity-dependent coupling may not be subject to modulation by attentional processes. (C) 2009 Elsevier B.V. All rights reserved.

Relative frequency of knowledge of results and task complexity in the motor skill acqusition

The aim of this study was to investigate the effects of knowledge of results (KR) frequency and task complexity on motor skill acquisition. The task consisted of throwing a bocha ball to place it as close as possible to the target ball. 120 students ages 11 to 73 years were assigned to one of eight experimental groups according to knowledge of results frequency (25, 50, 75, and 100%) and task complexity (simple and complex). Subjects performed 90 trials in the acquisition phase and 10 trials in the transfer test. The results showed that knowledge of results given at a frequency of 25% resulted in an inferior absolute error than 50% and inferior variable error than 50, 75, and 100 I frequencies, but no effect of task complexity was found.

Effect of bandwidth knowledge of results on the learning of a grip force control task

An experiment was conducted to investigate the persistence of the effect of ""bandwidth knowledge of results (KR)"" manipulated during the learning phase of performing a manual force-control task. The experiment consisted of two phases, an acquisition phase with the goal of maintaining 60% maximum force in 30 trials, and a second phase with the objective of maintaining 40% of maximum force in 20 further trials. There were four bandwidths of KR: when performance error exceeded 5, 10, or 15% of the target, and a control group (0% bandwidth). Analysis showed that 5, 10, and 15% bandwidth led to better performance than 0% bandwidth KR at the beginning of the second phase and persisted during the extended trials.

On the functional integration between postural and supra-postural tasks on the basis of contextual cues and task constraint

In order to evaluate the effects of uncertainty about direction of mechanical perturbation and supra-postural task constraint on postural control young adults had their upright stance perturbed while holding a tray in a horizontal position Stance was perturbed by moving forward or backward a supporting platform contrasting situations of certainty versus uncertainty of direction of displacement Increased constraint on postural stability was Imposed by a supra-postural task of equilibrating a cylinder on the tray Performance was assessed through EMG of anterior leg muscles angular displacement of the main Joints involved in the postural reactions and displacement of the tray Results showed that both certainty on the direction of perturbation and Increased supra-postural task constraint led to decreased angular displacement of the knee and the hip Furthermore combination of certainty and high supra-postural task constraint produced shorter latency of muscular activation Such postural responses were paralleled by decreased displacement of the tray Thesi results suggest a functional integration between the tasks with central set priming reactive postural responses from contextual cues and Increased stability demand (C) 2010 Elsevier B V All rights reserved

Motor perseveration during an ""A not B"" task in children with intellectual disabilities

This study was designed to identify perseverative reaching tendencies in children with intellectual disabilities (ID), over a period of 1 year, by using a version of the Piagetian ""A not B"" task modified by Smith, Thelen, Titzer, and McLin (1999). Nine children (4.8 years old at the beginning of the study) with intellectual disabilities (ID) (eight with mild ID; one with moderate ID) were assessed every 3 months for approximately 1 year, totaling four assessments. The results indicate that in a majority of the cases perseveration was resilient, and that the visual system decoupled from the reaching, especially towards the later assessment periods at the end of the year. Across assessment periods variability seemed to increase in each trial (A1 through B2) for reached target. These individuals, vulnerable to distraction and attention and to short-term memory deficits, are easily locked into rigid modes of motor habits. They are susceptible to perseveration while performing simple task contexts that are typically designed for 10- to 12-month-old, normally-developing infants, therefore creating strong confinements to stable, rigid modes of elementary forms of behavior. (C) 2009 Elsevier B.V. All rights reserved.

Hybrid Algorithms for Routing and Assignment Wavelengths in Optical Networks

This paper presents a strategy for the solution of the WDM optical networks planning. Specifically, the problem of Routing and Wavelength Allocation (RWA) in order to minimize the amount of wavelengths used. In this case, the problem is known as the Min-RWA. Two meta-heuristics (Tabu Search and Simulated Annealing) are applied to take solutions of good quality and high performance. The key point is the degradation of the maximum load on the virtual links in favor of minimization of number of wavelengths used; the objective is to find a good compromise between the metrics of virtual topology (load in Gb/s) and of the physical topology (quantity of wavelengths). The simulations suggest good results when compared to some existing in the literature.