Learning Two-Person Interaction Models for Responsive Synthetic Humanoids

Imitation learning is a promising approach for generating life-like behaviors of virtual humans and humanoid robots. So far, however, imitation learning has been mostly restricted to single agent settings where observed motions are adapted to new environment conditions but not to the dynamic behavior of interaction partners. In this paper, we introduce a new imitation learning approach that is based on the simultaneous motion capture of two human interaction partners. From the observed interactions, low-dimensional motion models are extracted and a mapping between these motion models is learned. This interaction model allows the real-time generation of agent behaviors that are responsive to the body movements of an interaction partner. The interaction model can be applied both to the animation of virtual characters as well as to the behavior generation for humanoid robots.

Learning by the Dendritic Prediction of Somatic Spiking

Recent modeling of spike-timing-dependent plasticity indicates that plasticity involves as a third factor a local dendritic potential, besides pre- and postsynaptic firing times. We present a simple compartmental neuron model together with a non-Hebbian, biologically plausible learning rule for dendritic synapses where plasticity is modulated by these three factors. In functional terms, the rule seeks to minimize discrepancies between somatic firings and a local dendritic potential. Such prediction errors can arise in our model from stochastic fluctuations as well as from synaptic input, which directly targets the soma. Depending on the nature of this direct input, our plasticity rule subserves supervised or unsupervised learning. When a reward signal modulates the learning rate, reinforcement learning results. Hence a single plasticity rule supports diverse learning paradigms.

High frequency repetitive transcranial magnetic stimulation (rTMS) of the left dorsolateral cortex: EEG topography during waking and subsequent sleep.

Repetitive transcranial magnetic stimulation (rTMS) is a novel research tool in neurology and psychiatry. It is currently being evaluated as a conceivable alternative to electroconvulsive therapy for the treatment of mood disorders. Eight healthy young (age range 21-25 years) right-handed men without sleep complaints participated in the study. Two sessions at a 1-week interval, each consisting of an adaptation night (sham stimulation) and an experimental night (rTMS in the left dorsolateral prefrontal cortex or sham stimulation; crossover design), were scheduled. In each subject, 40 trains of 2-s duration of rTMS (inter-train interval 28 s) were applied at a frequency of 20 Hz (i.e. 1600 pulses per session) and at an intensity of 90% of the motor threshold. Stimulations were scheduled 80 min before lights off. The waking EEG was recorded for 10-min intervals approximately 30 min prior to and after the 20-min stimulations, and polysomnographic recordings were obtained during the subsequent sleep episode (23.00-07.00 h). The power spectra of two referential derivations, as well as of bipolar derivations along the antero-posterior axis over the left and right hemispheres, were analyzed. rTMS induced a small reduction of sleep stage 1 (in min and percentage of total sleep time) over the whole night and a small enhancement of sleep stage 4 during the first non-REM sleep episode. Other sleep variables were not affected. rTMS of the left dorsolateral cortex did not alter the topography of EEG power spectra in waking following stimulation, in the all-night sleep EEG, or during the first non-REM sleep episode. Our results indicate that a single session of rTMS using parameters like those used in depression treatment protocols has no detectable side effects with respect to sleep in young healthy males.

Matching Recall and Storage in Sequence Learning with Spiking Neural Networks

Storing and recalling spiking sequences is a general problem the brain needs to solve. It is, however, unclear what type of biologically plausible learning rule is suited to learn a wide class of spatiotemporal activity patterns in a robust way. Here we consider a recurrent network of stochastic spiking neurons composed of both visible and hidden neurons. We derive a generic learning rule that is matched to the neural dynamics by minimizing an upper bound on the Kullback–Leibler divergence from the target distribution to the model distribution. The derived learning rule is consistent with spike-timing dependent plasticity in that a presynaptic spike preceding a postsynaptic spike elicits potentiation while otherwise depression emerges. Furthermore, the learning rule for synapses that target visible neurons can be matched to the recently proposed voltage-triplet rule. The learning rule for synapses that target hidden neurons is modulated by a global factor, which shares properties with astrocytes and gives rise to testable predictions.

Implicit task sequence learning

Implicit task sequence learning (TSL) can be considered as an extension of implicit sequence learning which is typically tested with the classical serial reaction time task (SRTT). By design, in the SRTT there is a correlation between the sequence of stimuli to which participants must attend and the sequence of motor movements/key presses with which participants must respond. The TSL paradigm allows to disentangle this correlation and to separately manipulate the presences/absence of a sequence of tasks, a sequence of responses, and even other streams of information such as stimulus locations or stimulus-response mappings. Here I review the state of TSL research which seems to point at the critical role of the presence of correlated streams of information in implicit sequence learning. On a more general level, I propose that beyond correlated streams of information, a simple statistical learning mechanism may also be involved in implicit sequence learning, and that the relative contribution of these two explanations differ according to task requirements. With this differentiation, conflicting results can be integrated into a coherent framework.

Waking Up to Sleep Medicine Education

Background: Sleep disorders are an important cause of morbidity among our population with billions of dollars spent on direct and indirect costs attributed to sleep disorders. In spite of raising prevalence and morbidity, surveys have shown inadequate education in sleep medicine at all levels at medical school. According to national sleep disorders research plan data, in 1990 about 37 % of medical schools did not offer any sleep education and of the schools which offered it, the average time devoted to sleep medicine was about 2 hours. Sleep disorders have found to be uniformly under diagnosed in primary care settings. [See PDF for complete abstract]

SIGNALING MECHANISMS THAT CONTROL GAP JUNCTIONAL COUPLING BETWEEN RETINAL NEURONS

Gap junctions between neurons form the structural substrate for electrical synapses. Connexin 36 (Cx36, and its non-mammalian ortholog connexin 35) is the major neuronal gap junction protein in the central nervous system (CNS), and contributes to several important neuronal functions including neuronal synchronization, signal averaging, network oscillations, and motor learning. Connexin 36 is strongly expressed in the retina, where it is an obligatory component of the high-sensitivity rod photoreceptor pathway. A fundamental requirement of the retina is to adapt to broadly varying inputs in order to maintain a dynamic range of signaling output. Modulation of the strength of electrical coupling between networks of retinal neurons, including the Cx36-coupled AII amacrine cell in the primary rod circuit, is a hallmark of retinal luminance adaptation. However, very little is known about the mechanisms regulating dynamic modulation of Cx36-mediated coupling. The primary goal of this work was to understand how cellular signaling mechanisms regulate coupling through Cx36 gap junctions. We began by developing and characterizing phospho-specific antibodies against key regulatory phosphorylation sites on Cx36. Using these tools we showed that phosphorylation of Cx35 in fish models varies with light adaptation state, and is modulated by acute changes in background illumination. We next turned our focus to the well-studied and readily identifiable AII amacrine cell in mammalian retina. Using this model we showed that increased phosphorylation of Cx36 is directly related to increased coupling through these gap junctions, and that the dopamine-stimulated uncoupling of the AII network is mediated by dephosphorylation of Cx36 via protein kinase A-stimulated protein phosphatase 2A activity. We then showed that increased phosphorylation of Cx36 on the AII amacrine network is driven by depolarization of presynaptic ON-type bipolar cells as well as background light increments. This increase in phosphorylation is mediated by activation of extrasynaptic NMDA receptors associated with Cx36 gap junctions on AII amacrine cells and by Ca2+-calmodulin-dependent protein kinase II activation. Finally, these studies indicated that coupling is regulated locally at individual gap junction plaques. This work provides a framework for future study of regulation of Cx36-mediated coupling, in which increased phosphorylation of Cx36 indicates increased neuronal coupling.

Tracking the Flow of Information through the Hippocampal Formation in the Rat

The hippocampus receives input from upper levels of the association cortex and is implicated in many mnemonic processes, but the exact mechanisms by which it codes and stores information is an unresolved topic. This work examines the flow of information through the hippocampal formation while attempting to determine the computations that each of the hippocampal subfields performs in learning and memory. The formation, storage, and recall of hippocampal-dependent memories theoretically utilize an autoassociative attractor network that functions by implementing two competitive, yet complementary, processes. Pattern separation, hypothesized to occur in the dentate gyrus (DG), refers to the ability to decrease the similarity among incoming information by producing output patterns that overlap less than the inputs. In contrast, pattern completion, hypothesized to occur in the CA3 region, refers to the ability to reproduce a previously stored output pattern from a partial or degraded input pattern. Prior to addressing the functional role of the DG and CA3 subfields, the spatial firing properties of neurons in the dentate gyrus were examined. The principal cell of the dentate gyrus, the granule cell, has spatially selective place fields; however, the behavioral correlates of another excitatory cell, the mossy cell of the dentate polymorphic layer, are unknown. This report shows that putative mossy cells have spatially selective firing that consists of multiple fields similar to previously reported properties of granule cells. Other cells recorded from the DG had single place fields. Compared to cells with multiple fields, cells with single fields fired at a lower rate during sleep, were less likely to burst, and were more likely to be recorded simultaneously with a large population of neurons that were active during sleep and silent during behavior. These data suggest that single-field and multiple-field cells constitute at least two distinct cell classes in the DG. Based on these characteristics, we propose that putative mossy cells tend to fire in multiple, distinct locations in an environment, whereas putative granule cells tend to fire in single locations, similar to place fields of the CA1 and CA3 regions. Experimental evidence supporting the theories of pattern separation and pattern completion comes from both behavioral and electrophysiological tests. These studies specifically focused on the function of each subregion and made implicit assumptions about how environmental manipulations changed the representations encoded by the hippocampal inputs. However, the cell populations that provided these inputs were in most cases not directly examined. We conducted a series of studies to investigate the neural activity in the entorhinal cortex, dentate gyrus, and CA3 in the same experimental conditions, which allowed a direct comparison between the input and output representations. The results show that the dentate gyrus representation changes between the familiar and cue altered environments more than its input representations, whereas the CA3 representation changes less than its input representations. These findings are consistent with longstanding computational models proposing that (1) CA3 is an associative memory system performing pattern completion in order to recall previous memories from partial inputs, and (2) the dentate gyrus performs pattern separation to help store different memories in ways that reduce interference when the memories are subsequently recalled.

Long-lasting synaptic potentiation induced by depolarization under conditions that eliminate detectable Ca2+ signals.

Activity-dependent alterations of synaptic transmission important for learning and memory are often induced by Ca(2+) signals generated by depolarization. While it is widely assumed that Ca(2+) is the essential transducer of depolarization into cellular plasticity, little effort has been made to test whether Ca(2+)-independent responses to depolarization might also induce memory-like alterations. It was recently discovered that peripheral axons of nociceptive sensory neurons in Aplysia display long-lasting hyperexcitability triggered by conditioning depolarization in the absence of Ca(2+) entry (using nominally Ca(2+)-free solutions containing EGTA, "0Ca/EGTA") or the absence of detectable Ca(2+) transients (adding BAPTA-AM, "0Ca/EGTA/BAPTA-AM"). The current study reports that depolarization of central ganglia to approximately 0 mV for 2 min in these same solutions induced hyperexcitability lasting >1 h in sensory neuron processes near their synapses onto motor neurons. Furthermore, conditioning depolarization in these solutions produced a 2.5-fold increase in excitatory postsynaptic potential (EPSP) amplitude 1-3 h afterward despite a drop in motor neuron input resistance. Depolarization in 0 Ca/EGTA produced long-term potentiation (LTP) of the EPSP lasting > or = 1 days without changing postsynaptic input resistance. When re-exposed to extracellular Ca(2+) during synaptic tests, prior exposure to 0Ca/EGTA or to 0Ca/EGTA/BAPTA-AM decreased sensory neuron survival. However, differential effects on neuronal health are unlikely to explain the observed potentiation because conditioning depolarization in these solutions did not alter survival rates. These findings suggest that unrecognized Ca(2+)-independent signals can transduce depolarization into long-lasting synaptic potentiation, perhaps contributing to persistent synaptic alterations following large, sustained depolarizations that occur during learning, neural injury, or seizures.

Social skills as predictors of communication, performance and quality of collaboration in project-based learning

The assumption that social skills are necessary ingredients of collaborative learning is well established but rarely empirically tested. In addition, most theories on collaborative learning focus on social skills only at the personal level, while the social skill configurations within a learning group might be of equal importance. Using the integrative framework, this study investigates which social skills at the personal level and at the group level are predictive of task-related e-mail communication, satisfaction with performance and perceived quality of collaboration. Data collection took place in a technology-enhanced long-term project-based learning setting for pre-service teachers. For data collection, two questionnaires were used, one at the beginning and one at the end of the learning cycle which lasted 3 months. During the project phase, the e-mail communication between group members was captured as well. The investigation of 60 project groups (N = 155 for the questionnaires; group size: two or three students) and 33 groups for the e-mail communication (N = 83) revealed that personal social skills played only a minor role compared to group level configurations of social skills in predicting satisfaction with performance, perceived quality of collaboration and communication behaviour. Members from groups that showed a high and/or homogeneous configuration of specific social skills (e.g., cooperation/compromising, leadership) usually were more satisfied and saw their group as more efficient than members from groups with a low and/or heterogeneous configuration of skills.

More than synaptic plasticity: role of nonsynaptic plasticity in learning and memory.

Decades of research on the cellular mechanisms of memory have led to the widely held view that memories are stored as modifications of synaptic strength. These changes involve presynaptic processes, such as direct modulation of the release machinery, or postsynaptic processes, such as modulation of receptor properties. Parallel studies have revealed that memories might also be stored by nonsynaptic processes, such as modulation of voltage-dependent membrane conductances, which are expressed as changes in neuronal excitability. Although in some cases nonsynaptic changes can function as part of the engram itself, they might also serve as mechanisms through which a neural circuit is set to a permissive state to facilitate synaptic modifications that are necessary for memory storage.

Ca++/calmodulin-dependent protein kinase II: Messenger RNA in developing rat brain

Ca$\sp{++}$/calmodulin-dependent protein kinase II (CaM-KII) is highly concentrated in mammalian brain, comprising as much as 2% of the total protein in some regions. In forebrain, CaM-KII has been shown to be enriched in postsynaptic structures where it has been implicated in maintaining cytoskeletal structure, and more recently in signal transduction mechanisms and processes underlying learning and memory. CaM-KII appears to exist as a holoenzyme composed of two related yet distinct subunits, alpha and beta. The ratio of the subunits in the holoenzyme varies with different brain regions and to some degree with subcellular fractions. The two subunits also display distinct developmental profiles. Levels of alpha subunit are not evident at birth but increase dramatically during postnatal development, while levels of beta subunit are readily detected at birth and only gradual increase postnatally. The distinct regional, subcellular and developmental distribution of the two subunits of CaM-KII have prompted us to examine factors involved in regulating the synthesis of the subunit proteins.^ This dissertation addresses the regional and developmental expression of the mRNAs for the individual subunits using in situ hybridization histochemistry and northern slot-blot analysis. By comparing the developmental profile of each mRNA with that of its respective protein, we have determined that initiation of gene transcription is likely the primary site for regulating CaM-KII protein levels. Furthermore, the distinct cytoarchitecture of the hippocampus has allowed us to demonstrate that the alpha, but not beta subunit mRNA is localized in dendrites of certain forebrain neurons. The localization of alpha subunit mRNA at postsynaptic structures, in concert with the accumulation of subunit protein, suggests that dendritic synthesis of CaM-KII alpha subunit may be important for maintaining postsynaptic structure and/or function. ^

Neural analogs of associative learning in {\it Aplysia\/}: Experimental and theoretical approaches

A change in synaptic strength arising from the activation of two neuronal pathways at approximately the same time is a form of associative plasticity and may underlie classical conditioning. Previously, a cellular analog of a classical conditioning protocol has been demonstrated to produce short-term associative plasticity at the connections between sensory and motor neurons in Aplysia. A similar training protocol produced long-term (24 hour) enhancement of excitatory postsynaptic potentials (EPSPs). EPSPs produced by sensory neurons in which activity was paired with a reinforcing stimulus were significantly larger than unpaired controls 24 hours after training. To examined whether the associative plasticity observed at these synapses may be involved in higher-order forms of classical conditioning, a neural analog of contingency was developed. In addition, computer simulations were used to analyze whether the associative plasticity observed in Aplysia could, in theory, account for second-order conditioning and blocking. ^

Modulatory effects of bag cell peptides in {\it Aplysia}: Behavioral and electrophysiological studies

Reproductive hormones have effects on the nervous system not directly related to reproductive function. In the rat, for example, luteinizing hormone releasing hormone has dramatic effects on learning and memory. The present work attempts to examine the effects of reproductive hormones on non-reproductive behaviors and the neural loci and mechanisms underlying these effects in Aplysia, an animal whose behaviors, reproductive hormones and neural circuitry have been well characterized.^ In Aplysia, the neurosecretory bag cells release several peptides that are responsible for eliciting egg laying. The effects of these peptides on the defensive tail-siphon withdrawal reflex, as well as sensitization of this reflex, were examined. Sensitization, a simple form of nonassociative learning, refers to the behavioral enhancement of a response to a test stimulus after the presentation of a strong stimulus, that may last minutes (short-term) or days (long-term). An extract of the bag cells (BCE) inhibited the baseline siphon component of the tail-siphon withdrawal reflex and suppressed long-term, but not short-term, sensitization of the reflex. Preliminary experiments suggest that BCE also affects the tail component of the tail-siphon withdrawal reflex.^ To determine the neural mechanisms underlying the inhibition of the baseline reflex, electrophysiological studies were performed using an in vitro analogue of the tail-siphon withdrawal reflex to examine the ability of BCE, as well as the individual bag cell peptides (BCPs), to modulate the circuitry of the reflex. Bag cell extract attenuated the synaptic strength of the monosynaptic connections between tail sensory neurons and tail motor neurons. When individually applied only $\beta$-BCP produced a similar attenuation. This effect of $\beta$-BCP was not dependent on changes in duration of the presynaptic action potential.^ An in vitro analogue of long-term sensitization training was developed to examine the mechanisms by which the BCPs may affect long-term sensitization of the tail-siphon withdrawal reflex. This analogue exhibited both short- and long-term facilitation of the connections between the tail sensory and motor neurons.^ The results of these behavioral and electrophysiological experiments suggest that the BCPs inhibit the tail-siphon withdrawal reflex, at least in part, by modulating the synaptic strength of the connections between the sensory neurons and motor neurons underlying the reflex. One candidate for this effect is $\beta$-BCP. Thus, the peptides which elicit egg laying may also serve other functions such as the inhibition of defensive reflexes. In addition, these experiments raise the possibility that BCPs may exert a long lasting effect ($>$24 hr), suppressing long-term sensitization of the tail-siphon withdrawal reflex. ^

Convergent and divergent modulatory pathways in {\it Aplysia\/}: Cellular and network studies

Neuromodulation is essential to many functions of the nervous system. In the simple gastropod mollusk Aplysia californica, neuromodulation of the circuits for the defensive withdrawal reflexes has been associated with several forms of learning. In the present work, the neurotransmitters and neural circuitry which contribute to the modulation of the tail-siphon withdrawal reflex were examined.^ A recently-identified neuropeptide transmitter, buccalin A was found to modulate the biophysical properties of the sensory neurons that mediate the reflex. The actions of buccalin A on the sensory neurons were compared with those of the well-characterized modulatory transmitter serotonin, and convergence and divergence in the actions of these two transmitters were evaluated. Buccalin A dramatically increased the excitability of sensory neurons and occluded further enhancement of excitability by serotonin. Buccalin A produced no significant change in spike duration, and it did not block serotonin-induced spike broadening. Voltage-clamp analysis revealed the currents that may be involved in the effects on spike duration and excitability. Buccalin A decreased an outward current similar to the S-K$\sp+$ current (I$\sb{\rm K,S}$). Buccalin A appeared to occlude further modulation of I$\sb{\rm K,S}$ by serotonin, but did not block serotonin-induced modulation of the voltage-dependent delayed rectifier K$\sp+$ current (I$\sb{\rm K,V}$). These results suggest that buccalin A converges on some, but not all, of the same subcellular modulatory pathways as serotonin.^ In order to begin to understand neuromodulation in a more physiological context for the tail-siphon withdrawal reflex, the modulatory circuitry for the tail-withdrawal circuit was examined. Mechanoafferent neurons in the J cluster of the cerebral ganglion were identified as elements of a modulatory circuit for the reflex. Excitatory and inhibitory connections were observed between the J cells and the pleural sensory neurons, the tail motor neurons, and several classes of interneurons for the tail-siphon withdrawal circuit. The J cells produced both fast and slow PSPs in these neurons. Of particular interest was the ability of the J cells to produce slow EPSPs in the pleural sensory neurons. These slow EPSPs were associated with an increase in the excitability of the sensory neurons. The J cells appear to mediate both sensory and modulatory inputs to the circuit for the tail-siphon withdrawal reflex from the anterior part of the animal. ^