VHDL-based system design of a cognitive sensorimotor loop (CSL) for haptic Human-Machine Interaction (HMI)

This document is a summary of the Bachelor thesis titled “VHDL-Based System Design of a Cognitive Sensorimotor Loop (CSL) for Haptic Human-Machine Interaction (HMI)” written by Pablo de Miguel Morales, Electronics Engineering student at the Universidad Politécnica de Madrid (UPM Madrid, Spain) during an Erasmus+ Exchange Program at the Beuth Hochschule für Technik (BHT Berlin, Germany). The tutor of this project is Dr. Prof. Hild. This project has been developed inside the Neurobotics Research Laboratory (NRL) in close collaboration with Benjamin Panreck, a member of the NRL, and another exchange student from the UPM Pablo Gabriel Lezcano. For a deeper comprehension of the content of the thesis, a deeper look in the document is needed as well as the viewing of the videos and the VHDL design. In the growing field of automation, a large amount of workforce is dedicated to improve, adapt and design motor controllers for a wide variety of applications. In the specific field of robotics or other machinery designed to interact with humans or their environment, new needs and technological solutions are often being discovered due to the existing, relatively unexplored new scenario it is. The project consisted of three main parts: Two VHDL-based systems and one short experiment on the haptic perception. Both VHDL systems are based on a Cognitive Sensorimotor Loop (CSL) which is a control loop designed by the NRL and mainly developed by Dr. Prof. Hild. The CSL is a control loop whose main characteristic is the fact that it does not use any external sensor to measure the speed or position of the motor but the motor itself. The motor always generates a voltage that is proportional to its angular speed so it does not need calibration. This method is energy efficient and simplifies control loops in complex systems. The first system, named CSL Stay In Touch (SIT), consists in a one DC motor system controller by a FPGA Board (Zynq ZYBO 7000) whose aim is to keep contact with any external object that touches its Sensing Platform in both directions. Apart from the main behavior, three features (Search Mode, Inertia Mode and Return Mode) have been designed to enhance the haptic interaction experience. Additionally, a VGA-Screen is also controlled by the FPGA Board for the monitoring of the whole system. This system has been completely developed, tested and improved; analyzing its timing and consumption properties. The second system, named CSL Fingerlike Mechanism (FM), consists in a fingerlike mechanical system controlled by two DC motors (Each controlling one part of the finger). The behavior is similar to the first system but in a more complex structure. This system was optional and not part of the original objectives of the thesis and it could not be properly finished and tested due to the lack of time. The haptic perception experiment was an experiment conducted to have an insight into the complexity of human haptic perception in order to implement this knowledge into technological applications. The experiment consisted in testing the capability of the subjects to recognize different objects and shapes while being blindfolded and with their ears covered. Two groups were done, one had full haptic perception while the other had to explore the environment with a plastic piece attached to their finger to create a haptic handicap. The conclusion of the thesis was that a haptic system based only on a CSL-based system is not enough to retrieve valuable information from the environment and that other sensors are needed (temperature, pressure, etc.) but that a CSL-based system is very useful to control the force applied by the system to interact with haptic sensible surfaces such as skin or tactile screens. RESUMEN. Este documento es un resumen del proyecto fin de grado titulado “VHDL-Based System Design of a Cognitive Sensorimotor Loop (CSL) for Haptic Human-Machine Interaction (HMI)” escrito por Pablo de Miguel, estudiante de Ingeniería Electrónica de Comunicaciones en la Universidad Politécnica de Madrid (UPM Madrid, España) durante un programa de intercambio Erasmus+ en la Beuth Hochschule für Technik (BHT Berlin, Alemania). El tutor de este proyecto ha sido Dr. Prof. Hild. Este proyecto se ha desarrollado dentro del Neurorobotics Research Laboratory (NRL) en estrecha colaboración con Benjamin Panreck (un miembro del NRL) y con Pablo Lezcano (Otro estudiante de intercambio de la UPM). Para una comprensión completa del trabajo es necesaria una lectura detenida de todo el documento y el visionado de los videos y análisis del diseño VHDL incluidos en el CD adjunto. En el creciente sector de la automatización, una gran cantidad de esfuerzo está dedicada a mejorar, adaptar y diseñar controladores de motor para un gran rango de aplicaciones. En el campo específico de la robótica u otra maquinaria diseñada para interactuar con los humanos o con su entorno, nuevas necesidades y soluciones tecnológicas se siguen desarrollado debido al relativamente inexplorado y nuevo escenario que supone. El proyecto consta de tres partes principales: Dos sistemas basados en VHDL y un pequeño experimento sobre la percepción háptica. Ambos sistemas VHDL están basados en el Cognitive Sesnorimotor Loop (CSL) que es un lazo de control creado por el NRL y cuyo desarrollador principal ha sido Dr. Prof. Hild. El CSL es un lazo de control cuya principal característica es la ausencia de sensores externos para medir la velocidad o la posición del motor, usando el propio motor como sensor. El motor siempre genera un voltaje proporcional a su velocidad angular de modo que no es necesaria calibración. Este método es eficiente en términos energéticos y simplifica los lazos de control en sistemas complejos. El primer sistema, llamado CSL Stay In Touch (SIT), consiste en un sistema formado por un motor DC controlado por una FPGA Board (Zynq ZYBO 7000) cuyo objetivo es mantener contacto con cualquier objeto externo que toque su plataforma sensible en ambas direcciones. Aparte del funcionamiento básico, tres modos (Search Mode, Inertia Mode y Return Mode) han sido diseñados para mejorar la interacción. Adicionalmente, se ha diseñado el control a través de la FPGA Board de una pantalla VGA para la monitorización de todo el sistema. El sistema ha sido totalmente desarrollado, testeado y mejorado; analizando su propiedades de timing y consumo energético. El segundo sistema, llamado CSL Fingerlike Mechanism (FM), consiste en un mecanismo similar a un dedo controlado por dos motores DC (Cada uno controlando una falange). Su comportamiento es similar al del primer sistema pero con una estructura más compleja. Este sistema no formaba parte de los objetivos iniciales del proyecto y por lo tanto era opcional. No pudo ser plenamente desarrollado debido a la falta de tiempo. El experimento de percepción háptica fue diseñado para profundizar en la percepción háptica humana con el objetivo de aplicar este conocimiento en aplicaciones tecnológicas. El experimento consistía en testear la capacidad de los sujetos para reconocer diferentes objetos, formas y texturas en condiciones de privación del sentido del oído y la vista. Se crearon dos grupos, en uno los sujetos tenían plena percepción háptica mientras que en el otro debían interactuar con los objetos a través de una pieza de plástico para generar un hándicap háptico. La conclusión del proyecto fue que un sistema háptico basado solo en sistemas CSL no es suficiente para recopilar información valiosa del entorno y que debe hacer uso de otros sensores (temperatura, presión, etc.). En cambio, un sistema basado en CSL es idóneo para el control de la fuerza aplicada por el sistema durante la interacción con superficies hápticas sensibles tales como la piel o pantallas táctiles.

Development of a sensorimotor algorithm able to deal with unforeseen pushes and its implementation based on VHDL

Development of a Sensorimotor Algorithm Able to Deal with Unforeseen Pushes and Its Implementation Based on VHDL is the title of my thesis which concludes my Bachelor Degree in the Escuela Técnica Superior de Ingeniería y Sistemas de Telecomunicación of the Universidad Politécnica de Madrid. It encloses the overall work I did in the Neurorobotics Research Laboratory from the Beuth Hochschule für Technik Berlin during my ERASMUS year in 2015. This thesis is focused on the field of robotics, specifically an electronic circuit called Cognitive Sensorimotor Loop (CSL) and its control algorithm based on VHDL hardware description language. The reason that makes the CSL special resides in its ability to operate a motor both as a sensor and an actuator. This way, it is possible to achieve a balanced position in any of the robot joints (e.g. the robot manages to stand) without needing any conventional sensor. In other words, the back electromotive force (EMF) induced by the motor coils is measured and the control algorithm responds depending on its magnitude. The CSL circuit contains mainly an analog-to-digital converter (ADC) and a driver. The ADC consists on a delta-sigma modulation which generates a series of bits with a certain percentage of 1's and 0's, proportional to the back EMF. The control algorithm, running in a FPGA, processes the bit frame and outputs a signal for the driver. This driver, which has an H bridge topology, gives the motor the ability to rotate in both directions while it's supplied with the power needed. The objective of this thesis is to document the experiments and overall work done on push ignoring contractive sensorimotor algorithms, meaning sensorimotor algorithms that ignore large magnitude forces (compared to gravity) applied in a short time interval on a pendulum system. This main objective is divided in two sub-objectives: (1) developing a system based on parameterized thresholds and (2) developing a system based on a push bypassing filter. System (1) contains a module that outputs a signal which blocks the main Sensorimotor algorithm when a push is detected. This module has several different parameters as inputs e.g. the back EMF increment to consider a force as a push or the time interval between samples. System (2) consists on a low-pass Infinite Impulse Response digital filter. It cuts any frequency considered faster than a certain push oscillation. This filter required an intensive study on how to implement some functions and data types (fixed or floating point data) not supported by standard VHDL packages. Once this was achieved, the next challenge was to simplify the solution as much as possible, without using non-official user made packages. Both systems behaved with a series of interesting advantages and disadvantages for the elaboration of the document. Stability, reaction time, simplicity or computational load are one of the many factors to be studied in the designed systems. RESUMEN. Development of a Sensorimotor Algorithm Able to Deal with Unforeseen Pushes and Its Implementation Based on VHDL es un Proyecto de Fin de Grado (PFG) que concluye mis estudios en la Escuela Técnica Superior de Ingeniería y Sistemas de Telecomunicación de la Universidad Politécnica de Madrid. En él se documenta el trabajo de investigación que realicé en el Neurorobotics Research Laboratory de la Beuth Hochschule für Technik Berlin durante el año 2015 mediante el programa de intercambio ERASMUS. Este PFG se centra en el campo de la robótica y en concreto en un circuito electrónico llamado Cognitive Sensorimotor Loop (CSL) y su algoritmo de control basado en lenguaje de modelado hardware VHDL. La particularidad del CSL reside en que se consigue que un motor haga las veces tanto de sensor como de actuador. De esta manera es posible que las articulaciones de un robot alcancen una posición de equilibrio (p.ej. el robot se coloca erguido) sin la necesidad de sensores en el sentido estricto de la palabra. Es decir, se mide la propia fuerza electromotriz (FEM) inducida sobre el motor y el algoritmo responde de acuerdo a su magnitud. El circuito CSL se compone de un convertidor analógico-digital (ADC) y un driver. El ADC consiste en un modulador sigma-delta, que genera una serie de bits con un porcentaje de 1's y 0's determinado, en proporción a la magnitud de la FEM inducida. El algoritmo de control, que se ejecuta en una FPGA, procesa esta cadena de bits y genera una señal para el driver. El driver, que posee una topología en puente H, provee al motor de la potencia necesaria y le otorga la capacidad de rotar en cualquiera de las dos direcciones. El objetivo de este PFG es documentar los experimentos y en general el trabajo realizado en algoritmos Sensorimotor que puedan ignorar fuerzas de gran magnitud (en comparación con la gravedad) y aplicadas en una corta ventana de tiempo. En otras palabras, ignorar empujones conservando el comportamiento original frente a la gravedad. Para ello se han desarrollado dos sistemas: uno basado en umbrales parametrizados (1) y otro basado en un filtro de corte ajustable (2). El sistema (1) contiene un módulo que, en el caso de detectar un empujón, genera una señal que bloquea el algoritmo Sensorimotor. Este módulo recibe diferentes parámetros como el incremento necesario de la FEM para que se considere un empujón o la ventana de tiempo para que se considere la existencia de un empujón. El sistema (2) consiste en un filtro digital paso-bajo de respuesta infinita que corta cualquier variación que considere un empujón. Para crear este filtro se requirió un estudio sobre como implementar ciertas funciones y tipos de datos (coma fija o flotante) no soportados por las librerías básicas de VHDL. Tras esto, el objetivo fue simplificar al máximo la solución del problema, sin utilizar paquetes de librerías añadidos. En ambos sistemas aparecen una serie de ventajas e inconvenientes de interés para el documento. La estabilidad, el tiempo de reacción, la simplicidad o la carga computacional son algunas de las muchos factores a estudiar en los sistemas diseñados. Para concluir, también han sido documentadas algunas incorporaciones a los sistemas: una interfaz visual en VGA, un módulo que compensa el offset del ADC o la implementación de una batería de faders MIDI entre otras.

Role of intrinsic synaptic circuitry in collicular sensorimotor integration

The superficial gray layer of the superior colliculus contains a map that represents the visual field, whereas the underlying intermediate gray layer contains a vector map of the saccades that shift the direction of gaze. These two maps are aligned so that a particular region of the visual field is represented directly above the neurons that orient the highest acuity area of the retina toward that region. Although it has been proposed that the transmission of information from the visuosensory to the motor map plays an important role in the generation of visually guided saccades, experiments have failed to demonstrate any functional linkage between the two layers. We examined synaptic transmission between these layers in vitro by stimulating the superficial layer while using whole-cell patch-clamp methods to measure the responses of intermediate layer neurons. Stimulation of superficial layer neurons evoked excitatory postsynaptic currents in premotor cells. This synaptic input was columnar in organization, indicating that the connections between the layers link corresponding regions of the visuosensory and motor maps. Excitatory postsynaptic currents were large enough to evoke action potentials and often occurred in clusters similar in duration to the bursts of action potentials that premotor cells use to command saccades. Our results indicate the presence of functional connections between the superficial and intermediate layers and show that such connections could play a significant role in the generation of visually guided saccades.

Preferential localization of self-stimulation sites in striosomes/patches in the rat striatum

Histological sections of the mammalian striatum reveal a “matrix” that is histochemically distinguishable from patches, or “striosomes”. The latter are cross sections of a compartment that consists primarily of tube-shaped structures radiating through the matrix. As a test of the hypothesis that the function of the striosome/patch compartment includes the mediation of behaviors related to reward, the present study examined electrical self-stimulation of the caudoputamen in rats with electrodes in either of the two compartments. Rats acquired and maintained bar-pressing responses that were contingent on stimulation through electrodes making contact with striosomes/patches more reliably than animals with electrodes terminating exclusively in the matrix. The results provide in vivo evidence that the striosome/patch compartment is functionally differentiated from the matrix compartment: Stimulation centered in or around the striosome/patch compartment but not in the matrix led to rapid acquisition of a new behavior.

Song selectivity and sensorimotor signals in vocal learning and production

Bird song, like human speech, is a learned vocal behavior that requires auditory feedback. Both as juveniles, while they learn to sing, and as adults, songbirds use auditory feedback to compare their own vocalizations with an internal model of a target song. Here we describe experiments that explore a role for the songbird anterior forebrain pathway (AFP), a basal ganglia-forebrain circuit, in evaluating song feedback and modifying vocal output. First, neural recordings in anesthetized, juvenile birds show that single AFP neurons are specialized to process the song stimuli that are compared during sensorimotor learning. AFP neurons are tuned to both the bird's own song and the tutor song, even when these stimuli are manipulated to be very different from each other. Second, behavioral experiments in adult birds demonstrate that lesions to the AFP block the deterioration of song that normally follows deafening. This observation suggests that deafening results in an instructive signal, indicating a mismatch between feedback and the internal song model, and that the AFP is involved in generating or transmitting this instructive signal. Finally, neural recordings from behaving birds reveal robust singing-related activity in the AFP. This activity is likely to originate from premotor areas and could be modulated by auditory feedback of the bird's own voice. One possibility is that this activity represents an efference copy, predicting the sensory consequences of motor commands. Overall, these studies illustrate that sensory and motor processes are highly interrelated in this circuit devoted to vocal learning, as is true for brain areas involved in speech.

Enhancement of sensorimotor connections by conditioning-related stimulation in Aplysia depends upon postsynaptic Ca2+.

Classical conditioning of Aplysia's siphon-withdrawal reflex is thought to be due to a presynaptic mechanism-activity-dependent presynaptic facilitation of sensorimotor connections. Recent experiments with sensorimotor synapses in dissociated cell culture, however, provide an alternative cellular mechanism for classical conditioning-Hebbian long-term potentiation (LTP) of sensorimotor connections. Induction of Hebbian LTP of these connections is mediated by activation of N-methyl-D-aspartate-related receptors and requires the postsynaptic elevation of intracellular Ca2+. To determine whether the enhancement of sensorimotor synapses during classical conditioning in Aplysia-like LTP of sensorimotor synapses in culture-also depends upon the elevation of postsynaptic Ca2+, we carried out experiments involving the cellular analog of classical conditioning of siphon withdrawal. We examined changes in the strength of monosynaptic siphon sensorimotor connections in the abdominal ganglion of Aplysia following paired presentations of sensory neuron activation and tail nerve shock. This training regimen resulted in significant enhancement of the monosynaptic sensorimotor excitatory postsynaptic potential, as compared with the sensorimotor excitatory postsynaptic potential in preparations that received only test stimulation. Infusing the motor neuron with 1,2-bis(2-aminophenoxy)ethane-N,N-N',N'-tetraacetic acid, a specific chelator of intracellular Ca2+, prior to paired stimulation training blocked this synaptic enhancement. Our results implicate a postsynaptic, possibly Hebbian, mechanism in classical conditioning in Aplysia.

Rapid endocytosis of a G protein-coupled receptor: substance P evoked internalization of its receptor in the rat striatum in vivo.

Studies on cultured cells have shown that agonists induce several types of G protein-coupled receptors to undergo internalization. We have investigated this phenomenon in rat striatum, using substance P (SP)-induced internalization of the SP receptor (SPR) as our model system. Within 1 min of a unilateral striatal injection of SP in the anesthetized rat, nearly 60% of the SPR-immunoreactive neurons within the injection zone display massive internalization of the SPR--i.e., 20-200 SPR+ endosomes per cell body. Within the dendrites the SPR undergoes a striking translocation from the plasma membrane to endosomes, and these dendrites also undergo a morphological reorganization, changing from a structure of rather uniform diameter to one characterized by large, swollen varicosities connected by thin fibers. In both cell bodies and dendrites the number of SPR+ endosomes returns to baseline within 60 min of SP injection. The number of neurons displaying substantial endosomal SPR internalization is dependent on the concentration of injected SP, and the SP-induced SPR internalization is inhibited by the nonpeptide neurokinin 1 receptor antagonist RP-67,580. These data demonstrate that in the central nervous system in vivo, SP induces a rapid and widespread SPR internalization in the cell bodies and dendrites and a structural reorganization of the dendrites. These results suggest that many of the observations that have been made on the internalization and recycling of G protein-coupled receptors in in vitro transfected cell systems are applicable to similar events that occur in the mammalian central nervous system in vivo.

Mirror neurons in the tree of life, development and evolution of sensorimotor matching responses

Mirror neurons in the tree of life rappresenta lo sviluppo e l' evoluzione del sistema dei neuroni specchio nei primati umani, non - umani e di alcune specie di uccelli, utilizzando metodi cooptati dalla filosofia della biologia e la biologia teorica, per integrare dati relativi al sistema nervoso e al comportamento delle specie in esame.

Multimodal characterisation of sensorimotor oscillations

The studies in this project have investigated the ongoing neuronal network oscillatory activity found in the sensorimotor cortex using two modalities: magnetoencephalography (MEG) and in vitro slice recordings. The results have established that ongoing sensorimotor oscillations span the mu and beta frequency region both in vitro and in MEG recordings, with distinct frequency profiles for each recorded laminae in vitro, while MI and SI show less difference in humans. In addition, these studies show that connections between MI and SI modulate the ongoing neuronal network activity in these areas. The stimulation studies indicate that specific frequencies of stimulation affect the ongoing activity in the sensorimotor cortex. The continuous theta burst stimulation (cTBS) study demonstrates that cTBS predominantly enhances the power of the local ongoing activity. The stimulation studies in this project show limited comparison between modalities, which is informative of the role of connectivity in these effects. However, independently these studies provide novel information on the mechanisms on sensorimotor oscillatory interaction. The pharmacological studies reveal that GABAergic modulation with zolpidem changes the neuronal oscillatory network activity in both healthy and pathological MI. Zolpidem enhances the power of ongoing oscillatory activity in both sensorimotor laminae and in healthy subjects. In contrast, zolpidem attenuates the “abnormal” beta oscillatory activity in the affected hemisphere in Parkinsonian patients, while restoring the hemispheric beta power ratio and frequency variability and thereby improving motor symptomatology. Finally we show that independent signals from MI laminae can be integrated in silico to resemble the aggregate MEG MI oscillatory signals. This highlights the usefulness of combining these two methods when elucidating neuronal network oscillations in the sensorimotor cortex and any interventions.

Shared action spaces:a basis function framework for social re-calibration of sensorimotor representations supporting joint action

The article explores the possibilities of formalizing and explaining the mechanisms that support spatial and social perspective alignment sustained over the duration of a social interaction. The basic proposed principle is that in social contexts the mechanisms for sensorimotor transformations and multisensory integration (learn to) incorporate information relative to the other actor(s), similar to the "re-calibration" of visual receptive fields in response to repeated tool use. This process aligns or merges the co-actors' spatial representations and creates a "Shared Action Space" (SAS) supporting key computations of social interactions and joint actions; for example, the remapping between the coordinate systems and frames of reference of the co-actors, including perspective taking, the sensorimotor transformations required for lifting jointly an object, and the predictions of the sensory effects of such joint action. The social re-calibration is proposed to be based on common basis function maps (BFMs) and could constitute an optimal solution to sensorimotor transformation and multisensory integration in joint action or more in general social interaction contexts. However, certain situations such as discrepant postural and viewpoint alignment and associated differences in perspectives between the co-actors could constrain the process quite differently. We discuss how alignment is achieved in the first place, and how it is maintained over time, providing a taxonomy of various forms and mechanisms of space alignment and overlap based, for instance, on automaticity vs. control of the transformations between the two agents. Finally, we discuss the link between low-level mechanisms for the sharing of space and high-level mechanisms for the sharing of cognitive representations. © 2013 Pezzulo, Iodice, Ferraina and Kessler.

The embodied nature of spatial perspective taking:embodied transformation versus sensorimotor interference

Humans are able to mentally adopt the spatial perspective of others and understand the world from their point of view. We propose that spatial perspective taking (SPT) could have developed from the physical alignment of perspectives. This would support the notion that others have put forward claiming that SPT is an embodied cognitive process. We investigated this issue by contrasting several accounts in terms of the assumed processes and the nature of the embodiment. In a series of four experiments we found substantial evidence that the transformations during SPT comprise large parts of the body schema, which we did not observe for object rotation. We further conclude that the embodiment of SPT is best conceptualised as the self-initiated emulation of a body movement, supporting the notion of endogenous motoric embodiment. Overall our results are much more in agreement with an ‘embodied’ transformation account than with the notion of sensorimotor interference. Finally we discuss our findings in terms of SPT as a possible evolutionary stepping stone towards more complex alignments of socio-cognitive perspectives.

Premonitory urges and sensorimotor processing in Tourette syndrome

Most patients with Tourette syndrome report characteristic sensory experiences (premonitory urges) associated with the expression of tic symptoms. Despite the central role of these experiences to the clinical phenomenology of Tourette syndrome, little is known about their underlying brain processes. In the present article we present the results of a systematic literature review of the published studies addressing the pathophysiological mechanisms of premonitory urges. We identified some preliminary evidence for specific alterations in sensorimotor processing at both cortical and subcortical levels. A better insight into the brain correlates of premonitory urges could lead to the identification of new targets to treat the sensory initiators of tics in patients with Tourette syndrome. © 2013 - IOS Press and the authors. All rights reserved.